The Scabbards narrative against Kaido has been constructed since the Punk Hazard. Imagine when it's time to finally face Kaido, Oda makes the Scabbards take 1 Hit KO from Kaido, so It is obvious that Kaido took more Screentime.
Also, the Scabbards weren't even the focus of that fight against GB, but Momo's.
Honestly, I need to see GB's full potential to affirm GB>Kaido or Kaido>GB.