How is it baseless? If Luffy can do to Kaido what he did to Doffy he would be on Kaido's level, maybe little bit weaker.
Luffy has no chance of beating DD solo without Law.
Luffy has no chance of beating Kaidou without help.
Would Luffy have a chance against DD without GK seriously hindering DD´s durability and endurance? Nope. Even if you argue that G4 was stronger than DD physically, ultimately Luffy would not have any chance to finish the fight in 20 minutes against a fresh DD with Awakening and then he would be a goner.
And while the circumstances in regards to Kaidou are different, since Kaidou is exposed, the premise is the same. Kaidou needs to be handicapped, so that Luffy can do most of the work. But he would still get shared credit just like with DD.
And Luffy being a high diff fight for Kaidou with AdCoA, FS and so on is not far-fetched.