the yonko themselves are overhyped, so much so that many fans viewed them as untouchable gods.
In reality, they only have individual strength and are victims of situation, just like every other character in the series. What I mean by this is even if Big Mom can neg diff most of the straw hats in a 1v1, she would still face difficulty fighting them altogether.
Similarly, Kaido will probably take serious damage from the scabbards and his victory will be by a much smaller margin than what most people think. Maybe he won't win at all, perhaps the scabbards will actually get the best of him and nearly kill him, with him only surviving due to luck or perhaps big mom or a calamity intervening.
It wouldn't be the first time something like that has happened to kaido.