Kaido has been defeated seven times. I couldn't care less about what "people say".
Rayleigh's sword says nothing, it only says that you're assuming he won't spam blasts of advanced armament without getting to cut anything, for example. Again, that Oden cut Kaido sounds way less impressive the moment you remember that advanced armament frees you from the trouble of his skin to begin with because you directly reach the inners. It is like Lucci making a big deal of cutting someone with hard skin using rankyaku even though he may just resort to rakuogan and don't waste time with it. And again, the power of destroying someone from the inside was something that not even Hyogoro, the guy talking about breath of things as haki, could teach Luffy. Right now, as far as we know, such level of armament isn't required to master swordsmanship (only canon definition of this so far: cutting anything but nothing), so Oda could very well portray black swords as something that doesn't need it.