With the exception of Joy Boy, and even Joy Boy himself literally went life and death, no one wanted to war with Kaido.
Moriah - annihilated
Doflamingo - made a deal
Hawkins - became subordinate
Apoo - became subordinate
Kidd - annihilated
Killer - annihilated, forced to eat smile
Big Mom - stayed away
Whitebeard - stayed away even abonding his principles, something only BB equalled
Shanks - admittedly stopped Kaido and King but that was most likely whole RHP vs 2 away from Wano
Akainu - forced Kizaru to stand down
Ryukogyu - didn't dare to enter Wano as long as Kaido was there
You can't really blame Doffy for being cautious around Kaido, that ogre was built differently.