Nah. Roger was the Pirate King, and only two people could match him at that level: Garp, and Whitebeard. Like Kaido said, Haki is what matters most.
Peak Kaido is a drunk who openly abuses his daughter and enslaves people because he wants to die by creating a giant world war.
Peak Whitebeard is a drunk who takes in children, no matter who their parents are, and protects them with everything he has, while providing protection to other territories, like Fishman Island.
Roger and Whitebeard knew what they wanted, and fought for it. Kaido gave up, he went from, "I'm the only person who can change the world!" to "You couldn't be Joy Boy, either."
Prime Kaido can't beat Roger or Whitebeard. Kaido is a beast, and the most powerful creature in the modern era, but he's not on Pirate King level, and Oda has made that VERY clear through the story. Yonkou is a title for people with enough power to dominate the New World, but not enough power/desire to become Pirate King.
A Kaido that fought with the belief/courage that Luffy/Roger has would be a whole other problem, but, prime Kaido never challenged the world like they did, so...