It's a fact. Kaido dedicated 28 years of his life to selling Seastone to the WG. He never claimed Pluton, went to Elbaf(country of warriors), or even got to fight in the Marineford war because he was running from Shanks. Even after Whitebeard died, Kaido was a sitting duck while Bm/Blackbeard pirates claimed new territories. New Yonko like Shanks and Blackbeard were able to gain territories without every worrying about Kaido ever. When we consider the fact that Kaido didn't even ask Oden about the One Piece, its clear what happened. He retired from piracy to sell seastones to the WG.