Yeah if we forget that kaido and king went first to marineford and face the whole Whitebeard pirates. Doesn't make sense if they retreat against RHP because of them.
Shanks gave something to kaido that's why the latter went away.
It's an interesting theory, but I can't imagine what Shanks could have given to him.
Imho, Kaido wanted to challenge WB, but Shanks interfered and Kaido himself didn't want to fight against Shanks ad his entire crew (with WB he was willing to take the risk, maybe because he admired him and wanted to kill him or die trying).
But we will see. If Shanks gave him something, It'll be revealed one way or another.