I think Kaido was loyal to Rocks; that's why BM is telling him to not trust him completely. One thing for sure is that they all believed in Rocks regardless of their differences and followed him. I think the most loyal one of them all was WB followed by Kaido. Kaido was an honest guy and follows morals, even now, whenever he can.
WB, Kaido, Big Mom, Shiki, and most other Rocks pirates survived God Valley, meaning they all escaped the battle at one point or another during the fight. I'd say Kaido witnessed this happen to Rocks i.e., his underlings abandoned the battle and started fleeing one after another as soon as they realized they were losing until, eventually, Kaido too was forced to flee the scene—experiencing firsthand how things work with pirates. I believe his experience in God Valley made him more wicked.