Since Wano started, and even during WCI, i've had the sense of Kaido (and Big Mom) not being outright evil. Just like Doflamingo they do have moral compasses albeit twisted and go about their business a bit more gruesome than Straw Hats.
My head canon has been for a while that Kaido either aimed to be Joy Boy and fell short (or he is looking for him) in order to destroy the true evil - the World Government. King fits very well into this theory as someone oppressed by the WG and putting his hopes into Kaido. Perhaps Jack, an oppressed Fishman, is with Kaido for the same reason.