I was looking at this wondering why a friend of mine found and liked a comment of mine from december of 2022 then I saw that someone revived this and it made sense.
This is a crazy crack theory but what if he defeated/killed Kidd because he and Law defeated Big Mom who had heavy ties to the underworld?
She's also the only Emperor he doesn't really have beef with. Meanwhile he clashed with Whitebeard and Kaidou, and was scarred by BB.