Even if Imu did destroyed the world or Davy Jones did most of it before him, Joyboy was still able confront Imu if Uranus doesn't kill Joyboy, and they should have clashed before Imu spamming Uranus.
I’m not trying to insist that they never met, but I do think it’s a possibility and it kinda makes me laugh.
In any case, I don’t think Joyboy was defeated and killed as the result of a fight. It feels like he already knew he had lost even before dying. Otherwise, why leave the One Piece at Laugh Tale? Why leave an apology letter to Poseidon and the Haki knot to Emet? Those all feel more like the actions of someone like Roger, a man who knows he can’t do anything anymore and instead sets the stage for a future victory.
Right now, I can’t really picture Joyboy going out with his crew to fight Imu in some heroic last stand like Oden with his samurai. He gives me more Roger and his crew vibes. At some point they realized there was nothing more they could do and eventually disbanded.
The act that truly ended the conflict was most likely the use of Uranus that destroyed the world. I don’t really see how someone like Imu could have defeated Joyboy without relying on an Ancient Weapon.