This is not a point that I made, it is a point that
@Fleet Admiral Lee Hung made. Go argue with him about it.
Why? I don't recall Rayleigh ever being an individual that sought out to be the greatest at anything. His only reason for being a pirate was to follow Roger on his adventures. Once Roger was dead, Rayleigh lost his reason to be an active pirate and so he retired.
Mihawk meanwhile sought to be the pinnacle of swordsmanship and fought so many duels and defeated/killed so many folk as to be unambiguously considered the greatest swordsman alive on a planet where 9/10 people that use a weapon, choose a sword.
Would a bloke that wants to "live peacefully" hunt down a 50 ship armada because he was bored?
Would a bloke that wants to "live peacefully" be a character that laments the lack of decent challengers to his blood combat title?
Would a guy that wants to "live peacefully" train another individual with the open intent on dethroning him of his title purely so that he could get a decent duel once more?
Maybe what Mihawk says will make more sense once the official translation comes out and what he says will actually read more like "Bruh, I can't be arsed running a big ass pirate crew". That would be a much better argument than defending the idea that a dedicated, unrepentant blood knight character that has lamented the lack of worthy fights in his life for the past 1050 chapters suddenly wants to "live peacefully".