Rayleigh at worst is ~ Shanks. Shanks is literally just a modern day Rayleigh, they have the exact same makeup as a fighter, and Shanks even dresses like Rayleigh, he's most definitely a Rayleigh wannabe.
The only reason Rayleigh is being downplayed these days is because he is tied to Zoro, and the last flashback didn't focus on him. The reality is Old Rayleigh has looked better than both Old Garp and Old Whitebeard. Rayleigh is an absolute monster who at his worst is on par with any Yonkou.