Is Z Canon? If you consider Z than you should consider Bullet, someone Who was already equal to Rayleigh before his prime and still wanted to fight Yonkos and Admirals to show he's the strongest according to Oda himself, yeah, fighting Yonkos and Admirals Is needed to prove you're strongest, not fighting Mihawk.