Imagine making the same mistake repeatedly after the last arc proved you wrong.
If Kizaru is here and is the strongest antagonist, he's Luffys fight. That's how it works. That's how it worked for Wano while half this forum whined for ZKK.
Zoro has Fujitora. In fact Zoro clashed with Fujitora, which is more than can be said about any other Admiral. There's nothing specifying they won't fight.