Is there any reason why Sanji is fighting Brogy and not Dorry for the vast majority? I voted for Brogy just because he has already been wounded by Imu 
Am I missing something?

Am I missing something?

Sanji's opponents are usually goofier.
Plus, Dorry uses a big ass sword, while Brogy an axe.
But it can be also a 2 vs 2, so the specific opponent doesn't matter.
