I get some of the implications for Z and S fighting Dorry and Broggy, but honestly, the focus Sommers is getting implies he is fighting Sanji, and the way he treats women is pretty "unique" if u know what i mean, Absalom and Queen were like that as well.
So Sommers is being focused on, while Killingham is gonna be left behind for now just like what we saw with Queen getting all the screentime while King got offscreened.
In regards to Sommers strength, don't mind him jobbing to Gaban and Rayleigh, these guys are worth 3 Billion berry bare minimum, and besides he got caught by surprise in both situations. Look how Sommers talks to Garling and Shamrock like he was in a special position among the Holy Knights, i think that means something..
I think that during GV we gonna see some feats from Sommers, he is not defeated yet just like that female Holy Knight Whitebeard cut in half by accident.
We saw Sommers trading some blows with Gaban and blocking his CoC attacks even while missing a hand, and he still managed to last a for a while. Obviously they have no chances against Top Tiers but i feel like Oda is trying to show us this arc the level of Zoro and Sanji compared to Gaban and Rayleigh's.
And Sommers is gonna be special because Sanji has no Conquerors Haki while Zoro and Luffy already do, so he is gonna get a special focus in this arc due to that in a way or another despite of Sanji getting CoC this arc or not...
Zoro and Sanji, or even Luffy, might eventually be involved in clashes against Dorry and Broggy but i don't expect Zoro and Sanji to fight them and beat them like their main and long lasting battles...
Sommers and Killingham are Imu's direct subbordinates, thus fighting Zoro and Sanji makes the most sense if Luffy is meant to face Imu, it is trully simple as that