Because he is their greatest fear. Their dream of OP becoming Zoro Piece needs Zoro to reach Luffy's level, Sanji is the only character that brings them down on earth since he is obviously weaker than Luffy (like Zoro), everyone accepts it and Sanji and Zoro do have a rivalry-like relation.
If Sanji and Zoro are Luffy's arms, Zoro will always be, at least, second to Luffy and not because he is the one taking a step backwards.
And it hurts so much