From my experience in the One Piece community, most Sanji haters actually have a lot of his attributes when you get to know them, and they detest those aspects of themselves so when Sanji is on his bullshit it's like Oda is holding up a mirror to them and they react with disgust. Meanwhile Zoro is an ideal of who they want to be like, so they flock to him instead. Sanji fans are usually more detached from his behaviour so they can laugh at it and not have a knee-jerk reaction.
Obviously this is not a stone hard rule for every Sanji hater, for some he's simply cringe, or maybe he reminds them of someone, but as someone who has spent a lot of time getting to know people personally in the OP community, you'd be surprised by how many times I've come across this archetype. You find out the biggest Sanji hater is secretly e-dating someone on discord or paying for OF. All while wearing a Zoro avatar.