How is he supposed to be a brawler with one arm LOL? You need two arms in order to brawl with someone LOL.
They know that he is a swordsman but they don’t wanna admit it. They hate the fact that he is not the WSS.