If he keeps pushing you in error then eventually he's going to be ignored. And if not then the players who sheep him will need to get good. No reason to let it get any more emotional than that. Just defend what you can, if you think you've made yourself townie enough then it's up to us to be able to see that then.
While I agree there is a level of mean-spiritedness that could be present here, for the most part people just want to play Mafia.