And what should I do about that?
Personally, in your place, I would be looking at the people who are town-reading me, and trying to determine how valid their reasons seem. Do they stand up to scrutiny?
Unfortunately, I think a large part of the difficulty you will have doing this, is with so much 'meta' reading going on, it is objectively difficult to distinguish "real" town meta reads on you, and "fake" town meta reads given by scum. Because if the read is valid, then it's valid for both sides.
I'm not sure about the rest of the player list, but I might actually be the only player who can't meta-read you.