Sounds like you're their servant. They probably trash-talking you.
Cat 1: have you guys seen that woman over the next block? She
dared tried to pet me!
Cat 2: yes, I have, Boulebard! She tried to pet me too, the absolutely audacity!
Cat 3: I'd rather be chased by a dog for the rest of my life than let a human pet me.
Cat 4: I agree with you, Stepefoup. She does make some good food though...for a human.
*All cats laugh*