Sanji's infamy is always subject to a joke in regards to how Sanji perceives it. Wether they can't take his picture, or they confuse him with someone else, or he's angry because of what adds to his bounty. That's probably always going to be a thing.
Well we've seen X Drake, Law and Hawkins all be awed by Sanji because of his Germa ties, so his infamy was significant in Wano for most of the relevant WG members. Then we have Queen separate his and Zoro's posters and comment on Sanji because of his Vinsmoke name as one of the priority threats.
Also do remember that in terms of nortoriety among the SH's Oda gave special treatment to Zoro and Sanji. They are the only one's who get to build their notoriety within Wano in the early acts through distinct narrative lines
getting to be known as the Murderer Zorojuro and the Rebel Sangoro
by the whole Capital and through out the country, becoming notable wanted men.