That was a much bigger issue in the 90s/2000s. People reading mangas/watching anime were really seen as weirdos, back then. Today, it's much more mainstream and I don't think people really care if they know you like those things.
There's even a countless number of ridiculously attractive men/girls that you don't typically associate with those things, who start youtube channels to react to anime or other geeky stuff and pretend they're genuinely into those things because it's cool to do so. Times have really changed.