The NW is completely divided into yonko territories. Mihawk traveled freely between those for years. Shown to us very very early by Oda. Also, he travels in a nutshell. An unmotorized vessel with zero ability to flee if someone assaults him. Add 1+1. The only explanations are a) no one ever attacked him for years b) he wrecked everyone that attacked him. Both are probably true.
The thing is, Mihawk doesn't actually need to do anything. He simply is the most powerful being in the verse, remember Rocks was a swordsman, Shanks is a swordsman and inferior to him. People accept him like they accept gravitation.
How many times does Oda need to state it on panel?