Why would Mihawk go to an Island that was already conquered by someone else? The entire Country fell to Kaido. Do you think that screams "There are countless strong Swordmen here!" Their most renowned Swordsman died, and the rest of the Country didn't have the strength to keep Kaido and his crew from taking over....
Mihawk didn't go to Wano, because there was no need to. He didn't have to go there to know there was nobody there worth fighting.
Him going to Wano would have been bad writing. It would have made him seem incompetent to not know that there were no worthy opponents in a Country that was already completely subdued by an outside force.