For the same reason Gaban isn't a swordsman. Just that Daz Bones has even more versatile weaponry, turning his entire body into blades and saws.
The point is: Harald can ditch his sword for an axe, but does that make him stronger? No. He would be at most as strong as with his sword, if not arguably weaker, given Harald most of the time fought with a sword, which is his primary fighting style, with Harald even calling his sword "famous" and only ditching it because he rammed it into Jarul's head.
If Gaban beat Mihawk -> doesn't disqualify Mihawk from the WSS title, because Gaban is specialized on fighting with his axes, even though Gaban would probably replicate 80-90% of his power and abilities with swords.
But if Zoro beats Mihawk wielding axes, it definitely means that Zoro is the stronger swordsman, because using axes Zoro would only mimic his swordsmanship.