A person holding a sword is not always swordsman btw. Just someone who is skilled with it.
For example, Luffy is not a swordsman.
Kaido is not a swordsman or a gunner.
This is a pretty bad comparison.
Luffy isn't designed with a sword. He isn't a swordsman.
Luffytaro is pretending to be a samurai, hence why he has a sword.
Is Luffytaro (meant to be) a swordsman? Yes.
But we all know Luffytaro is simply just Luffy, who isn't actually a swordsman.
Kaido isn't designed with a sword, he's designed with a Kanabo.
Luffy dressed up as a swordsman and Kaido used a sword in a very particular instance.
These aren't good examples to bring up for a "swordsman" debate.
Roger, Shanks, Law, and Fujitora are all designed with a sword and all fight with a sword.
They're swordsmen.
Another common "counter-example" I see is Rayleigh.
Some consider Rayleigh to not be a swordsman because he's used haki with his hands.
Is Hygoro no longer a swordsman then? I would imagine very few people think of him as a "hakiman".