King didn't exactly deny he is a swordsman; he simply confirmed he doesn't confine himself to one style of fighting i.e., he doesn't mind going beyond the ways of the swordsman to get the desired results. King is a swordsman who doesn't follow a single style. But you can't say the same about someone like Fujitora, for example. He uses the way of the sword even when he uses his fruit, period. Even for Kizaru, fighting with a sword is considered him fighting seriously as the only time he was frustrated in a fight is when he fought with a sword. Let's see how GB fights.
Then he aint a swordsman. If King is a swordsman.....Queen is a swordsman. No ifs or buts