First of all, "You didn't call yourself a Swordsman" =/= "You're not a Swordsman"
Second of all, King basically said he doesn't believe in relying on just one fighting method but using other stuff like Fireballs, Magma Flames, Explosions, Zoan Abilities, Sword Breaker trick to disarm, Spikes etc,. to win at any cost.
And that's when Zoro talked about King not calling himself one.