Zoro did not say, King is not a swordsman. He said, King did not call himself a swordman. There's a difference between the two. One is not a swordman, the other is a swordsman just not by name.
King is someone who doesn't believe he needs a sword style or to be called a swordsman to fight. He believes that limits his fighting ability.