Yes when it comes to swordsmanship only, people can still fight without a sword; such as Luffy and Sanji. And Vista was still equal to Mihawk.
Mastering other aspects improves Zoro as a fighter, his position in the crew is being a fighter;
Just like Luffy improves himself with Haki doesn't make him a swordsman, it makes him a better fighter. The main point between fighter and swordsman is skill , just like Mihawk said grace directly related to a strong sword and strong swordsmanship.