Haki isn’t swordsmanship. You can be a swordsman with or without haki
All haki does is boost your fighting style
Again all it took was a single punch from king not his lunarian or devil fruit powers. King was even able to give zoro a sword duel without being a swordsman himself.
Being skilled with a sword doesn’t limit you into being just a swordsman. Same way being good with a gun doesn’t limit you into being just a sniper
Also let’s not pretend like oda hasn’t left room for shanks to be strong even outside of his sword