I think Roger and Shanks are swordsmen but they are different than Mihawk/Ryuma
The only difference is that Mihawk and maybe Ryuma had the sword and swordsmanship as their way of life and their "dream", while for Roger and Ahanks seems to be more a weapon.
At least from depiction, Mihawk seemed interested in being the strongest swordsman and Shanks wasn't, but at the same time this doesn't mena Shanks is not.
Is like Brook and Pinzoro.
Pinzoro lives by the sword, Brook doesn't. But both are swordsmen.