
So my followers are called "Erkanite".
1) Disciple of Erkan: Mihawk's title is World's Greatest Swordsman meaning he is the most skilled wielder of a blade, not the strongest. Anyone that uses a blade in combination with anything else like devil fruits (e.g. Law and Linlin), demonic shenanigans (Imu) or even just advanced Haki (e.g. Shanks, feckin somehow) don't count as swordsmen. Sword wielders and swordsmen aren't the same and Oda made a distinct difference if you autistically micro analyse words while ignoring context. All consistent followers of this train of logic will downplay Mihawk in general and will obviously consider Imu to be superior to him.
.
That's simply how Oda sees things, does it make sense? Perhaps not, perhaps does, but it's his manga so that's what he says. It shouldn't have been difficult to add this at least "master swordsman" title to Xebec, Roger, Shanks in their info boxes;
Master swordsman, Pirate hunter.
Oda could easily say; "master swordsman; great pirate Roger", Or "master swordsman; pirate captain Shanks." But he doesn't do that.
How Oda sees it;
Being a swordsman in One Piece: sticking to traditional swordsmanship forms and techniques only. Otherwise as Zolo said; they are "blood thirsty brutes".