I wouldn't be very skilled with a sword imo
And some swords are dangerous to use and take too much effort, i.e Enma and on a lesser note Shusui.
Overall they're just weapons that regular human fodders can't use or use at their full potential, while Devil Fruits are far different.
I'd take a Devil Fruit over a sword, typically, unlike it's some dog shit like Chopper's fruit ofc