Precisely because "only haki can trascend all" Gandhi's weaker than Mihawk, just like Shanks.
A is a stronger swordsman than B. B cannot possibly have stronger haki than A, because then B would defeat A in a swordfight (which involves the use of haki).
So A being a stronger swordsman than B means A is stronger than B in both sword skills and haki.
Give B a Devil Fruit, A is still stronger than B, because he still has the stronger haki.
Conclusion: Gandhi is a swordsman (that much is obvious) and Mihawk is stronger.