To be fair, this is less of a strength thing and more of a reputation thing. Gaban and Rayleigh might be equals/near equals in strength, but reputation wise Rayleigh clears him - especially to other pirates.
I doubt Gaban would've been able to bluff as well with "left hand" status