No one hates Gaban. I am as indifferent to him as I am to Rayleigh or Fraudger
Ray had an interesting introduction. He was portrayed as badass swimming through the Calm Belt topless. Then he stopped Kizaru nonchalantly. After that, his scenes started to loss charm.
Roger has been demolished as a character post-timeskip. Everytime Roger reappears I want to close the chapter. Mofo was assassinated through and through.
Gaban feels like a retreading of Ray's character. A white-haired badass old gen who toys around with the arc threats, but ultimately has to acknowledge that he's getting too old for this.
IDK if Oda has any other Roger Pirate in store for the future, but it'd rather he tried some new approach to them, like with Crocus, instead of making them goodie-two-shoes with hax haki who have to be sidelined via family problems because otherwise they'd solo the arc.