He's barely a character yet, so any other reason you think you have is largely superficial lol. He's still a Roger pirate, and still is someone that can be used to assist the crew, namely give info on the 4th Road Poneglyph, or tease Lodestar/Laugh tale in a way other members havent.
We’ve seen the Roger Pirates interact, and we’ve seen Roger’s goodbye to them. Shanks got special treatment with his mysterious crying after Laugh Tale, Ray got special treatment by being the one Roger personally spoke to before leaving. We also saw throughout that Ray is the one Roger is closest to, the only one who doesn’t call him captain, his right hand man etc.
While Roger and Gaban barely spoke to each other and Gaban really got nothing other than a quick mention that he’s strong.
Oda gave nothing to make us think that Gaban is somehow taken it upon himself to be some last guardian of Laugh Tale and the Poneglyphs. Why would he be doing that when Ray, who was far closer to Roger and knew him better, didn’t feel the need?
Elbaf doesn’t need another Roger Pirate, it‘s already got Shanks linked with it, Saul, all the Ohara research and the long-lived giants with their knowledge of Nika.