Gaban is most likely the gurdian of the final Road Poneglyph. He wasn't sure whether he should give it to the Strawhats, when he only new about Luffy from Shanks. Now that he has met them, he made up his mind. That is why he told them to come to him before departing. Strawhats often depart with gifts, or atleast get them offered (like the gold in Skypiea).
Problem is Gaban made it conditional. They should only look for him if Elbaf is still save then. The island is right now under attack and Gaban himself questioned whether Elbaf was still safe.
Gaban made easy work out of the nightmare creatures though.
He doesn't know yet, that his son is about to be kidnapped
But Jinbe and Brook are with him, they might be able to repell Gunko.
The Holy Knights are not that strong. At best, they steal some children. So perhaps Oda wants to prepare us for an even bigger threat to arrive, with those Gaban thoughts. I think Blackbeard will show up. Shanks aswell. They will finally have their fight. Gaban will hide, maybe leave on his black ship, just in case. The crew is not heading straight to Laughtale. Something else will come up. They won't be able to fight alongside the Redhair Pirates. Sakazuki's plan wil finally be set in motion: Barto's execution.

Problem is Gaban made it conditional. They should only look for him if Elbaf is still save then. The island is right now under attack and Gaban himself questioned whether Elbaf was still safe.

Gaban made easy work out of the nightmare creatures though.

He doesn't know yet, that his son is about to be kidnapped

But Jinbe and Brook are with him, they might be able to repell Gunko.
The Holy Knights are not that strong. At best, they steal some children. So perhaps Oda wants to prepare us for an even bigger threat to arrive, with those Gaban thoughts. I think Blackbeard will show up. Shanks aswell. They will finally have their fight. Gaban will hide, maybe leave on his black ship, just in case. The crew is not heading straight to Laughtale. Something else will come up. They won't be able to fight alongside the Redhair Pirates. Sakazuki's plan wil finally be set in motion: Barto's execution.