Gaban referenced God’s Valley because it was still the most impressive fight haki-wise he ever felt. He had another decade to sail with Roger, and they clearly never encountered that level of combat ever again. The only time that Gaban said another extreme PL statement is when he said that Harald was unstoppable to anybody.
Also, Xebec being weaker than Roger will never make sense. He was at least equal to whatever “peak Roger” is before his death.