Roger and Garp were middle aged men, and Gaban compared Imu’s haki to God’s Valley. This means that the all-out haki battle between DR Xebec and Roger/Garp were the highest caliber haki that Gaban has felt next to Imu. So no, Roger barely grew stronger if he even did after God’s Valley.