Let's not take things too far. From what I can tell, Gaban clowned pre-G4 Luffy. I presume he did not use CoC either. Even most YC1s would dominate pre-G4 Luffy when he's not using CoC, so someone like Gaban should be able to handle him fairly easily.
Not to take anything from Gaban, he's got a fair share of hype from the fact that Luffy felt the need to go G5 directly after competing with him, and Zoro stepped in to save Luffy, which he usually doesn't do. That's decent enough hype for an old-timer. This isn't a negative for Luffy.