He is not some newbie guy that has to progress. He is a pirate for like 4 decades. If he was some young newcomer pirate at the start of the story i would admit thats its ok for him to progress as you said, but this guy was in WB's ship since forever and probably has fought 10x more fights than Luffy. For him to suddenly being a mid fighter for 40 years an suddenly becoming stronger in the TS because reasons doesnt make sense.