I suspect this is true, he doesn't get stronger, only faster and have more recipes from the Okama. He only went from mid veteran level to top veteran level during 2 years.
The gentleman/pervert Duo Sanji and Brook are the only ones that have unfavorable timeskip training. Sad.