But if the world grows with Luffy, that make even less sense that he would be pirate king in his twenties, if others are growing along with him. If the two year Timeskip only made the Strawhats stronger than him doing everything in the next four years could make sense. But with others growing stronger and after it too him doing everything so young doesn't work. Roger and the others in their Era were in their forties and fifties. And had twenty plus years more experience.
When you have someone like Blackbeard who is inching his way to the Top, at a rate of just as fast, if not faster than Luffy, then it becomes abundantly clear that the Straw Hats can not afford to be gone for a decade, or even half a decade. Oda would have to put the entire world on pause, and that is just not feasible.