No, because as far as we know in the 800 years bar Ryuma who did build Wano's reputation nobody emerged as powerful as the current individuals.
Most importantly nobody was around carrying the D's will at that point in time.
So we agree that the Yonko (both in individual and military power) and other power structures are not something the WG can just casually erase. Hence they allowed Roger to conquer the Grand Line and start the age of piracy in the first place.
If Imu and the WG were so supreme, they wouldn't need the Yonko to keep the balance at the cost of having powerful pirates potentially turning against themselves.
- Destroy the Yonko
- Occupy the new world
- Have fodder pirates deal among themselves and being destroyed by the WG if they try to turn against the WG.
The WG rules without restraint. The problem though: They only do so with fodders.
Again, I do believe that Imu is extremely powerful. But he ain't just walking around and erasing everyone, even if he wished.