this is not a good explanation… gorosei already showed that they prefer to hide things instead of creating mess that may generate disorders all over the world. For example, they didn’t say anything about a pirate freeing alabasta, and they ordered doflamingo to kill moria because he was defeated by luffy, they did not replace him