Who could've imagine that Admirals are so incompetent in the face of a Yonko the Gorosei had to personally come to clean up after Lizaru's mess who is taking a rest while Sakazuki keeps shuffling through paperwork?
No wonder they waited until Yonko were out of their territory to do something.