No, that's not the case. What's likely happening is that even Haki CAN counter the power of the Gorosei. Saturn used his power that allowed him to make the other Gorosei teleport during the Egghead Arc.
So what else can Haki do?
Can Haki rebuild Oden castle? I mean why not? Oden castle was destroyed by a Devil Fruit, right?
Can Haki revive people from the dead? Take Ace for example, he was killed by a Devil fruit. Will EOS Twika take a Haki shit and bring Ace back to life? Think really hard before you definitively tell me that Oda won’t use Haki to revive Ace from the dead.
Literally the fucking absurdity of Haki being able to undo a summoning of this nature is the AI fucking this manga beyond repair. Watch, Haki shitting will become the new Ocular Jutsu. You can shit out whatever the fuck with Haki now.
Oda is rolling in his grave.