Four Gorosei got de-transformed and sent packing to Marijoa by an Haki blast comparable to Shanks'. Meanwhile, Aramaki took Shanks' Haki solo and made it clear it just wasn't the time to fight Shanks AND his entire crew... but sure, the Gorosei can have it but not Aramaki lol.