Meteorites are the most useless ability for fights, it is specifically designed for collateral damage. Doffy and Law cut them. Sabo isn't affected by them since he is logia. The rest can just avoid the trajectory. As for the rest, why would Fujitora hold back? Sabo's fruit causes enormous destruction, so it would make sense for Fujitora and Greenbull to counter their DC with their own DC. Or else Sabo would burn the whole castle to ashes with things like Entei.