After the fight with Roger, WB showed that he can destroy the whole island
The damage to the surroundings never mattered because Aokiji and Akainu would always fight with others in a place full of people....
So this excuse will always exist that they are not fighting with 100% strength