The Gates of Justice are huge, but they seem to be just 2 huge steel plates. If someone is strong enough, it's possible to inflict enough damage to rip each other apart, drop them and sink them onto the sea (with either a punch/blunt force attack or by gripping them)... an earthquake isn't needed to destroy it.