This isn't Crocodile's First Appearance,
We've seen him in Arabasta Saga, He is Very Smart & Calculating, He doesn't get his hands dirty and He can Operate from the Shadows.
(Even Gorosei didn't know about Baroque Works, and Revos who were looking for Robin, couldn't find her at all while She was with Crocodile)
So if He wants anything done, He can do it without a single faction discovering anything about it,
All while He is chilling in his Casino with a Cigar,
Crocodile won't bother going to Impel Down, nor can He order Mihawk to do so,
Instead, He can simply put "Secret" Bounties on Impel Down Staff or anyone who Opens the Cells/Free Prisoners ... etc
And use Underworld to give orders to any Spies who were ever put there (Just like He had Spies in Arabasta Army)
Crocodile was Free for 2 Years, He already planned everything
So it's gonna be an Inside Job, only problem is Magellan
But They might find a way to make him leave his post or time their escape with when WG tries to assassinate Doffy
Either way, Crocodile & Mihawk don't need to leave their Island
Their Operations might have already started