Scaling distances based on characters' talking distances is about as inaccurate as it can get. Oda's not gonna make them carry speakers and microphones to talk with those at a distance when he wants them to talk.
This is the distance b/w the Shichibukai and WB who were standing right in the middle of the bay. This doesn't even constitute a quarter of the island.