Another thing to point out in this case Luffy was basically holding Kaido so Momo can move the island away, and he doesn't punch Kaido on the island, so one could make an excuse for this and it makes sense
He was gonna aim the punch towards him anyway but at risk of punching onigashima which is a no with everyone inside there.