If that's how Black Holes were created, then you'd have a valid point. Unfortunately for you, it's not, and the Gravity fruit can't create black holes.
Black holes are created when matter is squeezed into a space so tight the gravity becomes so great, that nothing can escape it. That's what the Darkness fruit does. It compresses the space around it, which causes the gravity to become so intense, it pulls people in. The fruit itself, however does not control Gravity.