Big Mom thought she could have single-handedly defeated the other 3 Yonko had she taken control of Elbaf's army which is considered to be the strongest in the world. Even the marines are scared of angering them which means that they must have powerful warriors that can hang with the Yonko that we just don't know of because they stay in Elbaf and keep to themselves. It also makes sense because as Luffy is now a Yonko himself there must be other threats out there that can hang with luffy other than Shanks and Black Beard or it would just be boring.