For now the pirate alliance took care of the Gifters, Pleasurer and the Waiters, converting them into allies. The next couple of chapters the remaining Numbers will either be beaten down or become allies somehow too.
Only then will the real fights (F6, All Stars, Yonkos) truely begin.