That's why I mentioned great influence.
Tbh, I doubt it would even be mentioned if this Warlord was some scrub. Why giving this W to Ace then instead of some weaker character? Ace wouldn't benefit defeating a weakling after all.
You don't just become Warlord by being weak.
Either this former Warlord had great influence (like Buggy), took this position by other means or he was simply strong enough to get this position. Plus, I think this former Shichibukai sure will appear again. There's no way Oda's just dropping some...