The only reason people underestimate the likes of Fuji and Ryo, is that they expected these characters to bully a bunch of non-haki supernova fodders like back in the days.
When Aokiji appeared, the SH's didn't even know haki (zoro had the "voice of all things" that was only later retconned). They didn't even have their EL PU's.
When Kizaru appears, they still don't know haki, and it's said that CoO is necessary to keep up with Kizaru's overwhelming speed.
Akainu is in the middle of a war with a bunch of characters that could easily fuck up all the SH crew.
Even WB less known allies could shit on most of the SH's by just using basic CoA at the time.
Then they expect Fuji and GB to appear and to the same things against characters that are waaaaaaaaaaaaay stronger than the SH's pre-TS ever were.
Perspective is key.