Why, though? Only they are the legendary leaders of the Giant Pirates. The crew died when they left.
Being legendary doesnt necessarily mean strength either. Fisher Tiger died early but Jinbei far surpassed him. Arlong (who I do feel is very under rated) was surpassed by Hody Jones.
Perhaps in those 100 years between Dorry/Brogy & Hajrudin some guys came along that surpassed them? Could even be bitter guys that feel that no matter what they do they will always be in the shadow of those that came before.