You obviously need to take into account how the story's scope has went way beyond what Oda had in mind back in Little Garden but if you are willing to accept that, the Giants' portrayal was legitimately great when put into context.
They were introduced with bounties of 100M, the highest we knew at the time and higher than Crocodile's frozen bounty. The warlords were already established as a big deal by then and these guys just pulled up with a bounty superior to the saga's main villain which was already nearly 3x Luffy's own. The intent behind that obviously was for the readers to take them seriously.
Mr. 3 never actually beat them. He outright admits that he would have no chance in a face to face fight and devises a whole roundabout plan to capture them instead. Brogy was restrained after injuring and exhausting himself in two back to back duels with Dorry. Manga goes out of its way to point out that he wasn't fresh anymore heading into the second duel.
Mr. 3 still has to take him by surprise just after the second fight and while he is grieving the loss of his friend to restrain him.
Dorry himself was done in due to going into the second duel with his insides blown up by explosives on top of his previous injuries. He couldn't even withstand the impact of his own attacks at that point as seen in his short fight with Luffy, a fight that, mind you, he actually won either way.
That's the guy who somehow managed a win against Crocodile 2 arcs later being made quick work of by Dorry on death's door.
Anyway the point is that these guys were always meant to be a big deal. Reading Little Garden with the context and scope of the time makes it pretty evident imo. They were simply hit pretty badly by how long the manga would go on for, exactly like Crocodile in fact...