ngl I do wanna vouch for Dorry and Broggy a bit here as what happened on little garden doesn't exactly imply they're weak just that Mr. 3 outsmarted them and planned well with his devil fruit
Snuck bombs in one's drink to damage them horribly on the inside and use the emotional turmoil after the other wins the fight to capture him
Not to mention the fellla got the drop on Zoro as well