Fair.
When one of your underlings could have walked in Whole Cake island when it was still under Big Mom, solo everyone there, and then walk out. Or like you said, capture someone like Garp the Fist.
That bastard Teach must feel reaaaaaally confident