It was mentioned multiple times in the WCI Arc, how didn't you understand something like that?
The fact that Judge said why he didn't pick up a sword, because this is what he trained him in, shows that he wasn't bad at it.
And you are out here screaming headcanon. You are the one making BS up without any proof. You try to justify your own headcanon with ''SaNjI wAs CaLlEd A fAiLuRe'' - which has nothing to do with him being bad at things, but simply him not being able to keep up with his brothers. What is there not to understand?
This discussion really reminds me of the analogy discussion...how can you not understand that?