That's precisely Aokiji's character in a nutshell.
A person is weak when they don't follow their will.
As an admiral, Aokiji was weaker than his full potential because he was following orders (= someone else's will) rather than his own. Now that he's a free agent (*cough* *cough* infiltrating the BBPs *cough*), his spirit can truly show through his attacks. That's why he appears nerfed against Garp: because Aokiji didn't want that fight, he wasn't at his best.
Kiz is content being a soldier. Broccoli is the same. I don't expect them to be powerhouses while working for the WG.
Fuji is following his will, hence he was shown more competent than the aforementioned two, and Akainu is bound to rebel to the Gorosei and go full berserk intent on making his own justice in the world.
Kiz & GB are the weakest by design. One is a lazy monkey; the other is a plant who takes what the environment gives him, and uses that to nurish himself, without chasing more, without seeking the best.