In Act 1 and 2 the SHs weren't together and split up..what do you want them to do? They still added stuff where they could whilst still following the story.
-Sanji and Law running things while Luffy and Zoro are away
-Scabbard stuff, make us care about them
-More Zoro and Hiyori
Instead we got Batman top 1 and Sumo fight of the decade