Factually untrue. Luffy was completely fine with the scabbards having a hack at Kaidou. They were simply failing/failed. Zoro didn't even take getting upstairs seriously until he saw Kiku's arm.
Luffy, Zoro, Sanji are also much much much stronger now than they were at the beginning of the raid. They were most likely letting Momo, Scabbards, Yamato try and handle it out of respect, but monitoring the situation, they'd be ready to back them up no doubt.
Luffy was rushing, climbing stairs. Along with Jinbe, and Sanji. Zoro was rushing too to Rooftop too. Only back then they were interrupted. Luffy and Zoro were rushing but then bite by Queen and slammed down. Sanji was distracted by Maria, Jinbe by WSW, but ultimately they were still in hurry to the Rooftop.
Here against GB, same 4 characters are watching from afar, sitting comfortably. They don't rush.
While it maybe true that they could interfere if things went worse, that didn't happened. Because Shanks scared tf outta GB from picking the new leaf, just like he scared tf outta sea king from picking new leaf Luffy.
I thought CoC was made to scare animals and fodders, never thought it was made to scare Admirals.