Luffy probably had the same reaction with Aramaki. That's why he had himself and his 3 boys ready to go jump him.
Yamato wanted to call the crew to jump him too. Even Shanks had to threaten him.
If it was some weak rando, Shanks probably wouldn't have felt the need to stop him.
Admiral stocks are about to go up, might as well buy some shares.