I can't think of anything. We know that against his peers, Shanks goes 1 on 1. He clashed with Whitebeard while everyone left the ship, he probably clashed with Kaido solo, which is the reason Kaido placed Shanks on one pedestal with Roger, WB, Xebec and Oden, we know he acted quick against Kidd and wasted him along with Killer.
Maybe in Loki's case, Shanks' crew really gangbanged him while Shanks sit back, which in Loki's interpretation, who was also fueled by his saltiness of being captured, classified as Shanks being a coward for not fighting himself, but it's hardly an argument against Shanks that his subordinates are strong enough to do it themselves. It just means that Shanks fighting himself would've made it worse for Loki.
Or it could simply be that Loki wanted to provoke Luffy when realizing he is close to Shanks, which was indicated by his expression. He wanted to exploit Luffy's feelings.
Many possibilities here, yet people just want their agenda to be true.