Tbh, this is quite reasonable but let's hope that Aramaki really meant this as he was talking about Kaido's previous connections with the WG. Nevertheless, the Shanks stuff depends on his reaction. If he has a similar reaction like the admirals in the movie, it's definitely an L. But if he was concerned that he'd have to fight the RHP alongside the alliance which would be too much for him, it's definitely not bad to retreat.
Either way, the Momo issue is fortunately wrong and he was calling him a Kaido wannabe.