Instead of the "admiral lvl bodyguards", Saturn took Kizaru with him lol.
On the other hand Rocks crew is not as impressive as it seems like (Kaido an apprentice, BigMom far from her Prime which was probably when she got fat again, WB still 2 years away from his Prime and in 2 years character can grow as much as author wants, Shiki's lvl unknown but probably far from his peak as well since he fought Roger 24 years ago and God Valley was 38 years ago). So Garp + Roger pirates defeating them makes sense.
Ofc now that we know about this group, it's worth considering that they were there. But why would they team up with worthless "people"? At most I'd say they were keeping CD safe but didn't join the battle, more like a third party observing and only acting when some idiot pointed swords at their masters (or "kind").
Still, when Sengoku said this, Holy Knights didn't even exist.