Man, no one cares about the Holy Knights. Oda really did wait too long to introduce them (or rather he just didn't come up with until 100 chapters ago).
Even if Shamrock stays, I can't see the 4 of them being a threat. It feels like they are just here to hype Loki when he is freed.