Why are we even debating about this at this point?
The Japanese says: よせと言ったよな?!
which to my best knowledge, means: Will you shut up already?! Or: Stop your babling, alright?
I'm not saying King and Queen did not try to fight Ryokugyu, I believe they did, only this sentence is not much proof, unfortunately.