Honestly, that one part always struck me as absurd. Because even if he says it it doesn't mean it can't possibly happen.
Look at Luffy as an example, he said he wouldnt run from Emperor and Admirals anymore, then he procceeds to run from BM 2 arcs in a row.
I think that just because characters say something, it doesn't necessarily mean that its set in stone.
He could lose to a swordsman but then quickly have a round two and win. Meaning he overral won.