Duration of duel has no correlation to whether something was extreme-diff or lower. You can have a short fight that's extreme-diff ending within hour, or something lasts long with the one fighter still capable of going for more while the other not. Simply depends on the abilities of the characters at play. Just as you can have a 3 round boxing match at extreme-diff, or a 10-round boxing match with one dude still capable of going for multiple more rounds.
He specifically states he's received countless scars:
He then clarifies that the ones on his face aren't specifically from an adventure or from his fight with Mihawk. Meaning there are other scars he possesses from his fight with Mihawk and his various adventures.
Kuzan isn't pushing Saka to extreme-diff.