Idk because both used Haki skills in their battles.
Maybe Harald is physically more powerful but I wouldn't exclude that Xebec can also physically compete with Harald considering his sword skills are quite powerful.
Yes both used Haki skills. But not at same degree.
And no Xebec cannot " physically " compete with Harald. Yes these people are superhuman but Harald/Loki just built different.
One jumped off Redline and another was thrown down Sun world to Underworld as a baby.
Xebec without Haki is not surviving that.
They are just built different,
Hordy bite on Luffy shoulder when Luffy was defending with haki, gave an example on how Stronger force can bypass Haki defense, " Stronger " haki is not neccesarily the case.
So even if Harald Haki is weaker than Xebec, harald Physical strength balanced it out.
Also Xebec was using conqueror as a shield to cancel out attacks.
That is not the case with harald. He takes everything on like a tank.