You are using circular logic. You say the Yonko wouldn´t fight Garp because Xebec will fight them instead, and when I ask why, you say its because Xebec fought Garp....
It makes no sense. The fact is, even when Shakky was saved, the Yonko didn´t go back for a fight, so they simply ran away....