He was probably shocked to see a swordsman of this caliber here.
I doubt it was fear, but rather surprise like Mihawk showed when Zoro used 3000 worlds or Kaido showed when Luffy revived.
Either way Zoros presence was not what he expected since all he was doing was killing a non combatant