Imu doesn't care about people seeing him, he had no problem showing himself to Cobra, it is the Gorosei who prevent that and they weren't even there at the time.
Why did the narrator say "Xebec knew what he couldn't do"? That's literally stating that Xebec would get destroyed by Imu, which is why he ran away.
Imu didn't care about Xebec at all, his reaction was nothing.