He said he can attack anything that hears his attacks/music. Zoro, Luffy, and Kid we're all able to stop Apoo attacked by just coming their ears. Apoo has a dangerous ability that can be easily countered and stopped.
Most of the battle he was running from Zoro/Drake. He went on the offensive very little. He was only able to use his music attack was when he could create some distance between him and Drake/Zoro. Since Zoro has once again showed us he has incredible high speed and range attacks it will be very difficult for him to create the distance.