First, that description was by the WG to picture them as bad and evil. Sengoku will not be saying "oh, they were good pirates but we eliminated them" LOL, Imu-sama would babyshake him (as KOL would say)
Secondly, WB and Kaido only implied that it's too much to handle for the captain to have people like that. WB never said that we killed each others. He just implied that their ego got the crew into their demise. And that's not speaking anything about Xebec. It just means he was screwed over because some of his crew members were too egotistical while he had his hands full with Garp and Roger to begin with.
Again, the only people describing them as evil is coming from WG side, which is not trusted when it comes to topic regarding Void century