Tbh i dont even see the ID ones betraying him. They practically owe him their lives and Shiryu essentially joins him out of respect
The only questionable one is Kuzan, but even then Im not 100% sure
I think Kuzan actually likes Teach and a lot of the Crew.
IMO, I think Kuzan is even willing to let Teach get Pluton, because Kuzan, after Ohara, Lulusia, and Egghead, should have enough brains to know how the WG will use Pluton: to blow the crap out of islands/organizations that fight back.
If Kuzan gives the WG Pluton, he's effectively dooming the modern era.
Teach, on the other hand, isn't a mindless murderer. Like, look at Drum Island compared to Zunesha or an island Linlin attacks: Teach is much less destructive than most Pirates.
Teach wants to rule the world, not blow it up. He wants to use Pluton against the WG.
The One Piece, tho? I think that's where Kuzan draws the line, and that's his actual mission from Sword: to find and secure the One Piece.
I think after Marineford, the top Marines discussed the New Era, and realized they needed an actual force going after One Piece. I think Akainu and Kuzan's fight was over who would go after Teach, and that Kuzan actually won.