this might be valid if he was one of the other rulers, with no connections to the 20 families whatsoever. however, we know that the nefertari family was one of the 20 families, but they chose to remain in alabasta instead of going to marijoa, and the five elders considers them traitors for it. the elders also mention that cobra wishes to speak privately to them about something, and the gandhi one says it's time for "a great cleansing". most likely, cobra asked about pluton/the alabasta poneglyph, the gorosei felt threatened and had him killed