VP said in this chapter that he doesnt know who was right or not, which means the CDs who are murderers, slave owners and rapists will be on Luffys side when its all set and done.
He says he doesn't know which side of the war was right.
The current CD society has evolved (or devolved) into what it is over 800 years. It most likely no longer reflects the society that started the war in the first place, let alone the ideal they stood for.
VP isn't justifying slavery and genocide. He's saying he doesn't know enough about what brought us to this point in time.
'Cause there is still a good possibility that the CDs' lifestyle is a warped interpretation of Joyboy's "freedom at all costs" motto. Freedom is good, but to this extreme is its opposite. Thus VP can't condemn one but not the other, so he refrains from doing either for the time being.