I think the best thing about Doffy's character is he still maintains his CD entitlement from his Mariejois days while operating in the lower world. He was trained to think of humans as scum since an infant, but his "family" was all normal humans. It's this contradiction that makes his character.
While he felt betrayed by his biological family, the regular humans are the people that didn't betray him and took him in. Not just humans, but a motley crew of social outcasts like Vergo, Trebol, Diamante, etc. People that CDs would consider "trash". That's why Law betraying him, stung him a lot. He saw himself in Law as a smart, young kid who was left all alone. Law was his future right hand, and his eventual successor. And when he abandoned him he felt that pain that his biological family left him with.
Anyhow, that character has so many layers you can write entire essays on him. That's just not what you can say for most OP villains. I think he simply benefited from the fact that Oda introduced him so early and could add these layers.