As I understand it from these panels is that the Lineage Factor of Sanji is in fact modified like his brothers, however his modifications (Inborn superhuman strength, Exoskeleton, Some super power like his siblings) didn't manifest thanks to Sora's drug, that's why he's considered as a normal human being.