It was selflessness, sanji, Nami brook, even jimbei, they all knew luffy had no chance to beat all the big mom porzhes by themselves, so sanji at that point in time didn't know about bege and jimbei's plan, he tried to push luffy away so luffy wouldn't be in danger
It was literally stated twice during whole cake
If he said "I'm going to take care of family issues" luffy would've wanted to help him, that's not what sanji wanted since he wanted luffy to be safe
He didn't willingly put them in danger, he made a deal with big mom, she lied, he thought they were away from danger, he didn't leave him, he thought he was protecting them, it's not his fault
Also you and I have very different definitions of selfishness, for me going against your best interests for someone's else sake is pure selflessness