Luffy is the man who said he'd come beat her at the beginning of the New world chapter, and reiterated his intentions in Punk Hazard and Dressrosa. When she called him out on not even throwing a punch, he went and did just that. The idea of sneaking off wasn't his own, here's how he actually reacted to the suggestion
He specifically has stated that accomplishing his goal is beating them.
He declined to leave WCI without beating Katakuri, which is what helped him earn the fifth emperor ephitet where others beating less than her top man did not.
His peer, Kid, has also specifically stated his intentions on challenging Shanks and Kaido even if he left her territory without facing her.