As I said in another thread, Vivi might not have her own pirate adventures, but she is not your average Joe. She is a very important figure in the world politics and there comes a certain risk with it. Luffy has to trust that she can overcome most of those dangers by herself as the second most powerful person and the future Queen of Alabasta Kingdom.
He's completely off base by bringing up Ace into the mix. Ace =/= Vivi, there's a reason Luffy treats Ace and Vivi differently, he doesn't see Ace as a damsel in distress Vivi was literally that. In every sense and definition of that phrase.
You are wrong about this too. Pirates live a life full of danger and death can strike at any time. Ace didnt regret his decisions. He died smiling.
The point is it's plainly just odd timing for that example to be made. Yea the crew is worried about Vivi's life, "thanks a lot for reminding me about Ace the guy 100 times stronger than Vivi who died cause he was left alone, you certainly convinced me not to want to go after Vivi now 👍"