I don't think you're suppose to take it so literally. Besides, this was after he had just lost Ace, so he's feeling empty right now. It's a very human thing to feel you've lost everything after one loss. The point, however, is that Zoro helped to remind Luffy he's never truly alone.