As we know, Luffy has a dream that Shanks, Ace, Sabo, and the SHs know of, but we the audience don't. This dream is related to, but is separate from Luffy's dream to become Pirate King.
imo, this dream has to be related to Luffy's mom. I mean think about it.
Luffy's dream is the same as Roger's dream, and Roger's dream is the same as the dream of an unspecified "great man" in Oden's journal who more than likely is Joyboy.
The only way Luffy, Roger, and probably Joyboy can have the same dream is if this dream is the Will of D, passed down from generation to generation amongst members of the D clan. This isn't to say that Luffy perfectly understands the Will of D, or even knows what D means, but his dream contains the essence of the Will of D.
Anyways, this is something imo that Luffy could have only gained from his mom. Why? I feel like she's the only possible D that Luffy could have had a close relationship to, at least before he met Ace. I mean yeah, its true that Garp is Luffy's grandpa. But Garp was never that close to Luffy. Garp was too busy with his duties as a marine to take care of Luffy. That's why he was completely absent during the events of chapter 1 and subsequently left Luffy to be raised by Dadan.
Luffy never inherited Garp's will. Garp wanted Luffy to become a marine, something he never did, with Luffy ultimately choosing to be a pirate instead. So it just doesn't make sense for Luffy's dream to come from Garp. Then who?
I feel like this person had to have been Luffy's mom. Out of all of the people who were involved in Luffy's life growing up, she's the one person we know nothing about. So imo it has to be her.
Ultimately, I don't think it makes much sense for the will of D to be this magical force that drives Luffy and Roger to have the same dream. Rather I think it is simply the Will of the D clan that has been passed down from generation to generation from parent to child. Luffy got it from his mom, who got it from her parents, and so on and so on until the chain leads back to Joyboy.
