Well, it does.
The journal tells you the location of the 4 Phoneglifs, can you tell me how can any random guess that Zou has a Phoneglif???
Well, but now, a Random with no Phoneglif Idea, can know where 3 of the 4 phoneglifs are.
The diary is far more relevant than what you want to admit, because any kind of hint about the One Piece in there means the exclusion of Yamato as Nakama.
Even the "He laughed" could be considered a big spoiler for Luffy, and a reason for Yamato to not Joining the Crew.