Who decided that you can’t go backwards in time in One Piece? Are you really questioning this? Lol
The One Piece itself has never been found, there’s an idea that Toki or somebody else sent it into the distant future. That‘s why nobody can find it, and why Roger implied that they were too early. It’s also why Imu doesn’t care about destroying whatever is currently on laughtale right now, the real thing that could threaten him hasn’t arrived. There’s no way that Imu doesn’t know where Laughtale is located, he’s been alive for centuries.