Garp did fight Roger, but it was not a one-on-one battle. Garp was part of a group of Marines that were sent to take down the Pirate King, Roger. Garp and his team fought alongside Vice Admiral Monkey D. Bogard, who was leading the charge against Roger. During the battle, Garp and his team were able to hold their own against Roger's crew and even managed to land some blows on the Pirate King himself. However, in the end, it was Bogard who delivered the final blow that defeated Roger and ended the battle.