1. Garp being portrayed as relative to Roger, and WB clashing with Roger automatically shows he has the capacity to do the same with Garp
2. Garp as a marine, is chasing the trio, who have a specific objective to accomplish at God Valley under the direction of Rocks, which they have clearly chosen to stick by, as shown when they're taking out any pirates that aren't in line with their objective.
3. If they chose to fight Garp, they would be put in a situation where even when they take him down, they would be disadvantaged, not just at a health perspective (taking damage, wasting energy) but also a time perspective. There is very little purpose of spending time combatting marines chasing you (particularly Garp) whilst potential battles between pirates are being settled (Involving your own crew) when your presence could make the difference.
4. Narratively, if WB wasn't there for Roger to use as an excuse to give Ray his lay up, Oda couldn't ultimately give Rayleigh the lone moment he had with Shakky, which is clearly what he was going for this chapter, just look at the title of the chapter.
Conclusion: What we should really be focusing on, is how much of a mihawk upscale this chapter is, if WB slash could have took out Big mom, and was a wide enough range to be a danger to flying pirate shiki, imagine Mihawk chasing after then sending a MF size encompassing slash, he could literally end their lives right then and there in one fell swoop.
It's interesting because I've seen youtubers redditors etc etc, hold onto this notion that Xebec is somehow stronger than current Mihawk and Shanks, and that they wouldn't fare better in a 1v2 against Garp and Roger, (as that one panel led people to speculate). But the fact of the matter is these characters are still growing, and Xebec has not been shown to be anywhere in the ball-park of Shanks and Mihawk.
Not in mastery or haki.