Did Greenbull actually direct the words "I'm not willing to fight you, at least not yet" really towards the RHP though, or towards the Scabbards and Momo? Since the scabbards and momo did not question his decision, he might've made this statement towards them as a kind of excuse.
Either way, the "not yet", if directed towards the RHP, is pretty ambiguous. Obviously he would be willing to engage with the SHP with proper force. The Marines were also willing to fight with Whitebeard. That is, when having all their forces; all admirals, fleet admiral, Shichibukai, Pacifista and ton of prep in one place.
The fact stands, that a bloodlusted fascist admiral, who wanted to kill thousands of lives to make a point, went away shaking and with his arms up in the air to signal surrender. Let's not sugar coat that.