Garp's rep would take a major hit if it's later revealed that he fought in a battle against Kaido and co considering the latter doesn't have him in his top 5 combatants.
I have never in my mind doubted the fact that Xebec was Roger's rival. I am honestly confused by the whole battle. Lets assume Garp Ray and Scopper took on WB, Kaido and Big Mom. That leaves out Capt. John, Silver Axe and Shiki who is probably as strong as Goku and Garp.