Sengoku said Rocks was his first and greatest rival. And the thing is, we saw Roger was pretty young in God Valley, so he was as far from his prime as Whitebeard perhaps, and Whitebeard was part of Rocks crew, as was Shiki (same generation), Big Mom and young Kaido. And Xebec was never explicitly stated to be stronger than Roger. They were in the same ballpark.