I think part of that is also to contain Xebec's mystery and not reveal too much in Wano, Kaido simply had knowledge of things that Oda wasn't going to let him reveal that early on, because he already knew when it was going to be revealed.
This is before Xebec's char introduction and he was still being kept as a silhouette.
Roger being used as the go-to example simply makes more sense because he was the Pirate King that kicked off the greatest pirate era.
I also don't think just because Kaido opted to mention rocks, that this must mean Kaido lost respect for Xebec, it simply means his respect for Roger can be even greater because of what Roger ended up achieving.
