Like he said, what happened to Luffy while in Kuri in just a short period of time for him to advance in strength so much.
It's a statement of his potential to not just hurt him, but also the fact he has greater capacity to defeat him which can't be said of Yamato or the Scabbards.
It's a statement of strength to say that Luffy has potential to be strong like Whitebeard, Shanks, Oden, Roger, etc.
Can't say this about the others, but strength is definitely what defines greatness.
The strength to potentially defeat him is what Luffy had, this can't be said of Yamato and Scabbards, so there is a difference rather than just listing some people who just happened to do some damage to Kaido.