Should have seen that coming with this line.
Could you imagine Oda making Kizaru casually say this about Xebec, Roger or Primebeard while treating the man who defeated one of them like a toddler?
Fellow top tiers in the verse don't treat Kaidou as anything special. Never have, never will.