How is Mihawk in this conversation. lol
Kaido is perceived to be the strongest pirate by many people in the OP verse, but that doesn't mean it is reality. Killer isn't omniscient. It is a strong indicator, however. This was also a very off-hand comment. Killer could have been referring to the hype of his status and crew just as much as his personal strength.
Mihawk's placement is very simple. He is implied to be >= Shanks by character interactions and third-party statements and >= other swordsmen by his title. He could very well be weaker than both Shanks and other random swordsman because characters don't have perfect knowledge and titles aren't absolute fact, but for now these are strong indicators that he isn't.
Now you can argue where Shanks falls, but Kaido actually being #1 in the world (which is what I believe) doesn't contradict Mihawk's existence in any capacity. The only ambiguity we have to contend with is whether Big Mom is equal to Kaido and whether Oda thinks of her as a swordsman (I doubt it). But we've got a long time till EoS to explore that. No big deal.