I think people have to start looking at antagonists more laterally than vertically in this series again. There was a point in Pre-TS where Enel would have likely rocked Lucci or Moria like nothing, yet he came after.
Its all about how the villain can be perceived as a threat. Kizaru arguably is the fastest man alive, and could fuck with Luffy's observation haki. Blackbeard having 2 or 3 of the strongest DFs where one negates Luffy's powers largely poses a bigger issue for where hed need to rely more on Haki than powers. Akainu just straight up has one of the most dangerous DFs in the series. Oda has never drawn another villain with the ability to quite literally scar, maim and eviscerate bodies the way he did with Akainu and Ace, Jinbe, Luffy and Whitebeard.
All of these characters just have different threats to the MC. That could also include Im and the Gorosei, or even Shanks for that matter (I personally think BB and Shanks are polar opposite in that regard with respect to top DF powers and Haki).