Very undeveloped and spontaneous theory.
Were all the 20 kings maybe Ds?
What if the original Ds were carrying haki in their genes? Lilly went rogue, made a couple kids and 800 years later the genetic predisposition for haki is spread all around the world. Due to a mutation or a very rare gene, few individuals express CoC. Even fewer express VoaT (in case it's not related to the general haki types). But at least we know quite for sure that all VoaT guys possess CoC too (Momo not proven but very, very likely)
Remember CoC = kingly ambitions. 20 Kings/Queens rising to the top.
Back in the time when DFs were created from people's wishes/dreams, Imu might have wished for a haki fruit = spirit fruit. It grants all types of known hakis but op af along with unlimited lifeforce.
Imu might have noticed Sabo and Wapol so late cause they got kinda sloppy/bored over the centuries. I could imagine imu being able to CoO the shit out of the world and hear every little activity and even thoughts.
The 20 kings and queens wanted to reign with their DFs and their haki but Lilly enabled the entire population to fight back.
The 5 gorosei all have some mythical zoan God fruits. Nika fruit missing since Lilly stole it. And as immortal original Ds they will have ridiculously high haki levels.
It's haki bros. It's always haki that matters. Haki reigns supreme.