Conqueror Specialists have characteristics that differ from the characteristics of Armament Specialists, and these characteristics embody a powerful soul and weapons skills.
Regarding their weapons, Conqueror Specialists are also known as traditional users of Color of the Supreme King Haki, and they often require Supreme Grade weapons to use the full of extent of it.
On the other hand, Armament Specialist weapons are described separately. Instead of using the highest quality of weapon, they use Armament Haki to cause their weapons to become unbreakable.
However, at high levels of mastery, an Armament Specialist can tap into a unique characteristic specific to them due to accessing Conqueror Haki. Their weapons can become Black Blades.
Achieving that would characterize the user of a Black Blade as the King of Blades.
I think this is an interesting part to consider. Of course, the exact process is still a bit vague, but if that's the framework then it'd be fine with me.
There are very few swordsmen who have the absolute resolve to stand above all other swordsmen, so there are plenty of people who can use Armament Haki, but few who can also access Conqueror Haki once they reach high mastery. Furthermore, there are even less Conquerors themselves who are committed to that specific dream making the characteristics of a Sword King/God unique and inaccessible to people who aren't like them.