Roger started as a none swordsman, but at some point he picked up a blade, not just any blade, but one of the 12 strongest. Plus, he named his son after his swords' name, who is coincidentally so, ends up being Luffy's brother, who died in his hands. The question is: who is going to wield Ace? The pirate king's sword itself.