Its been a while since I brushed up on my pirate lore and vocabulary, but feel free to fact check me if im mistaken
The first Mate is merely the first person recruited onto the crew, that doesn't necessarily mean he holds the most authority or is the strongest, the person in most pirate crews that serves as the Captains second in command is called the Quarter Master they command the ship whenever the captain is incapacitated or otherwise busy, it would seem the confusion is cause by Oda seemingly combining the two into one role