He is not, he is also shouting that "he is going to be the pirate king", the only difference is he doesn't have the strength, the haki and the crew.
The focus is not on him so sure he is less annoying to you.
The plot armor excuse is a poor one, anybody in the main cast will always be protected by the plot, even Kidd is. This shouldn't affect their characters, but it does affect people's bias though.