Even if it's 10% that would still constitute at least 2 million marines since the total force is "10s of millions"
I do genuinely believe that when it comes to recruiting members of sovereign nations, the government goes after elite soldiers so they can groom them to become high-ranking officers. This is supported by Kaido's mini-flashback.
Now there might as well be a couple of million marine soldiers out in the world. But having 1.5 million troops that are basically sitting ducks isn't really worth a damn against a yonko or a top-tier who has insane AoE.
That's why they only called in high-ranking/elite officers to Marineford. Cuz at the very least, the weakest troops among them have the ability to survive massive aoe attacks and hopefully hold back powerful enemies.
It's kinda like what Zoro told Sanji on fishman island. 100k fishmen with barely any training are sitting ducks against a mildly powerful pirate group.
What makes the marines strong isn't really their numbers, it's simply that they are better equipped and have a bigger amount of trained soldiers than any yonko.