I'm going with a cool 100 million because not all islands are created equal in size and density, same as for the Kingdoms. The population was probably much higher before the Void Century took place and any number of additional genocide/extermination campaigns by the WG. Races such as Lunarians, Buccaneers must have had decent pop.
If we assume that there is one CoC user for every ~1,000,000, then we only have
18 confirmed canon users so far in the story (from wiki). Even if that metric is biased, there should be plenty of CoC users out there that we aren't aware of and Oda can give. I expect Gorosei, Imu, Buggy-Sama, and others.