The idea about the origins of languages is that in the beginning language had a great deal of phonemes, but as groups of people broke off to move away in various distances, each of these groups was not only smaller but also less diverse than the group that remained behind in Africa, almost per definition, and some of the excess phonemes fell into disuse (different ones in each group, of course) as the groups were relatively homogeneous and tended towards the use of the same phonemes. And the idea is that the big changes in language happened especially in that early phase, and less so later on (when cultures and peoples had developed, who had good reasons to want to conserve their language). Of course the numbers became very different later on, with no African language ranking among the ten, maybe even twenty, most-spoken languages in the world.

The point of the "the amount of phonemes will grow as more people speak a language" thing is again diversity - it's not the numbers in se that do it, it's the side-effect of sub-cultures being created, speakers being more likely to live far away from each other and see their speech evolve slightly differently.

If the groups are equal in size and diversity, it wouldn't. But the pattern of human dispersal across the globe is one of small groups moving out of Africa in various directions, then later on small groups moving out Asia and Europe to the Americas and Australia, and finally small groups crossing the Pacific in boats and reaching the various islands there.