If the gene that creates sparkers became latent, it would take generations to produce a child that would have the active trait potential. In a case such as that, it would take a long time to produce a sparker because the population would be composed of individuals whose genes did not express the capacity to channel. There are many methods to arrive at such a situation but the easiest would be to consider the channeling gene as a rare recessive with each pairing of a channeler to a non-channeler, such as Egwene/Gawyn, Rand/Min and Nynaeve/Lan, reducing the potential to produce a channeling descendant who could spark sharply decreases, while the potential to produce latent channelers who could channel if taught would decrease, but less sharply. What would happen then is that latent channelers would become more and more common and eventually sparkers would be bred out. But whether or not sparkers exist, the One Power is a separate thing entirely and is not affected by their absence or presence save that no one would be channeling it.

And we've heard RJ say that the Dragon Reborn was not always able to channel; this means that even though his soul can channel, his genetic inheritance/body and environmental factors are such that he would not be able to touch the One Power - say if he were latent in a world in which there were no active channelers to train him to be able to touch the Source.