I don't see any issues with how you think the Power flows into channelers. Only that it would affect shileding. We know for a fact that shielding is a matter of concentration (on both ends). So couldn't just as convincing a case be made, using exactly how you visualize things, that men, who need to focus so much on controlling saidin, are a lot more susceptible to shields?
We've only seen the strongest men have no issues shielding women. In fact, even Rand says shielding Shalon takes a considerable amount of the OP. We've seen no weaker men claim any ease in shielding women, and there have been multiple instances where this could have been stated.
Instead, all we have is RJ saying that shielding someone of the opposite sex is harder. He doesn't say men have it easier than women.