On the one hand, those connections and friendships have long moved past WoT - in fact, many of them never involved WoT at all, like Jens who's never even read the books, or others who probably did read the books but with whom I only ever really discussed other books, or politics, or other things.
On the other hand, I still have a residual fondness / nostalgia for WoT that's independent from the people from the community. A lot of that has to do with having read it in such a formative period of my life, but it's also just a pretty impressive series overall, despite its many flaws.
I think as you say, the problem is not Rafe changing things as such - the series was never remotely as suitable for a TV adaptation as aSoIaF or some other series, any adaptation was always going to have to make some quite major changes and in a way a showrunner who didn't dare to change enough would've been a bigger problem. The problem is when the changes don't make any sense or are just plain ridiculous and insulting the viewers' intelligence. Like Moiraine's whole 'I don't know which of you five is the Dragon, I don't know what we could even do at the Eye of the World besides die, in short this is a suicide mission and likely won't do any good, but let's do it anyway' speech - or the way they actually swallowed that and agreed to follow her (except Mat, but that had to do with the recasting mess more than the writers' actual choice...).