and at least on the audience side, most reviews you see are either 4-5 stars or zero/half a star, with very little inbetween. So it's more a question of how many of each they happened to get, than a very helpful weighted average... and there may be a fair few people like yourself who hated the first season so much they never even watched the second.
Anyway, what's more interesting is the arguments given by those reviewers who explicitly compare both seasons. People apparently figure there's better dialogue and character development, better flowing storytelling and thematic depth, more screentime for the villains. Better production values (sets, costumes, special effects) too apparently - I didn't think there was anything wrong with the production values of the first season to begin with.
I can sort of see most of those, I suppose, from the perspective of someone who either hasn't read the books or has read them but doesn't think the adaptation needs to make any sense.
Rand gets more screentime and more depth, the new Mat actor is better than the old one, Elayne is introduced and actually a success even by my critical reader-standards (fine actress and the writers haven't butchered anything important in her character or storyline yet), Aviendha not quite a success but could've been worse, Nynaeve and Egwene are developed further. Lanfear and Ishamael are shown more and reveal more of their personalities than in the books, with more depth than just some cartoonishly evil villain.
So yeah, the main reason why us readers are calling it worse is that it doubles down on changing far more from the books than we had bargained for, in ways that make no sense whatsoever to people who know the books, who cares if it looks good and is acted well, if the storyline is such nonsense? But then I start asking myself, how critical am I really about the logical consistency of movies or TV shows that aren't based on books, or on books that I haven't read? I do love, for instance, Buffy the Vampire Slayer, which doesn't exactly have a strong consistent overarching storyline either, with plenty of plotholes or characters conveniently getting whatever power or weapon they require to defeat the episode's bad guy, which then usually is never brought up again.
I don't think I could ever enjoy this show the way I enjoy BtVS or other such shows - but I guess it makes sense that non-readers can.