Jackie Robinson wasn't all that long ago, and it wasn't until afterwards that black participation in MLB became the norm, rather than an oddity.

Now imagine that you are one of the pioneers who fought for your right to participate in baseball... and then the younger generation simply doesn't want to play it, anymore. It'd piss you off. It'd be like if, shortly after Rosa Parks refused to give up her seat on the bus, African Americans decided that buses are lame, and to not ride them anymore. It's certainly their right to do so, but it would be hard for that generation of Civil Rights workers to accept.

And consider this, as well. With the extensive farm league system, there are far more paying jobs playing baseball than in Basketball or Football... likely combined. The higher end salaries in baseball make a good deal more money than their football or basketball counterparts, and the lifespan of the average baseball athlete is also longer, significantly so when compared to football.

I think I can see why the pioneers would be irritated.