All we got from that was that he was more of a man of action. We got a very stilted perspective of a jilted lover more than a decade and a half after his death, and even then, there was little in the description to indicate he was "not a nice person." Hothead does not remotely have anything to do how nice someone is. St Peter is described as a hothead in the Gospels, as simply one example.
And that means what, exactly, on the 'possible rapist' scale?

No, he isn't. RtDB,
But Ashara’s daughter had been stillborn, and his fair lady had thrown herself from a tower soon after, mad with grief for the child she had lost, and perhaps for the man who had dishonored her at Harrenhal as well. She died never knowing that Ser Barristan had loved her. How could she? He was a knight of the Kingsguard, sworn to celibacy. No good could have come from telling her his feelings. No good came from silence either. If I had unhorsed Rhaegar and crowned Ashara queen of love and beauty, might she have looked to me instead of Stark?

There is nothing about rage or blame in his recollection, just melancholy regret. Nor is there any direct correlation between the "man who dishonored her" and either Stark. For all we know her "dishonor" came about because her relationship with the Stark was unrequited. And then there is the issue of what a fusty old virgin knight considers dishonorable and what the rest of the world does. Edric Dayne's & Harwin's accounts of Ned's & Ashara's relationship implies that it was relatively innocent and even if there had been sex, it was not scandalous. As Harwin states, they were both young and unbetrothed (and she was Dornish, with their much looser sexual mores). What one man might call dishonored, others laugh off as youthful high spirits. Not only that, from Edric's account, there does not appear to be any resentment towards Ned or the Starks for those events, and that's considering Ned was the guy who killed the family hero. If he had killed Arthur AND had a dishonorable relationship with Ashara, or his brother had raped her, there would have been a serious grudge between Houses Dayne & Stark. Edric would have grown up taking in resentment of the Starks with his mother's milk.

Not only that, it was the The Year of the False Spring, not the year of the Rebellion or any other significant event that would premept the weather for naming rights. Elia was present at the tournament, which is highly dubious for a woman of ill-health who would be pregnant or newly delivered of a child, so it had to be before Aegon was even conceived. This would put the tournament much too long before the rebellion (which lasted about a year) for Ashara to have recently lost a stillborn child conceived at Harrenhal, when she died, after the fighting was over.

And finally, even if the sex had been completely illicit and unseemly and something about that encounter HAD been dishonorable, that's still a long long way from rape.

And THAT'S all assuming that Barristan has all the facts. As I noted, it appears to be fairly common, if wildly inaccurrate gossip that Ned sired Jon on her. Selmy was recovering from grievous wounds suffered at the Trident at the time, and highly unlikely to have been superbly informed as to what was taking place on the other side of the kingdom, with a woman who had no idea of his affections. While Varys might have known, Barristan was consistently portrayed as hostile and suspicious of the Spider. Stannis cites Barristan telling him that Aerys' reign started to go downhill when he hired Varys, so he is not likely to have gone to him for details about the loss of his unrequited crush, and I frankly see no other way for him to have obtained that kind of intimate knowledge.