Casually calling someone a racist and then doubling down with the arguably even more offensive Uncle Tom implication is a big deal and the sort of comment where the burden of proof is very much on the accuser. So I think you need to justify the racist aspect with some direct by-sentence support.

What exactly did he say that is racist? Your original comments were practically in agreement with him in whole or part across the board, so what specific bit(s) were racist?