I don't understand what convinces you so much it's a detail Jordan left in the AMOL outline, drafts and attached notes

Brandon does research the background notes and character files (actually, Maria who knows these notes much better as she indexed them for RJ all those years more likely gathered summarized files of what she found relevant to the finale, for Brandon's sake. For a lot of the notes, Brandon needs to go to Charleston to consult).

You are aware that Jordan's own notes for the series, aside from that bundle TJ have assembled for AMOL, have a larger word count than the novels themselves?

The provenance of the rocks is no doubt mentionned in RJ's basic notes about the BT and its layout/physical description (and yes, he had those for every visited location).

The theory that those black rocks the Asha'man have fetched somewhere to build the BT walls, and that big one Taim used for his little speeches have come from SG is about as old as the Taimandred theory, comparing their description to those of SG and the valley.

Those mysterious black rocks which provenance RJ unusually kept for the readers have long been considered by many to be one of the early clues RJ put of a connection between the BT and the Shadow. And it appears to have been confirmed it was meant as a clue.

The secret is out since KOD that Taim serves the Shadow, so Brandon saw no need to RAFO the question (but he hesitated, wanting to see first if Terez's question related to a big theory that he wasn't aware of but which he may not want to kill. By Terez's answer it wasn't the case, so he answered. Brandon is on his guard when dealing with people from Theoryland - they have tons of pet theories he's not always familiar with).

Brandon candidly admitted a few times he's neither as secretive as RJ was, nor does he enjoy nearly as much having the fans squirm or to mislead them. The theories that make sense to him, or that he considers based on purposeful deceptions by RJ he usually refuse to squash or confirm, but what he considers too much in the left field or crazy, he prefers to put an end to when he can... unless it's a pet theory on a MB and squashing it would kill discussions. Minutia he considers no longer relevant to the finale he also hesitates far less than RJ to give answers about (when he knows the details).

The fact he answered the question at this point is very much a sign it's not of any great relevance in AMOL. It doesn't mean the rocks may not have some properties that will be revealed, but it's very unlikely they'll turn into this big plot point (in the vein of Rand shedding his blood on those rocks...) or Brandon would have RAFOed the question (he's not as secretive as RJ, but he still dislikes spoilers and Harriet plain hates them).