Only these 4 clans could strike an agreement that they would attack the Treekillers under Janduin's leadership. Likely the Taardad were the Aiel, who gave the Avendoraldera to Cairhien years ago, and thus felt insulted the most. They probably searched for some support and at least 3 other clans followed them.

I think the relationship between Aiel clans is a bit similar to those between different Ajah in the WT. Some clans are more likely to work with each other, while others are mainly enemies or even have a blood feud (e.g. the Taardad generally don't like the Shaido).

Aviendha alludes at that as well in TGS, ch. 11:
"Did Rand al'Thor know how hard the Wise Ones worked behind his back to maintain Aiel loyalty? Probably not. He saw them all as one homogeneous group, sworn to him, to be used. That was one of Rand's great weaknesses. He could not see that Aiel, like other people, did not like being used as tools. The clans were far less tightly knit than he believed. Blood feuds had been put aside for him. Couldn't he understand how incredible that was? Couldn't he see how tenuous that alliance continued to be?"