According to Jackie Mansas who comments on the bottom of this article (he did his Doctorate thesis on this castle and the whole area), this never belonged to the Count of Comminges but was an administrative decision of the Barousse elected officials back in 1993 to say it did in order to attract tourists (he was at that meeting). https://paratge.wordpress.com/2012/11/02/bramevaque/
I'm totally bilingual English French and spent half of my life in the South of France.
It's all very complicated and explained in his blog here. It was a féodale military castle and they are still unsure of the exact owners at that time. But the legends are great!
http://capsbourrutdespyrenees.over-blog.com/2021/05/les-deux-bramevaque-des-pyrenees-2.html