It sounds like you are describing the Confederate Reunion Grounds near Mexia, Texas. The site was established as a meeting place for Confederate veterans and their families, and it does have a history of family gatherings and reunions. The buildings you remember, including the large hall with a porch and cabins, were likely part of the facilities that were used for hosting these gatherings.
The Confederate Reunion Grounds are now recognized as a historical site and are managed by the Texas Historical Commission. While it may be challenging to find detailed records of specific buildings from the 1960s and 70s, you might consider reaching out to local historical societies in Mexia or the Texas Historical Commission for more information. They may have archives or resources that can help you learn more about the site and its history during your family reunions.