Phyllis Ruth Brourman, Family