Coleman Louis Segal, Family