Sunney Kohlhoss Rivett, Family