Joanna Tooker Vogel, Family