Loe J. Chaelee, Family