Chandler J. Mankins, Family