Walter Pierce Hammon, Family