Howard Richard Broadman, Family