Jeffrey Hariss Blum, Family