Matthew L. Green, Family