Hillary Frayne Slevin, Family