Thomas Michael Payne, Family