Garrett Clark Dailey, Family