Robert Frederick French, Family