I would personally change "their" to "the". This is because "that they had left three years before" is presented as a defining clause, yet "their" already seems to (grammatically) determine which homes are meant. ".
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.