In this case I would certainly stick with "that". In my view, both "the life of those around him" as well as "the lives of those around him" work in your sentence. So , I guess" you can say that "that" can replace both singular and plural nouns (at least in some situations).
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.