That's a good explanation, but there are so many exceptions that you can't rely on it in all cases. In fact, you can say "He is at his office", or, more likely, "He is at the office". "in" makes it more specific, but "at" is also possible.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.