I think it depends on the context and I would say that there isn't a hard-and-fast rule, just subtle differences. If it's a question such as '5 + 3 =' and you do it silently, you're answering it in your head. You can answer in your mind if it's a conceptual thing and you're getting things straight without speaking.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.