It might be better understood by breaking it down like this:
'Amy and John visited Tom yesterday.'
- 'Where at?'
'At the hospital.'
But there's a twist here. With hospitals people are thought to be residing 'in' them. 'Tom is in hospital.' So the original statement could be rewritten like this:
Note that there is a difference between British English and American English.
If Tom worked at the hospital, you visit him at the hospital. If Tom is a patient there, we (Americans) say he is in the hospital, and you visit him in the hospital. (In short, we use "the.")