Upstairs/downstairs are only used for interior situations, not stairs that might be outside a building for example. They also refer to going up or down one floor only - so in a large building if you had to visit an office 5 floors away, you wouldn't normally use them. Upstairs/downstairs are locations - the destination up stairs/down stairs relate to the action.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.