Houses and their sides

So this might be more of a linguistics questions than a proper grammar question, but I'm curious as to what you all might have to say.

My understanding is that normally, in the vast majority of cases, one would describe an inanimate object's sides from their own (the spectator's) perspective. If one is looking at a painting, for example, "There's a boat someone painted on the bottom lefthand corner." That is to say, it's on YOUR left as the spectator. (And, ultimately, it was on the painter's left as well.)

But with cars and houses/buildings, I thought that it worked differently. A car's left side will always be its left side no matter how you're looking at it because you orient yourself, mentally I suppose, as the driver of the vehicle.

If you were a spectator on the street looking at the front of a house, you might say "the garage is on the left," meaning YOUR left. But don't you orient yourself as though you were the house, standing at the front door looking out? So isn't it truly, then, that the garage is on the RIGHT of the house?

What are your thoughts everyone? Thanks in advance!