to address a family

When we address a family, say.. with the family name of Brown, do we say 'Brown family' or 'the Brown family'? Do we say 'Browns' or 'the Browns'?

I'd always thought the ones with 'thes' are correct, but lately I came across a letter with an addressee of a famliy name without 'the', which kind of bothers me.

Thanks for clarifying it in advance.