They're both correct. You'd choose depending on where you are when you say it. If you're at home, your wife might say, "Why didn't you ever bring home the present they gave you at the office Christmas party?"
If you're at work, your boss might say, "Why didn't you ever take this darned thing home??"