"Wouldn't you say the party will be at our house". That's the way I would say it. "the party will be at me and my wife's house" sounds rather artificial.
"The party will be at me house"? Of course we would say "my house," so the grammatically correct answer is "The party will be at my and my wfe's house." Two people possess the house, so two possessive adjectives are required.
In everyday English, however, I'm sure most people would say, "The party will be at me and my wife's house." Is this grammatically correct