"would" goes with the past "didn't". Here's one version: I wouldn't be surprised if you didn't remember all the mean things you said about me. And here's another: I'm not surprised that you don't remember all the mean things you said about me.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.