These are both correct: (1) I thought you didn't go there. (2) I thought you hadn't gone there. (1) says that I thought that you didn't usually go there as a habit.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.