Regardless of what you hear, both me neither and me either are wrong. The correct expression is nor/neither did I. Me is ungrammatical in your example as it isn't the object of a verb and no other grammatical reason requires it.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.