Both are right. The second one ('really not') is a stronger expression than the first one ('not really'). The word groups are 'really [not to my taste]' and 'not [really to my taste]', because negation affects everything to its right in the sentence.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.