You can use both and there is not a great deal of difference in meaning. However, contrary suggests exact opposite. You think George is a genius I think he is an idiot.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.