In my opinion - We don´t often go to the circus. / We don´t go to the circus very often. Frequency adverbs are usually put BEFORE the verb (but AFTER the verb "be" = I am often tired.). If you use VERY with OFTEN it usually goes at the end of the sentence.
I believe SELDOM is the closest equivalent. Although some grammar books say that the opposite of OFTEN is NOT OFTEN, the oposites of USUALLY are HARDLY EVER, RARELY, SELDOM. In my view, the difference is very small.