Actually, no. Both forms are OK. 'To/With whom' is very formal and normally only appears in written English. Native speakers more frequently use the 'Who...to/with' form in speech.
The 'Whom would you compare John to/with' form is unacceptable in either medium, though you will hear it from those who are trying to be correct but do not know how to.