Your suggestion is a perfectly good one. In fact, I prefer that style, but the 'to do' is optional. 'To upset you' can be classified as a noun, so can function as the object of the verb want.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.