"not" is an adverb, and it's modifying the infinitive verbs "to speak (to him)" and "to buy (a new car), both of which function as the object of "prefer" and "better", respectively. I do not [prefer to wait]. I prefer [not to wait].
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.