Neither is OK, and both lack a main clause. Say eg If you don't like / want me to ask you anything, that's fine.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.
Anonymous(want me to) ask anything to youNo. The verb ask does not take that sort of complement (with to and a (pro)noun). It has to be 'ask you anything', i.e., with the pronoun first. This is when you want someone to answer your question. It's often followed by an indirect question: