They both have incredibly similar meanings and are interchangeable. If some said the statement "I'm going to the shops", and someone replied, "What for?", it would more have the meaning "What's the reason you are going?", whereas "I'm going to the shops", with the reply, "For what?", could more have the meaning "What specifically are you going for/to buy?". I don't know if anyone would agree with
I see what you're talking about, I was looking at it as a rhetorical question, but still given your example I don't see much difference. If one says "I'm going to the store" and someone answers either "what for" or "for what" they seem to be asking why they are going to store or what they are going to purchase at the store either way.
You're right. I think the only real difference is in how they sound when you say them. Say them both out load and you will discover that "For what?" is shorter and more punchy than "What for?", literally in the amount of time it takes to say them.
Thanks for yor attention. By saying them interchangeable do you mean that we can use "What for?" instead of "For what?" and "For what?" instead of "What for?" without fear?