The expression usually has a sense of nagging to it. In other words, someone might be constantly telling/reminding you to do something, or someone might be constantly criticizing you, etc. And you want that to stop.
"Leave me alone" is a pretty mild response. "Get off my back" is much stronger.
If your girlfriend insists on rubbing your back, you'd say, "Leave me alone." If your ex is screaming at you about back child support, you'd say, "Get off my back."
"Get off my back." is a strong expression used when a person feels they are being harrased or otherwise pestered or nagged by someone who will not "get off their case."
Common use example:
Will you get off my back! I will have the room cleaned by the time they get here.
Assuming the speaker is responding to inordinate criticism:
a) People can criticize others out of anger, or (alternatively) in an effort to improve their performance.
b) The speaker makes the point that the criticism cannot improve his performance, since it is already optimal; and that (to the contrary) it may impair his performance by upsetting him emotionally.
You are quite right of course. There is a world of difference between knowing the words and comprehending the common use of phrases, idioms, collocation and local expressions.