I already have plans to go somewhere. By using "come" I am including you in my plans. We are going to lunch - would you like to come with us. Come has the sense of joining you.
If I already plan to go dancing, I may ask if you want to come (with me). If I have no plans, but would like to do something with you, I might propose we go out dancin