It sounds more natural for me, but others may prefer the more informal "come over" or "come around". Because it's dinner, the occasion seems more formal and this suits "come", in my view. Kind regards, Michael
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.