" If you used "With kind regards" the person you sent the letter to might wonder why you didn't use just "Kind regards" instead. It's just one of those letter things that is so out of custom. You are probably right that "With kind regards" is a use in which the "with" has now been dropped.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.