"So be it then" would be very strange, and depending on the conversation, outright rude. It depends on how the conversation went. " (Pause - answer is "no").
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.