They're both okay. "I could not sleep at night" is by far the more common. "I could not sleep in the night" is rare. "I could not sleep during the night" would be somewhere between the two. I hear "in the night" for legal categories, like "burglary in the night." Actually, "in the nighttime" is more common than "in the night.''
Thank you so much for the explanation. So, all the time I have been using the less common and unnatural ones. It is good to know some are more common and some are not.