It doesn't seem quite idiomatic. When you speak of the weather, you're usually speaking of the weather outdoors. The change would be gradual, and surely not timed with your leaving the building. Perhaps you are getting colder as you leave the building. It sounds like you're trying to compare the weather outside the building with the weather inside the building. Pe