Which usage is appropriate?

Neither version will work. Perhaps:
When it was time to clean the window in the upstairs kitchen, we ran into a problem.

I didn't notice the apostrophe after upstairs. It shouldn't be there. Here, upstairs is an adjective that modifies kitchen.
An apostrophe would be used if you were talking about a kitchen owned or used by someone.
The student's kitchen = The kitchen used by the student (one student).
The students' kitchen = The kitchen u

This is really something I should thank for it mr/mss/mrs Blue Jay,
I really didn't know this even though I took a year long English course when I was about to do my masters in Canada. Now I learned it thanks a lot.
Yes upstairs is more appropriate cause we use it not anybody else. But I would like to refer that there were two kitchen and the one on the upstairs had such a window. That wa