Well if the clause is adverbial it normally modifies the verb or entire sentence and answers a question like when, why etc (as you know)...Since it doesn't, I was wondering how correct the clause was (although it does sound absolutely fine of course)
It is not adverbial, I think-- or if it is, it is of a different kind. It is an independent comment; it can appear like this: Why do we need a jumper? It's not even cold out! Or maybe 'when' = 'because' and we're back to an adverb: Why do we need a jumper– because it's not even cold out!
However, I'm not going to root through my Quirk to find out more now. If I r