Test caught him by surprise. That's how you'd need to use this idiom. Sometimes, the words can't be taken for their literal meaning, and if you carefully look at this, you'd come to know that "caught by suprise test today" doesn't sound natural. It is as good as saying "caught by a thing".