The speaker is relating the statement to the present utterance of the listener ( 'I saw them yesterday' ). In spoken English, this is fine and common; in written English, the writer would probably reconsider and change it to couldn't . It is an odd formation, really, isn't it?
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.