'Twenty and four' has no precedent; but otherwise, there are no 'rules' yet, because the century is too young. Around me, I feel that most native speakers are saying 'two thousand and four' or 'two thousand four' now, and that this will shorten to 'twenty ten' by, oh, about 2010. You can become a part of deciding this usage, though-- if enough people say 'twenty oh four ', then that's what will be more 'acceptable'.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.