You can say both, Vincent, but they mean different things. at the classroom - suggests that they are doing their decorating outside the classroom. in the classroom - suggests they are doing it inside.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.