Both are OK, and you are right about 'the'. We also commonly say eg I'm going swimming.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.
AnonymousQ: What are you going to do this holiday?A: ...I'm going to spend all day at the pool. (The speaker and the listener both know which pool.)