"Dancing with the music" is not incorrect English, but it's not very idiomatic in normal situations. A native speaker might, unusually, use "with" -- to me it has a slightly different nuance which is hard to explain -- but normally you should stick to "to".
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.