Only the first version is correct. 'a matter of degree' is an idiomatic phrase so you can't change it.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.
Anonymous"It's just a matter of degrees."This is also correct, but the meaning is different. This suggests that what is important is to have degrees, such as college degrees, or perhaps that the difference is just a matter of a few degrees Fahrenheit/Centigrade.