I think that people may understand this as it'd be in the context of a phone call but it's not really correct. Your voice 'breaks' at the age of thirteen or thereabouts; it's about the change into adulthood that gives you a deeper voice. What you need to say is, as you started off with, that someone's voice is 'breaking up'.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.