Hmmm.... In what context? You can certainly say: "He was breaking DOWN the walls" (for example, if you were talking about someone who defies perceived race or gender barriers).
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.