"small damage" works in some other contexts but not here. The car suffered only small/minor damage. In your example, "little damage" means " practically no damage" and complements nicely with the "wasn't a bad crash" part.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.