It's part of a whole phrase rather than just the meaning of the word bump. If you hit another car when parking (so moving at slow speeds you may say that you bumped into it). In this case it is things that go bump in the dark, imagine a child scared of the dark.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.