I think it's difficult for a lot of people. I have no idea of your cultural background, but I know a lot of Hispanic people (such as my wife) who seem naturally careless about distinguishing V's and B's. Sometimes they pronounce Buena Vista as Vuena Bista.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.