So how exactly does pronunciation matter?

I am a non-native English speaker but lived in several Anglophone countries for a decade or so. I experience almost no difficulty in reading and understanding English publications. I can also understand almost every English-language movie, tv show and music. I sometimes make grammatical mistakes and unnatural English sentences when writing, but given that I have never received an F for my English essays, I think I can write something understandable in English(if it isn't, please let me know).

Speaking is somewhat problematic. I frequently pause and make very ridiculous mistakes when speaking in English, but I graduated from an American college, and nobody in America has ever complained to me that they could not understand what I was talking about(I sometimes had to say something more than once though). Also my pronunciation and accent hardly came under serious criticism. Interestingly, some Americans told me that I sounded like European, when I'm Asian.

So I've returned to my home country to apply for a military interpreter. When studying with other candidates, I often received some very serious remarks on my pronunciation. Of course I know my pronunciation is far from perfect, but these constant claims are just too hard to ignore.

First, they say I sometimes stress the wrong syllable, like diploMAcy.
they also say I just mispronounce some words, like "zebra," which I sometimes happen to pronounce "zevra" or so.

I thought these problems to be minor. But I'm just wondering...
Is this really a significant issue?