They could all be referring to the same number. It just depends on context. While "very few" certainly seems to be fewer than "a few" I could never tell you that "not many" is more than, or less than, "a few."
many (or not many) is relative to a typical or expected number.
If tonight 50 people are staying at a hotel that can hold 2,500 people, you might say that not many people are staying there tonight. But if all 50 crowded into the hotel dining room (which can hold 40 people) at one time, you would never say that not many people were in the dining room.