"A growing number" is correct in this case. The article "a" is singular, so the noun (number) should also be singular. "Growing numbers" would also work but the meaning would be less clear and probably less proper; it would sound like you were trying to inflate or vaguely fib through what you're trying to say.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.