" They're both correct and natural. " It actually refers to the people themselves. I'll see if I can find a good reference on it.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.
Son James[1]Large numbers of people(?)
[2]A large number of people(?)These are two slightly different senses of "numbers." They're both correct and natural.