Your explanation is rather confusing, too. However, in the phrases 'all of us' and 'all of my money', 'all' is a pronoun and it seems to me to refer unequivocally to 'us' and 'money' respectively.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.