"not that big of a project" is not correct in standard English (someone please correct me if it is accepted as standard in AmE). The correct standard form is "not that big a project". As far as I can think, there is no direct plural of this.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.