We do not normally use a singular noun after a plural possessive to mean 'one apiece'-- X 'we all brought our wife'. 'All my heart / all his heart / all our hearts' seems only logical as well.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.