This is apparently said in a hierarchy, with load of hay being the most valuable, a silver spoon being less valuable (and by a considerable amount), and a fly being worthless. The explanation for this is apparently that hay typically has good connotations in English, especially in a farm context. A haystack conjures up good images: the harvest has been good, there's plenty of feed for the livestock, and a haystack is often where romantic trysts are supposed to take place on a farm.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.