I would say "made from" (native speakers will correct me, if I am wrong). I think "made of" is used when the material an object is made of does not change, for example, "this table is made of wood" "Made from" is used when the material changes in its nature, for example, "wine is made from grapes" (grapes turn into wine). In your example, the material changes its nature, that is water freezes, that is why I think it will be "made from".
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.