"food" (and hence also "frozen food") may be uncountable or countable. In the uncountable sense, meaning the stuff in general, it has no plural. In the countable sense, meaning food of a particular type, it has a plural "foods".
To me, "frozen food" and "frozen foods" are both acceptable on signage.