I think that a "passport-size photograph" is ok. By adding a hyphen you're basically dealing with an adjective, not a noun. a passport-sized photograh would mean that the size of the photograph equals the size of a passport.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.