The phrase "now that you mention it" is idiomatic in English, and is exactly appropriate for this situation. So you can't say "now that you refer to it" in this situation. In fact, the phrase "now that you refer to it" is awkward, since it resembles "now that you mention it", and thus could be confusing.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.