I'm quite sure it's simply "madams," whether you're addressing Madams Jones and Smith, or whether you're saying something like, "I'm sure he knows all of the madams in Las Vegas."
Hi, sri, I stand corrected. Even my Am. Htg. dictionary lists the French plural with the silent "s's" for the nominative of address. It does seem strange to me that the French pronounciation is used for the plural while the American pronounciation is used for the singular.
I think the dictionaries are also segmented continent-wise. Better to satisfy with our respective dictionary spellings, and pronounciations. There seems to be cultural differences in the usage of English language. There must be a brainstorming session to prepare a universally acceptable dictionary.