Merriam Webster Collegiate says both FUR-mee and FAIR-mee are possible for the scientific unit, but only gives FAIR-mee for Fermi, Enrico Fermi, the physicist. By the way, FAIR-mee would actually be the closest English pronunciation to the Italian one.