Both are fine: both singular and plural nouns can be negated. Use whichever one seems to sound better to you in the circumstances. 'No man is an island' is familiar (and the alternative strange) because it is a proverb; 'no men are impatient with their third wives' should sound more 'normal'.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.