The problem with this is that you already have the word "and," which carries with it the sense of "both," thus, to say, "Both ____ and ____ aren't reliable." is redundant. Note that you could say, correctly:
The husband and the wife aren't reliable.
Both of them aren't reliable. (Where "them" is the husband and wife.)
BOTH is not usually used in the negative. If you say, both of them haven't passed the examination, what do you mean? Neither of them passed or either of them passed? If two brothers have passed the examination, you can say Both of them have passed. If one of them has passed, you can say Either of them has passed. If they haven't passed, you can say Neither of them has passed. When there are only