To me (a non-native speaker!) , it means things are not going to be fine between us. (Is the recipient of the message in love with/attracted by the sender? In case, the sender would be saying that s/he doesn't want to go into a relationship because s/he doesn't see any future for them as a couple)
Have a look at meaning no. 3 of "work" (as a verb)