The first one is formally correct but would not be used in everyday English. The second one (or the reduced form "I met a man I used to work with") would be normal in everyday English.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.