There is nothing wrong, in informal speech, in the way you have expressed it. If you wanted to be a bit more formal, for either writing or speech, you could say, The clients use the fact that the worker is new at the job to take advantage of him. or The clients exploit the fact that the worker is new at the job to take advantage of him.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.