The meanings are different (the first means 'never' and the second means 'soon') but both seem awkward. I would expect: We are often influenced without knowing it by the way he speaks.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.