Without more context, I'd say it suggests something illegal. It's a common expression for doing something in a way that will not attract attention. Also, in the old days, the "help" was not allowed to use the front entrance of swanky establishments.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.