They're both correct. My sense of it is that between the two, the "along" shows deliberateness, or intention. " "While leaving" is a little strange in this case.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.