Midway through means exactly what it appears. You're half-way or midway through, so that means you're not near the end or the beginning, but in the middle. So, "I will join the game midway through," means that you will join the game when the first portion is over and it's not near the end.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.