They mean the same, but usage can be different. I normally look out the window, while I throw something through it. I might peek through the window to see if someone is there.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.