If she addressed you, and told YOU to get out, then the HE in "he's being an idiot" can only apply to your brother. " and then paused and looked at your brother and said, "He's being an idiot," then the "he" is you.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.