Continuous Tense : I was having a great time

A:"So what happened last night? I came out of the bathroom and you had left."

B:"Oh yeah, well I just had to leave. Sorry I didn't think you'd mind anyway."
A:""Actually I was having a great time."
B:"Really?" 

Background: A and B were having a party yesterday. A went to bathroom halfway through the party and found that B left the party.

I'd like to know if this is the reason "Past Continuous Tense" is used, instead of "Simple Past (I had a great time) : When I was having a great time, you left the party." Continuous Tense seems to imply her departure interrupted me having a great time. What do you think? Thanks~