[That]'s what's so funny in you

The protagonist, Philip moved in with his uncle Mr. Carey, the Vicar of Blackstable after his mother's death.
He goes to the medical school St. Luke's in London.
He fell for a waitress. He goes out with her, but she doesn't like him.

............................................
"If I had an ounce of sense I'd never see you again," he said at last. "If you only knew how heartily I despise myself for loving you!" [Philip]
"That's not a very nice thing to say to me," she replied sulkily.
"It isn't," he laughed. "Let's go to the Pavilion." [Philip]
"That's what's so funny in you, you start laughing just when one doesn't expect you to. And if I make you that unhappy why d'you want to take me to the Pavilion? I'm quite ready to go home." [her]
[Of Human Bondage by Somerset Maugham]
I'd like to know what "that" means here.
Thank you in advance for your help.