Even though paid for by his own disappointment

The protagonist, Philip moved in with his uncle Mr. Carey, the Vicar of Blackstable after his mother's death.
He goes to the medical school St. Luke's in London.
He fell for the waitress Mildred. He went out with her, but she didn't like him, and she got married to g German.
After she was dumped, she came to Philip's apartment.


..................................

"What would you like to do tomorrow?" he asked.

"Oh, I'm going to Tulse Hill. You remember the manageress at the shop, well, she's married now, and she's asked me to go and spend the day with her. Of course she thinks I'm married too."
Philip's heart sank.

............................

"You don't grudge me a little pleasure, Philip? You see, it's the last time I shall be able to go anywhere for I don't know how long, and I had promised."
...............................
He thought of Mildred enjoying herself in Tulse Hill, and he found in himself a real satisfaction because she was happy. It was an act of self-sacrifice on his part that he did not grudge her pleasure even though paid for by his own disappointment, and it filled his heart with a comfortable glow.

[Of Human Bondage by Somerset Maugham]

I'd like to know if "his pleasure" is implied before "paid for.

Thank you in advance for your help.