"Object of my affection" is the more common phrase. "Subject" is not wrong, but not very commonly used.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.
Repiobject or subject of my affection?You [have affection for / love] another person.