Your first sentence, "That's unfair, but to whom" is unclear. " "That's unfair, but to which of them" is also unclear. ," then "to which of them" works.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.