You've already written your sentence with 'of'. Namely, you shouldn't add 's to father. I mean: If you wanna use that " father's ", that will be correct in this sentence: My father's friend.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.