Do these sentences sound natural?

I'm trying to make a sentence using the expression

"I'll give you that". Does the sentence below sound natural?

He is handsome and filthy rich, I'll give you that,

but he's just not my type.