The sentence with "Why" is the only correct. "Why" is used to enquire about the reason behind some action, which in your example is "caring of what he thinks".
"What" is used in requests to indentify something and does not fit well in your sentence. However, you could rephrase the question to make it use "what" this way: