Well, it is in the past, but not in the past tense--you have mentioned this morning with the present perfect tense, so it must be afternoon or evening (or later morning) now.
I assume you are still looking for equivalents to your original sentence:
I thought he would probably arrive this morning.
In that case, "He might have arrived this morning" is not correct. It is not an equivalent of the original sentence. An equivalent would be: "I thought he might arrive this morning".