Grammatically so, but not very elegant. How about 'the man was frightened by the prospect of a difficult climb down the tree', or 'the man faced a difficult, frightening, descent'.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.
AnonymousThe man was frightened by the thought that itI would change 'was' to one of the two choices shown above, though what you have is not wrong.was[would be / might be] difficult for him to climb down the tree.