Depending on the context, it might be possible to replace ' a little bit' with 'some' (which, arguably, should also be avoided in formal contexts, whenever possible). Still, I see no reason why a little bit' can't be used in a formal register. If I knew the specific context of your letter, perhaps I'd be able to come up with something better ( rather than just offering you this, vague and imprecise answer)..
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.
You could say, " I was somewhat disappointed ...." I hope that helps.