I would use "test the water with my finger". For some reason, I have a memory of a mother testing the temp of the milk in the bottle by putting a drop or two on her elbow.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.