Yes, it's okay. You don't need to have said specifically "On the one hand," but you must have raised one point that this second point is in opposition to. I'm sure a number of people will post after me and say the opposite, insisting that you do need to have the "On the one hand" first.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.