On one hand v.s. on the one hand

Hi,

Is it acceptable among native speakers to say "on one hand" in place of the all-too-well-known "on the one hand" ? Up to very recently I had thought that the only right choise was with "the", until I heard this sentence:

"I'm torn by Induism. On one hand they believe in Karma and Reincarnation where good things happen to people who do good deeds. Then after you die they take a look at your carma account and you come back as something better or worse depending on how you did. What worries me is there's an injunction against beef"

The above excerpt was said by Stephen Colbert (hope I got it right)

What do you think of "on one hand", is it okey ?

Thanks !