"I admit to having known him for a while" would be fine.
"Admit to" is better English. Also, the above sentence uses the Present Perfect ("have known" instead of "know", which is the Present Simple). With sentences in the Present Perfect, however, a time period which persists until the present moment must be stated.