The sort answe is that 'to' is more common than 'of' - 'to' has three times as many citations in COCA as 'of'. However, this does not mean that you should always use 'to'; there are many contexts in which only 'of' is correct, or example: If you ignore him, you stand a good chance of losing your job.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.