Home is adverb, and where is also adverb. That is why I think to should be deleted. I know you are right. I just wonder why the rule could not be applied to the adverb where.
Home is adverb -- In spite of some dictionary entries, I do not think this is so. A better interpretation is CSnyder's, making 'go home' idiomatic, with 'home' a noun and the preposition 'to' idiomatically omitted. To any native speaker, 'home' is a location, not a mode of action. I went home, I went to school, I went to the shop—these all carry the same semantic structure.