Just change it into "I must go" - then it's easier to analyze: "go" here is the full verb that is in its infinitive and follows the modal auxiliary must - without to! I have to go: "have to" is the substitute form for "must", "go" again is the full verb in its infinitive. "Have to" here is a fixed expression, and therefore "to" is actually part of the following infinitive form of the full verb, not a preposition.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.