What’s the full sentence of “Can’t have”?

Suddenly, people were pointing up at Harry all over the stands. His broom had started to roll over and over, with him only just managing to hold on. Then the whole crowd gasped. Harry's broom had given a wild jerk and Harry swung off it. He was now dangling from it, holding on with only one hand.
"Did something happen to it when Flint blocked him?" Seamus whispered.
"Can't have," Hagrid said, his voice shaking. "Can't nothing interfere with a broomstick except powerful Dark magic -- no kid could do that to a Nimbus Two Thousand."

What’s the full sentence of “Can’t have”?