Either charm or charms is OK.
But I have to revise what I said earlier. The pattern is as follows:
attracted to X by Y.
X is the goal of the attraction, the person or object one is impelled toward.
Y is what causes or initiates the feeling of attraction.
You can leave out either the to phrase or the by phrase.
So the question