As far as I can tell, "pulling" has its ordinary literal meaning, i.e. exerting a force towards oneself. "against me" seems to mean "in opposition to me", i.e. the two people were exerting forces in opposite directions. Does that make sense in the context?