Personally, "more often" refers not only to the number of conversations, but also the period in which they took place. If I knew someone for ten years and talked to them once a year, but you knew that person for six months and talked to them every month, I would have had more conversations, but you wold have spoken with them more often. Note the inherent ambiguity in your sentence.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.
Grammar GeekDid you talk to him more often than I talked to him? or Did you talk to him more often you talked to me?That's true. To avoid ambiguity, s/he could have said, "I've talked to him more often than you have" to convey