This is somewhat of a gray area, but since you are dealing with definite quantities, I believe that it is correct, in standard English, to say "more times," "fewer times," or "as many times," depending on the case. In colloquial spoken English, however, you are likely to hear the simpler version.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.