"~ or so" is a set idiomatic phrase meaning "approximately ~". "almost a month ago" and "a month or so ago" are both fine, but the combination "almost ... or so" does not work very well.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.