In that usage, both for and with mean in spite of (not although ). Neither for nor with are often used with that meaning. I would be inclined to use in spite of most often; for , less; and with , even less.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.