I personally wouldn't even use that kind of expression in a conversion. I would instinctively say "let's take a step back and reconsider this". That expression just randomly popped in my head and so now I'm willing to learn its usage. To me "back up" means "to move backwards as with a vehicle", "back off" means to withdraw and "back down" to give in. So none of them fit in this context?