The downside?

1) Does "get it over with" mean "complete the job as quick as possible"?
2) Does "the downside" mean "the negative effect"?

Context:

On hearing the news of the bombing of http://en.wikipedia.org/wiki/Hiroshima:

I agreed emphatically with http://en.wikipedia.org/wiki/Henry_Stimson. Once we had got ourselves into the business of bombing cities, we might as well do the job competently and get it over with. I felt better that morning than I had felt for years… Those fellows who had built the atomic bombs obviously knew their stuff… Later, much later, I would remember [the downside].^{http://en.wikipedia.org/wiki/Freeman_Dyson#cite_note-53}