We can go down to Providence

Pls provide some context.

It may simply mean to go to a place called Providence (going down to Providence: means P is located somewhere at the South of where the speaker is located)

here's the full line:

It doesn't have to be in our town. We can go, we can go down to Providence. Ok. We'll have dinner. Catch a movie. Nobody there knows us down there

It may mean going South (in that state) to Providence, or going to Providence from a larger city (New York City? I have no idea of how they are located wrt each other).

"Down there" means "over there, in that place (at a distance)"

"To go down to a place" may simply mean "going over to that place."

FWIW:
--------
to go down to someone or something

to travel to someone, something or some place that is downtown, at a lower level, or in the South.

Spears, American Idioms
--------

You may enjoy the discussion here on up and down:
http://tinyurl.com/h3pma

I'm sure there are other Providences in this world, but Providence, Rhode Island, is a city in New England, in the northeastern part of the U.S.