"Speak with Joy" can mean both things, depending on context. "Speak with Joy" can mean you want to speak with only Joy, but it can also mean you want to speak with Joy present.
With native speakers, you will rarely if ever hear someone use the sentence that way, however. It's considered vague, and frankly, most people would assume you meant t