It sounds like a play on words. We need more of the song to see what point he's trying to make. "Alone" is more poetic and has a broader meaning. "I feel so alone," is much more common that "I feel so by myself." He probably means that physically he's by himself but emotionally/spiritually someone is with him.
"On my own" could mean that you live alone, or it could mean yo