Mix sb up with sb else

Hi everyone,

I know the sentence below is a correct:
"In the street people often mix me up with him" [me and my brother]

But is the following sentence correct (to mean the same):
"In the street people often mix us up."

Thanks.