From a past reference point (when the person first came to London), you're talking about a prior past time (presumably from birth up to that first visit), so the past perfect is the right tense, but it's an odd construction:
"How old were you when you first came to London?"