X = the present moment
..... > time progresses from left to right
[ ] = the specific time of asking
{ } = possible times of asking
.....[ask?]........................................................X..........
Did he ask you? (At that time in the past.)
You want to know if he asked at that time.
You have a specific occasion in mind when