I think that omitting "taking part", and letting "in" do all the work, might work slightly better in some other contexts, such as "when (taking part) in the race" or "when (taking part) in the play". (Even in those cases, though, it seems clearer to include "taking part".) In your example, "when in the rite" by itself just sounds a bit odd and unclear to me. Unlike a race or play, say, I don't see