That sentence is fine. Sorry, I'm trying to understand what makes the sentences I wrote wrong or right, and not trying to find the correct way to write it.
You are right to say that "reluctantly" modifies the first part of your sentence - it modifies "to bring it up"
It is almost an idiom - which I think is what John is getting at. Sometimes we want to say: "I really didn't want to have to talk about this but it looks like I will have to."