If you definitely want to use that idiom, you could say: You know well, Florian, that I did not always give you the evil eye for your emotional philandering nor even your homecoming. Personally, I think "reproach" reads better, though the meanings are not identical (for one thing, "reproach" refers to words rather than looks).
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.