A classic, Maj. The logic is that in the first case, 'me', the person him/herself is a possible irritant, while in the second case, 'my', it is the sending of the flower that may be irritating. Therefore, formal grammar would require 'my'.
New words, one handy idiom, and a 2-minute quiz — delivered to your inbox to keep your streak alive.