a burr under (one's) saddle
A source or cause of intense and persistent annoyance or irritation. The traffic delays resulting from construction of the new inter-city tramline has been a burr under residents' saddles for months now. I'm not trying to be a burr under your saddle; I'm just ensuring we are following the letter of the law every step of the way.
a burr under (or in) your saddle
a persistent source of irritation. North American informal