drive (one) out of (one's) mind
To annoy or frustrate one to the point of exasperation. A week on vacation with my relatives is enough to drive me out of my mind. That loud beeping noise is driving me out of my mind!
drive one out of one's mind
Fig. to make someone go crazy; to frustrate someone. You are driving me out of my mind with your nagging. Henry was driven out of his mind by all the negative comments.