fall out of favor with (one)
To no longer be supported, preferred, or highly regarded by one. John fell out of favor with his boss ever since he managed to drive away the firm's most lucrative client. They were a fad, that's all—they're already falling out of favor with the public.