be out of favor (with one)
To no longer be supported, accepted, approved of, or regarded highly by one. John has been out of favor with his boss ever since he managed to drive away the firm's most lucrative client. Those stupid toys are just a fad—they'll be out of favor sooner or later.