He was seeing Mrs. Smith on the sly.She was supposed to be losing weight, but she was snacking on the sly.
on the sly
Secretly; in a clandestine or furtive manner. Even though he said he'd quit smoking, he still kept up the habit on the sly.The senator had been taking bribes from lobbyists on the sly for years before he was caught.
do something on the ˈsly
do something secretly: She didn’t seem to have much appetite for dinner. I wonder if she’s been eating chocolates on the sly? OPPOSITE: (out) in(to) the open