have an affair (with someone)
To engage in a secret sexual relationship despite one or both participants simultaneously being married to or otherwise in committed relationships with other people. Did you hear that Rachel had an affair? Her husband is just devastated. Cliché though it is, yes, my husband had an affair with his secretary. I can't believe you're having an affair with a married man!
have an affair (with someone)
to have a love affair with someone. When I was 20, I had an affair with a rock star, and really made a fool of myself.