a tough pill to swallow
Something, especially a fact or piece of news, that is unpleasant or difficult but which is unavoidable or must be accepted. It was a tough pill to swallow learning that my father's fortune had been squandered. Her recent breakup with Janet was a tough pill to swallow.