See not care/give a fig.

Not care at all, not give a damn. Fig has denoted something worthless since about 1400. Moreover, in Mediterranean countries the sign of the fig is an obscene gesture, made by clenching one’s fist and pushing the thumb between the first and middle fingers. According to John Ciardi, the thumb represents the penis, the fingers the labia, and the gesture means “Fuck you.” Dante so used it in the Inferno section of The Divine Comedy. Centuries later, however, the phrase’s obscene provenance began to be largely ignored, and the cliché survives.