go so far as to say (something)
To say something extreme, risky, or controversial. Often used in the negative. Although I wouldn't go so far as to say he's a coward, he definitely likes to avoid confrontation when he can. They went so far as to say they'd have their lawyer contact us if we didn't delete the video in question.
go so far as to say something
to put something into words; to risk saying something. I think that Bob is dishonest, but I wouldn't go so far as to say he's a thief. Red meat may be harmful, but I can't go so far as to say it causes cancer.