Why accuracy is a worthless metric for this
On a set that is 85 percent consistent, a model that always answers consistent scores 85 percent and has learned nothing. We measure ECE and AUC on a balanced set instead.
A confidence score is only useful if it is right about being confident. That property has a name, calibration, and a way to measure it that is not accuracy.
Take a realistic evaluation set where 85 percent of claim and evidence pairs actually agree. A model that ignores its input entirely and outputs consistent every single time scores 85 percent accuracy. It is a constant function. It has no information in it at all.
Expected calibration error asks a harder question: of all the times the model said 70 percent, did roughly 70 percent turn out to be right? A model can be accurate and badly calibrated, which is the worst combination in production, because it is usually right and gives no signal about when it is not.
The abstain channel is not a courtesy either. When the two spaces disagree, the correct output is that something is wrong here, not an average of two contradictory sources. Averaging a correct and an incorrect claim produces a confident sentence that is subtly false, which is harder to catch than an obvious error.