Explaining Low Perception Model Competency with High-Competency Counterfactuals

There exist many methods to explain how an image classification model generates its decision, but very little work has explored methods to explain why a classifier might lack confidence in its prediction. As there are various reasons the classifier might lose confidence, it would be valuable for this model to not only indicate its level of uncertainty but also explain why it is uncertain. Counterfactual images have been used to visualize changes that could be made to an image to generate a different classification decision. In this work, we explore the use of counterfactuals to offer an explanation for low model competency–a generalized form of predictive uncertainty that measures confidence. Toward this end, we develop five novel methods to generate high-competency counterfactual images, namely Image Gradient Descent (IGD), Feature Gradient Descent (FGD), Autoencoder Reconstruction (Reco), Latent Gradient Descent (LGD), and Latent Nearest Neighbors (LNN). We evaluate these methods across two unique datasets containing images with six known causes for low model competency and find Reco, LGD, and LNN to be the most promising methods for counterfactual generation. We further evaluate how these three methods can be utilized by pre-trained Multimodal Large Language Models (MLLMs) to generate language explanations for low model competency. We find that the inclusion of a counterfactual image in the language model query greatly increases the ability of the model to generate an accurate explanation for the cause of low model competency, thus demonstrating the utility of counterfactual images in explaining low perception model competency.

💡 Research Summary

The paper addresses a gap in explainable AI: while many methods explain why a model makes a particular prediction, few explain why a model lacks confidence in its prediction. The authors focus on “perception model competency,” a unified uncertainty metric that multiplies the maximum soft‑max probability (MSP) with an in‑distribution (ID) probability estimated via the reconstruction loss of an auto‑encoder. This P aR C E score is calibrated on a hold‑out ID set so that it directly reflects the model’s true accuracy.

To explain low competency, the authors propose five novel counterfactual generation techniques that transform a low‑competency image into a high‑competency (i.e., high‑confidence) counterpart while preserving visual similarity. The methods are:

1. Image Gradient Descent (IGD) – Optimizes the raw pixel space to maximize the competency score while minimizing LPIPS distance to the original image.
2. Feature Gradient Descent (FGD) – Optimizes the pixel space to raise competency but constrains the change in the classifier’s feature vector (using negative cosine similarity).
3. Autoencoder Reconstruction (Reco) – Uses the output of a pre‑trained auto‑encoder (trained for the competency estimator) as a counterfactual, relying on the auto‑encoder’s bias toward the data manifold.
4. Latent Gradient Descent (LGD) – Starts from the auto‑encoder’s latent code, performs gradient descent on the latent vector to increase competency while keeping latent distance small, then decodes back to image space.
5. Latent Nearest Neighbor (LNN) – Finds the nearest latent code in the ID calibration set (using L1 distance) and decodes it, providing a fast, manifold‑consistent counterfactual.

The authors evaluate these methods on two distinct datasets: a simulated lunar terrain dataset and a German speed‑limit sign dataset. For each dataset they synthesize 600 low‑competency examples, evenly split across six known causes of low competency (spatial anomaly, brightness, contrast, saturation, noise, pixelation). They assess counterfactuals using five properties adapted from the counterfactual literature:

• Validity – the counterfactual must receive a high competency score.
• Proximity – measured by LPIPS (visual similarity).
• Sparsity (Feature) – average cosine similarity between original and counterfactual feature vectors.
• Sparsity (Latent) – average cosine similarity between original and counterfactual latent codes.
• Realism – Fréchet Inception Distance (FID) against the calibration set.
• Speed – computational time.

Results show that Reco, LGD, and LNN outperform IGD and FGD across most metrics. LGD and LNN achieve the best trade‑off between high competency, low perceptual loss, and strong latent/feature similarity, while also being computationally efficient.

Beyond visual analysis, the authors explore the utility of these counterfactuals for language‑based explanations. They query pre‑trained Multimodal Large Language Models (MLLMs) with the original low‑competency image alone, and then with the original plus its high‑competency counterfactual. The inclusion of the counterfactual dramatically improves the MLLM’s ability to correctly identify the underlying cause of low competency (e.g., “the image is too dark” or “excessive noise”). This demonstrates that counterfactual images provide concrete visual evidence that helps language models generate accurate, human‑interpretable explanations.

In summary, the paper makes three key contributions: (1) formalizing the problem of explaining low model competency, (2) introducing five practical counterfactual generation methods—particularly latent‑space approaches (LGD, LNN) that are both effective and fast, and (3) showing that coupling high‑competency counterfactuals with MLLMs yields substantially better natural‑language explanations of uncertainty. The work opens a promising direction for building AI systems that not only report uncertainty but also transparently communicate why they are uncertain, leveraging the synergy between visual counterfactuals and multimodal language models.