Evidential Conformal Prediction for Risk Classification

Rouaa Hoblos – FEMTO-ST Dép. AS2M

Lundi 10 Mars 2025 à 15h30
Laboratoire de Mathématiques de Besançon – 316B bis

In a world full of predictions, it is important to know how confident or uncertain those predictions are. Classical machine and deep learning models provide single-point predictions. It is crucial, especially in the medical domain, to quantify the uncertainty of these predictions to know how reliable they are. We proposed an uncertainty quantification method that is based on conformal prediction and evidence theory to attain this goal.

Conformal prediction is a framework where prediction sets (a set of possible outcomes), with a specific level of guaranteed coverage, are generated. Coverage represents the percentage of times a set contained the true outcome. To generate these sets, a conformity score is chosen. It is a measure of conformity or strangeness of a data sample to the model. Integrating evidence in its computation makes it evidential allowing us to have more reliable results. The more evidence we get, the more conformal data is to the model, and the more confident predictions are.

This score is calculated as: \(1 − p_i\) where \(p_i\) is the predicted probability of the true class \(i\). This \(p_i\) is calculated in terms of evidence as: \(p_i = \frac{e_i + 1}{K + \sum_j^K e_j}\) where \(K\) is the number of classes we have and \(j \neq i\). This method is tested on a real dataset for Breast Cancer.