Conformal Prediction With Conditional Guarantees

[Damien-Bouet]

Is your feature request related to a problem? Please describe.

Prior work has shown that it is impossible to provide exact conditional coverage universally in finite samples. Thus, most popular methods only guarantee marginal coverage over the covariates. This paper bridges this gap by defining a spectrum of problems that interpolate between marginal and conditional validity. [...]
When the target class of shifts is finite dimensional, we show how to simultaneously obtain exact finite sample coverage over all possible shifts. For example, given a collection of protected subgroups, our algorithm outputs intervals with exact coverage over each group.
(from Isaac Gibbs, John J. Cherian, Emmanuel J. Candès (2023))

While there are existing methods for adaptive conformal prediction, they fail to simultaneously provide adaptive prediction intervals and guaranteed coverage for identified subgroups.

Describe the solution you'd like
The goal is to implement the solution presented in the Gibbs et al. paper into a new class, MapieCCPRegressor. This class will enable users to obtain adaptive prediction intervals with conditional coverage guarantees for every (potentially overlapping) subgroup. The definition of these subgroups, which depends on the data and objectives, will be incorporated into a $\phi$ function that learns the distribution of conformity scores. This will be implemented using a new PhiFunction class.

Additional context
We aim to create a solution that combines the strengths of the following methods while avoiding their drawbacks:

• Conformalized Quantile Regression (CQR): Romano, Y., Patterson, E., & Candès, E.J. (2019)
  • Offers good adaptivity but lacks coverage guarantees for subgroups of interest.
• Mondrian approach : Conformalizes the intervals for each disjoint subgroup to ensure coverage guarantees.
  • Requires subgroups to be disjoint, limiting the number of subgroups as their number can increase exponentially with the number of constraints/groups.