Conformal Prediction and Uncertainty Quantification
Overview
Implements conformal prediction methods for constructing prediction intervals (regression) and prediction sets (classification) with finite-sample coverage guarantees. Methods include split conformal, 'CV+' and 'Jackknife+' (Barber et al. 2021) doi:10.1214/20-AOS1965, 'Conformalized Quantile Regression' (Romano et al. 2019) doi:10.48550/arXiv.1905.03222, 'Adaptive Prediction Sets' (Romano, Sesia, Candes 2020) doi:10.48550/arXiv.2006.02544, 'Regularized Adaptive Prediction Sets' (Angelopoulos et al. 2021) doi:10.48550/arXiv.2009.14193, Mondrian conformal prediction for group-conditional coverage (Vovk, Gammerman, and Shafer 2005) doi:10.1007/b106715, weighted conformal prediction for covariate shift (Tibshirani et al. 2019) doi:10.48550/arXiv.1904.06019, and adaptive conformal inference for sequential prediction (Gibbs and Candes 2021) doi:10.48550/arXiv.2106.00170. All methods are distribution-free and provide calibrated uncertainty quantification without parametric assumptions. Works with any model that can produce predictions from new data, including 'lm', 'glm', 'ranger', 'xgboost', and custom user-defined models.
This release corrects six defects that affected results, and changes two
defaults. The three headline defects are listed first; the weighted,
Mondrian and Jackknife+/CV+ fixes further down also change what you get
back. Anyone using conformal_aci(), conformal_aps(), conformal_raps(),
or passing a formula or fitted model as model should re-run their analysis.
Bug fixes
conformal_aci()applied the online update with the operands reversed:alpha_t + gamma * (err_t - alpha)instead ofalpha_t + gamma * (alpha - err_t)(Gibbs and Candes 2021, Eq. 2). A miscoverage event therefore narrowed the next interval instead of widening it, turning the intended negative feedback into positive feedback, andalpha_tran away to a clip boundary. Under a variance shift this drove empirical coverage to 0.605 against a 0.90 target.
Show all changes in 0.4.0Hide the rest of the changes in 0.4.0
- The
modelargument was discarded whenever it was a formula or a fitted model object, and a plainlm(y ~ .)was fitted in its place.model = y ~ asilently fitted every column ofx;lm(y ~ poly(v1, 3) + v2)was refitted asy ~ v1 + v2;rangerhyperparameters were dropped. Formulas and fitted models are now honoured, refitted on each conformal split. Objects that cannot be refitted raise an error namingmake_model()instead of silently substituting a default. conformal_aps()andconformal_raps()could return the full label set for every observation. The set builder included the class that *crossed* the threshold rather than inverting the calibrated score, andrandomize = TRUErandomised the calibration scores but never the set construction. With oracle probabilities on a four-class problem, APS returned a mean set size of 3.90 out of 4 at 99.9% coverage; it now returns 2.69 at 88.6%.conformal_jackknife(),conformal_cv(), andpredict()clamped the interval bounds to the smallest and largest order statistics when the quantile index fell outside1..n. Barber et al. (2021) define those bounds as infinite, which occurs for fewer than 9 observations atalpha = 0.10. They now return-Inf/Inf, matchingconformal_split().conformal_weighted()substituted the mean calibration weight for the test-point weight, so every test point received the same quantile. The newweights_newargument gives the exact procedure of Tibshirani et al. (2019), in which each test point receives its own quantile. Omitting it with non-uniform weights now warns.- A fitted
glmpassed to any classification method failed with "missing columns for class levels", because the internal probability matrix was unnamed. coverage_by_bin()failed with "'breaks' are not unique" whenever predictions contained ties. Duplicate breaks are now collapsed, with a warning reporting the number of bins actually used.- Conformal functions called
set.seed()on the global random stream and left it altered. Theseedargument now applies for the duration of the call only, and the user's.Random.seedis restored on exit. Seeded calls remain reproducible. plot()failed with "need finite 'ylim' values" on unbounded intervals. These are now drawn to the plot edge with a message.conformal_mondrian()andconformal_mondrian_class()silently substituted the pooled quantile for groups with fewer than three calibration points, voiding the group-conditional guarantee for exactly the groups that needed it. Such groups now receive an unbounded interval, with a warning naming the number of calibration points required.- Data frames containing non-numeric columns produced the misleading error "must not contain NaN or Inf values". They now name the offending columns and suggest
stats::model.matrix().
Changes in default behaviour
conformal_aps()andconformal_raps()now default torandomize = TRUE, the method as published. Deterministic scoring (randomize = FALSE) remains available and now warns when the conformal quantile saturates at 1. Passseedfor reproducible randomised sets.- Prediction sets are now the exact inversion of the calibrated score. New
allow_emptyargument onconformal_lac(),conformal_aps(),conformal_raps(),conformal_class_split(), andconformal_mondrian_class(); the defaultFALSEkeeps the previous behaviour of replacing an empty set with the most probable class.
Other changes
conformal_cqr()'squantilesargument previously had no effect. It is now validated, recorded on the returned object, and checked for consistency withalpha.conformal_cv()andconformal_jackknife()now validate thatx_newhas the same number of columns asx.- Jackknife+ prediction makes one call to
predict_funper leave-one-out model rather than one per (model, test point) pair. conformal_cv(x_new = NULL)recordstrain_approximation = TRUE, andprint()states that those intervals do not carry the CV+ guarantee.inst/CITATIONnow reports the installed version rather than a hardcoded one.paper/,Makefile, andllms*.txtare excluded from the source tarball, which removes anR CMD checkWARNING about GNU extensions inpaper/slides/Makefileand cuts the tarball from 4.2 MB.- README and the accompanying paper: corrected the CRAN status, dependency count, and competitor versions, and softened the Mondrian and ACI exclusivity claims (see
conformalForecastandAdaptiveConformal).
From NEWS.md in the release
Install
Health
- OK13 OK2026-08-05
- NOTE12 OK, 1 NOTE2026-08-01
- OK5 OK2026-03-20
Documentation
- Help pages
- 32 (1 internal)includes a package overview page
- Examples that run
- 100%25 run in a plain check
- Exported functions with examples
- 100%
- Documented parameters
- 100%
- Return-value docs
- 100%
- Help pages citing references
- 34%
Downloads
Repository
Stars over time
Commits over time
8 development-tooling and community-health practices detected across 5 families in the upstream repository
Checks run against github.com/charlescoverdale/predictset on 2026-10-11.
GitHub Pages site last deployed 2026-09-19 · The repository has a site deploy workflow
pkgdown site last built 2026-09-19
Show all practicesHide 1 practice
Dependencies
Code & Tests
People & History
Pick two releases to compare their code metrics.
- 0.4.0Latest
- RR 4.6.0 released · 2026-04-24
- 0.3.02026-03-19
- RR 4.5.0 released · 2025-04-11
Package metadata
- First published
- 2026-03-19
- Total releases
- 2 / 1 yrs
- License
- MIT + file LICENSE OSI
- Minimum R
- ≥ 4.1.0
- Download size
- 105 KB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("predictset")The package provides a citation file that builds its entry when R runs it. Run citation() to see it.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-17, which the citation names so these numbers can be found later. More on citing and the projects behind them.