Skip to content

predictset

0.4.0Package record

Conformal Prediction and Uncertainty Quantification

0packages depend
2.8Kdownloads / year
90.0%test coverage
13/13flavors OK

Overview

About

Implements conformal prediction methods for constructing prediction intervals (regression) and prediction sets (classification) with finite-sample coverage guarantees. Methods include split conformal, 'CV+' and 'Jackknife+' (Barber et al. 2021) doi:10.1214/20-AOS1965, 'Conformalized Quantile Regression' (Romano et al. 2019) doi:10.48550/arXiv.1905.03222, 'Adaptive Prediction Sets' (Romano, Sesia, Candes 2020) doi:10.48550/arXiv.2006.02544, 'Regularized Adaptive Prediction Sets' (Angelopoulos et al. 2021) doi:10.48550/arXiv.2009.14193, Mondrian conformal prediction for group-conditional coverage (Vovk, Gammerman, and Shafer 2005) doi:10.1007/b106715, weighted conformal prediction for covariate shift (Tibshirani et al. 2019) doi:10.48550/arXiv.1904.06019, and adaptive conformal inference for sequential prediction (Gibbs and Candes 2021) doi:10.48550/arXiv.2106.00170. All methods are distribution-free and provide calibrated uncertainty quantification without parametric assumptions. Works with any model that can produce predictions from new data, including 'lm', 'glm', 'ranger', 'xgboost', and custom user-defined models.

Changes in 0.4.0

This release corrects six defects that affected results, and changes two

defaults. The three headline defects are listed first; the weighted,

Mondrian and Jackknife+/CV+ fixes further down also change what you get

back. Anyone using conformal_aci(), conformal_aps(), conformal_raps(),

or passing a formula or fitted model as model should re-run their analysis.

Bug fixes

  • conformal_aci() applied the online update with the operands reversed: alpha_t + gamma * (err_t - alpha) instead of alpha_t + gamma * (alpha - err_t) (Gibbs and Candes 2021, Eq. 2). A miscoverage event therefore narrowed the next interval instead of widening it, turning the intended negative feedback into positive feedback, and alpha_t ran away to a clip boundary. Under a variance shift this drove empirical coverage to 0.605 against a 0.90 target.
Show all changes in 0.4.0Hide the rest of the changes in 0.4.0
  • The model argument was discarded whenever it was a formula or a fitted model object, and a plain lm(y ~ .) was fitted in its place. model = y ~ a silently fitted every column of x; lm(y ~ poly(v1, 3) + v2) was refitted as y ~ v1 + v2; ranger hyperparameters were dropped. Formulas and fitted models are now honoured, refitted on each conformal split. Objects that cannot be refitted raise an error naming make_model() instead of silently substituting a default.
  • conformal_aps() and conformal_raps() could return the full label set for every observation. The set builder included the class that *crossed* the threshold rather than inverting the calibrated score, and randomize = TRUE randomised the calibration scores but never the set construction. With oracle probabilities on a four-class problem, APS returned a mean set size of 3.90 out of 4 at 99.9% coverage; it now returns 2.69 at 88.6%.
  • conformal_jackknife(), conformal_cv(), and predict() clamped the interval bounds to the smallest and largest order statistics when the quantile index fell outside 1..n. Barber et al. (2021) define those bounds as infinite, which occurs for fewer than 9 observations at alpha = 0.10. They now return -Inf / Inf, matching conformal_split().
  • conformal_weighted() substituted the mean calibration weight for the test-point weight, so every test point received the same quantile. The new weights_new argument gives the exact procedure of Tibshirani et al. (2019), in which each test point receives its own quantile. Omitting it with non-uniform weights now warns.
  • A fitted glm passed to any classification method failed with "missing columns for class levels", because the internal probability matrix was unnamed.
  • coverage_by_bin() failed with "'breaks' are not unique" whenever predictions contained ties. Duplicate breaks are now collapsed, with a warning reporting the number of bins actually used.
  • Conformal functions called set.seed() on the global random stream and left it altered. The seed argument now applies for the duration of the call only, and the user's .Random.seed is restored on exit. Seeded calls remain reproducible.
  • plot() failed with "need finite 'ylim' values" on unbounded intervals. These are now drawn to the plot edge with a message.
  • conformal_mondrian() and conformal_mondrian_class() silently substituted the pooled quantile for groups with fewer than three calibration points, voiding the group-conditional guarantee for exactly the groups that needed it. Such groups now receive an unbounded interval, with a warning naming the number of calibration points required.
  • Data frames containing non-numeric columns produced the misleading error "must not contain NaN or Inf values". They now name the offending columns and suggest stats::model.matrix().

Changes in default behaviour

  • conformal_aps() and conformal_raps() now default to randomize = TRUE, the method as published. Deterministic scoring (randomize = FALSE) remains available and now warns when the conformal quantile saturates at 1. Pass seed for reproducible randomised sets.
  • Prediction sets are now the exact inversion of the calibrated score. New allow_empty argument on conformal_lac(), conformal_aps(), conformal_raps(), conformal_class_split(), and conformal_mondrian_class(); the default FALSE keeps the previous behaviour of replacing an empty set with the most probable class.

Other changes

  • conformal_cqr()'s quantiles argument previously had no effect. It is now validated, recorded on the returned object, and checked for consistency with alpha.
  • conformal_cv() and conformal_jackknife() now validate that x_new has the same number of columns as x.
  • Jackknife+ prediction makes one call to predict_fun per leave-one-out model rather than one per (model, test point) pair.
  • conformal_cv(x_new = NULL) records train_approximation = TRUE, and print() states that those intervals do not carry the CV+ guarantee.
  • inst/CITATION now reports the installed version rather than a hardcoded one.
  • paper/, Makefile, and llms*.txt are excluded from the source tarball, which removes an R CMD check WARNING about GNU extensions in paper/slides/Makefile and cuts the tarball from 4.2 MB.
  • README and the accompanying paper: corrected the CRAN status, dependency count, and competitor versions, and softened the Mondrian and ACI exclusivity claims (see conformalForecast and AdaptiveConformal).

From NEWS.md in the release

Install

Health

CRAN checks
13OK
Slowest check: 2.3 min · r-release-macos-x86_64
Status changes
  • OK13 OK2026-08-05
  • NOTE12 OK, 1 NOTE2026-08-01
  • OK5 OK2026-03-20
Advisories

none known

Documentation

In release 0.4.0
READMEYes · 2,702 wordsVignettesYes · 1 · dynamicNEWSYes · 100% structured · mentions 0.4.0Citation fileYes · computed when R runs itWebsiteYes · pkgdown
Help pages
32 (1 internal)includes a package overview page
Examples that run
100%25 run in a plain check
Exported functions with examples
100%
Documented parameters
100%
Return-value docs
100%
Help pages citing references
34%

Downloads

2.8K
CRAN downloads in the past year
Rank #13,070 · ~8/day · ~234/mo
Daily download trend is not available in this view yet.
32630 days
85190 days
2.8K1 year
Compare downloads with other packages →
Also on53 r2u7 autocran

Repository

2Stars
0Forks
0Open issues
0Open PRs
0Releases
51Commits
1Commit identities
51 commits · Last activity 2026-09-19 · +100% stars, 30d
Responsiveness
not measured

Stars over time

2026-03-20 · 12026-09-18 · 2
PracticesBeta

8 development-tooling and community-health practices detected across 5 families in the upstream repository

Checks run against github.com/charlescoverdale/predictset on 2026-10-11.

Continuous integration (1)
GitHub Actions
Docs source (3)
pkgdown

GitHub Pages site last deployed 2026-09-19 · The repository has a site deploy workflow

pkgdown site last built 2026-09-19

Reproducibility and dev environment (1)
Makefile
CRAN release process (2)
cran-comments.mdCRAN-SUBMISSION
Show all practicesHide 1 practice
For agents and models (1)
llms.txt
How this is detected·Detection ruleset v3 (2026-09-26)

Dependencies

Declared dependencies
10 external dependencies (excludes base and recommended)

Installing predictset adds 1 package (1 with compiled code, 629 KB of source tarballs). Heaviest: cli 629 KB.

Depends (1)
R >= 4.1.0
Imports (4)
cligrDevicesgraphicsstats
LinkingTo (0)
none
Enhances (0)
none
Reverse dependencies
0direct
0indirect

Nothing depends on this yet.

People & History

People (1)
Author, Maintainer since 0.3.0 (2026), Copyright holder
Releases (2)

Pick two releases to compare their code metrics.

  • 0.4.0Latest
    record
    2026-09-08 · current release · diff ↗
  • R
    R 4.6.0 released · 2026-04-24
  • 0.3.0
    record
    2026-03-19
  • R
    R 4.5.0 released · 2025-04-11

Package metadata

First published
2026-03-19
Total releases
2 / 1 yrs
License
MIT + file LICENSE OSI
Minimum R
≥ 4.1.0
Download size
105 KB

Cite

Cite this package

Run in R for the authors' preferred citation:

citation("predictset")

The package provides a citation file that builds its entry when R runs it. Run citation() to see it.

Cite the R Observatory

For a number measured here: a download total, a coverage figure, an archival date.

APA

Balamuta, J. J. (2026). R Observatory: Metrics for predictset version 0.4.0 [Data set]. HJJB, LLC. Data release v2026-10-17. https://doi.org/10.5281/zenodo.21843040

From data release v2026-10-17, which the citation names so these numbers can be found later. More on citing and the projects behind them.

Report a problem with this page →

Privacy choices

These apply to this browser and are stored on this device only. Nothing about your choice is sent to us.

Read the privacy policy