R Observatory: Metrics for yardstick version 1.4.0

Balamuta, James Joseph

doi:10.5281/zenodo.21843040

yardstick

1.4.0Package record

Tidy Characterizations of Model Performance

33packages dependsuggested by 31

540.6Kdownloads / year

97.6%test coverage

13/13flavors OK

Overview

About

Maintained by Emil HvitfeldtCRAN page ↗GitHub ↗Website ↗Issues ↗Declared funder: Posit Software, PBC

Tidy tools for quantifying how well model fits to a data set such as confusion matrices, class probability curve summaries, and regression metrics (e.g., RMSE).

Changes in 1.4.0

Breaking Changes

The global option yardstick.event_first (deprecated in 0.0.7) now throws an error. Use the event_level argument of individual metric functions instead. (#632)
conf_mat() now throws an error if anything is passed to ... (deprecated in 1.0.0). This argument has had no effect since case weight support was added. (#632)
roc_auc(), roc_aunp(), roc_aunu(), and roc_curve() now throw an error if a non-empty list is passed to options (deprecated in 1.0.0). Use the pROC package directly if you need these features. (#632)

Deprecations

dots_to_estimate(), metric_summarizer(), and metric_vec_template() (soft-deprecated in 1.2.0) now warn for all users. See the yardstick 1.2.0 release notes for recommended replacements. (#632)

New Metrics

gini_coef() computes the normalized Gini coefficient for regression, which measures ranking ability based on the Lorenz curve. (#147)
mse() computes the mean squared error. (#560)
rmse_relative() computes the relative root mean squared error, normalizing RMSE by the range of the true values. (#527)
fall_out() and miss_rate() compute the false positive rate and false negative rate respectively. (#336)
markedness() computes the markedness metric (PPV + NPV - 1), the predictive power analog of j_index(). (#27)

Show all changes in 1.4.0Hide the rest of the changes in 1.4.0

roc_dist() computes the Euclidean distance from the (sensitivity, specificity) point to the ideal point (1, 1) in ROC space. (#148)
sedi() computes the Symmetric Extremal Dependence Index, a prevalence-independent skill metric for binary classification that remains reliable at extreme class imbalance. (#630)
ranked_prob_score() computes the ranked probability score for ordinal classification data. (#524)
weighted_interval_score() is a new quantile metric. (#569)

Improvements

Added checks to all metrics for the na_rm argument. (#349)
Added improved argument checking for metrics with additional arguments. (#519)
Added documentation pages for each metric type (e.g., ?class-metrics, ?numeric-metrics) listing all available metrics with their direction and range. (#547, #540)
All class metrics and probability metrics now include mathematical formulas in their documentation. (#605)
All metrics now document their valid range of output values. (#572)
Documentation pages for metrics with alternate argument values now emphasize using metric_tweak() when building metric sets. (#626)
Survival metrics performance has been improved. (#576)
brier_class() has gained the event_level argument. (#515)
get_metrics() has been added to return a metric_set() containing all metrics of a specified type. (#534)
metric_set() now provides a more informative error message when estimate is not explicitly named for class/prob or survival metric sets. (#504)
roc_curve() has gained a thresholds argument for specifying custom thresholds at which the curve is evaluated. (#488)

Bug Fixes

brier_class() no longer returns NaN with extreme value case weights. (#614)
classification_cost() documentation now correctly refers to the cost column of the costs data frame. (#343)
mpe() documentation now includes the formula and clarifies the interpretation of positive and negative values. (#345)
poisson_log_loss() now handles 0-valued estimates without returning Inf or NaN. (#513)
Fixed a bug where ranked probability metrics didn't work in combination with other classification metrics in metric_set(). (#539)
Fixed documentation to show equations correctly. (#541)

Developer

Added infrastructure for quantile metrics. (#569)
Added infrastructure for survival metrics on the linear predictor. (#551)
new_metric() and related functions gain an optional range argument to store the valid output range of a metric. (#572)
Removed crayon as a suggested package. (#574)

From NEWS.md in the release

Install

R (CRAN)pak (dev)Ubuntu (r2u)Fedora (COPR)openSUSE (OBS)

install.packages("yardstick", repos = "https://cloud.r-project.org/")

Installs the current CRAN release. Prebuilt binaries on Windows and macOS.

if (!requireNamespace("pak", quietly = TRUE))
  install.packages("pak", repos = "https://cloud.r-project.org/")
pak::pak("tidymodels/yardstick")

Development version from tidymodels/yardstick. Installs pak first if it is not already available.

Release

# enable cranapt (r2u) if it is not already set up, then install
grep -rqs r2u.stat.illinois.edu /etc/apt/sources.list /etc/apt/sources.list.d/ \
  || curl -fsSL https://raw.githubusercontent.com/eddelbuettel/r2u/master/inst/scripts/add_cranapt_noble.sh | sudo bash
sudo apt install -y r-cran-yardstick

Fast Ubuntu binaries via r2u (no compilation). The cranapt setup runs once and is skipped when it is already enabled. We list this channel from recorded r2u download activity rather than from a live index of the repository, so treat it as last known rather than guaranteed.

bspm (used by r2u) also needs make: sudo apt install make

# enable the iucar/cran COPR if it is not already present, then install
ls /etc/yum.repos.d/*iucar*cran*.repo >/dev/null 2>&1 \
  || sudo dnf copr enable -y iucar/cran
sudo dnf install -y R-CRAN-yardstick

RPMs from the iucar/cran COPR, rebuilt from CRAN.

For install.packages() integration in R, also: sudo dnf install R-CoprManager

# add the autoCRAN OBS repo (Tumbleweed) if it is missing, then install
zypper -q lr | grep -qi autoCRAN \
  || sudo zypper addrepo https://download.opensuse.org/repositories/devel:/languages:/R:/autoCRAN/openSUSE_Tumbleweed/devel:languages:R:autoCRAN.repo
sudo zypper install -y R-yardstick

RPMs auto-built from CRAN on openSUSE OBS (Tumbleweed). We list this channel from recorded autoCRAN download activity rather than from a live index of the repository, so treat it as last known rather than guaranteed.

Health

CRAN checks

13OK

CRAN check page ↗

Slowest check: 6.6 min · r-oldrel-windows-x86_64

Status changes

OK13 OK2026-08-04
NOTE12 OK, 1 NOTE2026-08-01
OK12 OK, 1 FAILURE2026-06-08
NOTE12 OK, 1 NOTE2026-06-07
OK14 OK2026-03-10

Advisories

none known

How this is measured·How advisories are found

Documentation

In release 1.4.0

READMEYes · 292 wordsVignettesYes · 3 · dynamicNEWSYes · 100% structured · mentions 1.4.0Citation fileNoWebsiteYes · pkgdown

Help pages: 91 (13 internal)includes a package overview page
Examples that run: 100%82 run in a plain check (15 of them under a condition)
Exported functions with examples: 89%
Documented parameters: 99%
Return-value docs: 88%
Help pages citing references: 41%

Vignettes (3)

How these are measured

Downloads

540.6K

CRAN downloads in the past year

Rank #614 · ~1.5K/day · ~45.1K/mo

Releases R versions

Daily download trend is not available in this view yet.

30.4K30 days

95.3K90 days

540.6K1 year

Compare downloads with other packages →

Also on15K r2u9 autocran42.3K conda_forge230 c2d4u

How downloads are counted

Repository

tidymodels/yardstick ↗Organization: tidymodels

405Stars

65Forks

29Open issues

6Open PRs

15Releases

1,469Commits

39Commit identities

1,469 commits · Last activity 2026-09-17 · +0.7% stars, 30d · Default branch at 1.4.0.9000, ahead of CRAN 1.4.0 (checked 2026-10-18)

Responsiveness

How these are sampled

PRs merged (all time): 94%
Median days to close, 50 most recently updated closed issues: 393
Median days to close, 50 most recently updated closed PRs: 0

Stars over time

2025-10-09 · 3912026-10-21 · 405

How these are measured

PracticesBeta

15 development-tooling and community-health practices detected across 7 families in the upstream repository

Checks run against github.com/tidymodels/yardstick on 2026-10-18.

Continuous integration (2)

GitHub ActionsR CMD check workflow

The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel

Governance and community (2)

Code of conductContributing guide

Docs source (4)

README.Rmdpkgdown

GitHub Pages site last deployed 2026-09-17 · The repository has a site deploy workflow

pkgdown site last built 2026-04-07

Coverage (1)

Coverage reporting (Codecov)

Show all practicesHide 3 practices

Reproducibility and dev environment (1)

data-raw/

CRAN release process (2)

cran-comments.mdrevdep/

Lint, format, editor (3)

AirVS CodeRStudio project

How this is detected·Detection ruleset v3 (2026-09-26)

AI coding toolsBeta

This repository shows signs of AI coding tools.

Claude Codefirst seen 2026-01-16
Seen in the repository: .claude, .claude/skills
Named in .Rbuildignore: .claude
Commits: at least 1 crediting it in the message

From outside contributors

Claude Code
Pull requests: 1 from outside contributors, first on 2026-03-05, 1 from forks

Models named in its commits

Claude Opus 4.61 commit · Apr 2026

Counts are commits, not lines of code. Commits by a tool's accounts count once across all of its accounts. Commits crediting it are the largest count among its credit lines, a minimum.How this is detected·Detection ruleset v4 (updated 2026-09-27)

Dependencies

Declared dependencies

17 external dependencies (excludes base and recommended)

Installing yardstick adds 17 packages (10 with compiled code, 5.9 MB of source tarballs). Heaviest: vctrs 1.0 MB, dplyr 902 KB, rlang 766 KB.

Depends (1)

R >= 4.1

Imports (11)

cli dplyr generics hardhat (>= 1.4.3)lifecycle rlang tibble tidyselectutilsvctrs withr

LinkingTo (0)

none

Suggests (8)

covr ggplot2 knitr probably rmarkdownsurvivaltestthat tidyr

Enhances (0)

none

Dependency license breakdown →

Reverse dependencies

33direct

28indirect

Imports 33

Suggested by 31

Show the 48 most downloaded packagesHide the 48 most downloaded packages

Depends, Imports or LinkingTo, most downloaded first

Suggests, most downloaded first

workflowsets finetune spatialsample stacks tabnet shapr FCPS brulee vetiver baguette tidyposterior maidr foundryR BayesERtools bioLeak

and 16 more packages that suggest it on CRAN ↗

How these are counted

Code & Tests

Datasets

People & History

People (4)

Emil Hvitfeldt

Author, Maintainer since 1.1.0 (2022) · added in 1.1.0

Max Kuhn

Author

Davis Vaughan

Author · added in 0.0.2

Posit Software, PBC

Copyright holder, Funder · added in 1.2.0

Previously maintained by Davis Vaughan (0.0.4 to 1.0.0) and Max Kuhn (0.0.1 to 0.0.3).

Listed in earlier versions (1)

RStudio

no longer listed · 0.0.1 to 1.1.0

Releases (16)

Pick two releases to compare their code metrics.

R
R 4.6.0 released · 2026-04-24
1.4.0Latest
record
2026-04-07 · current release · diff ↗ ·
added 32 exports
check_linear_pred_survival_metric, check_ordered_prob_metric, check_quantile_metric, fall_out, fall_out_vec, get_metrics, gini_coef, gini_coef_vec, linear_pred_survival_metric_summarizer, markedness, markedness_vec, miss_rate, miss_rate_vec, mse, mse_vec, new_linear_pred_survival_metric, new_ordered_prob_metric, new_quantile_metric, ordered_prob_metric_summarizer, quantile_metric_summarizer, ranked_prob_score, ranked_prob_score_vec, rmse_relative, rmse_relative_vec, roc_dist, roc_dist_vec, royston_survival, royston_survival_vec, sedi, sedi_vec and 2 more
R
R 4.5.0 released · 2025-04-11
1.3.2
record
2025-01-23 · diff ↗
R
R 4.4.0 released · 2024-04-24
1.3.1
record
2024-03-21 · diff ↗
1.3.0
record
2024-01-19 · diff ↗ ·
added 4 exports
demographic_parity, equal_opportunity, equalized_odds, new_groupwise_metric
1.2.0
record
2023-04-21 · diff ↗ ·
added 28 exports
brier_class, brier_class_vec, brier_survival, brier_survival_integrated, brier_survival_integrated_vec, brier_survival_vec, check_class_metric, check_dynamic_survival_metric, check_numeric_metric, check_prob_metric, check_static_survival_metric, class_metric_summarizer, concordance_survival, concordance_survival_vec, curve_metric_summarizer, curve_survival_metric_summarizer, dynamic_survival_metric_summarizer, new_dynamic_survival_metric, new_integrated_survival_metric, new_static_survival_metric, numeric_metric_summarizer, prob_metric_summarizer, roc_auc_survival, roc_auc_survival_vec, roc_curve_survival, static_survival_metric_summarizer, yardstick_any_missing, yardstick_remove_missing
R
R 4.3.0 released · 2023-04-21
1.1.0
record
2022-09-08 · diff ↗
1.0.0
record
2022-06-06 · diff ↗
R
R 4.2.0 released · 2022-04-22
0.0.9
record
2021-11-22 · diff ↗ ·
added 2 exports
poisson_log_loss, poisson_log_loss_vec
R
R 4.1.0 released · 2021-05-18
0.0.8
record
2021-03-28 · diff ↗ ·
added 5 exports
classification_cost, classification_cost_vec, metric_tweak, msd, msd_vec
0.0.7
record
2020-07-13 · diff ↗ ·
added 5 exports
mpe, mpe_vec, new_class_metric, new_numeric_metric, new_prob_metric

Show 10 earlier eventsHide 10 earlier events

R
R 4.0.0 released · 2020-04-24
0.0.6
record
2020-03-17 · diff ↗ ·
added 8 exports
roc_aunp, roc_aunp_vec, roc_aunu, roc_aunu_vec, sensitivity, sensitivity_vec, specificity, specificity_vec
0.0.5
record
2020-01-23 · diff ↗
0.0.4
record
2019-08-27 · diff ↗ ·
added 4 exports
average_precision, average_precision_vec, iic, iic_vec
R
R 3.6.0 released · 2019-04-26
0.0.3
record
2019-03-08 · diff ↗ ·
added 8 exports
huber_loss, huber_loss_pseudo, huber_loss_pseudo_vec, huber_loss_vec, mase, mase_vec, smape, smape_vec
0.0.2
record
2018-11-05 · diff ↗ ·
removed 1 export
mnLogLoss

added 46 exports
accuracy_vec, bal_accuracy, bal_accuracy_vec, ccc_vec, detection_prevalence, detection_prevalence_vec, dots_to_estimate, f_meas_vec, finalize_estimator, finalize_estimator_internal, gain_capture, gain_capture_vec, gain_curve, get_weights, j_index_vec, kap, kap_vec, lift_curve, mae_vec, mape, mape_vec, mcc_vec, metric_set, metric_summarizer, metric_vec_template, mn_log_loss, mn_log_loss_vec, npv_vec, ppv_vec, pr_auc_vec and 16 more
R
R 3.5.0 released · 2018-04-23
0.0.1
record
2017-11-12
R
R 3.4.0 released · 2017-04-21

Package metadata

First published: 2017-11-12
Total releases: 16 / 9 yrs
License: MIT + file LICENSE OSI · relicensed from GPL-2 in 0.0.8
Minimum R: ≥ 4.1
Bundled data: 133 KB / 6 files
Download size: 1.6 MB

Appears in task views

Machine Learning & Statistical Learning Analysis of Spatial Data

Cite

Cite this package

Run in R for the authors' preferred citation:

citation("yardstick")

Format

Hvitfeldt, E., Posit Software, PBC, Kuhn, M., & Vaughan, D. (2026). yardstick: Tidy Characterizations of Model Performance (Version 1.4.0) [Computer software]. https://doi.org/10.32614/CRAN.package.yardstick

This package has no citation file, so citation() returns this entry built from DESCRIPTION.

Cite the R Observatory

For a number measured here: a download total, a coverage figure, an archival date.

Format

APA

Balamuta, J. J. (2026). R Observatory: Metrics for yardstick version 1.4.0 [Data set]. HJJB, LLC. Data release v2026-10-22. https://doi.org/10.5281/zenodo.21843040

From data release v2026-10-22, which the citation names so these numbers can be found later. More on citing and the projects behind them.

Report a problem with this page →