Skip to content

Sum of Ranking Differences Statistical Test

0packages depend
7.5Kdownloads / year
56.3%test coverage
13/13flavors OK

Overview

About

We provide an implementation for Sum of Ranking Differences (SRD), a novel statistical test introduced by Héberger (2010) doi:10.1016/j.trac.2009.09.009. The test allows the comparison of different solutions through a reference by first performing a rank transformation on the input, then calculating and comparing the distances between the solutions and the reference - the latter is measured in the L1 norm. The reference can be an external benchmark (e.g. an established gold standard) or can be aggregated from the data. The calculated distances, called SRD scores, are validated in two ways, see Héberger and Kollár-Hunek (2011) doi:10.1002/cem.1320. A randomization test (also called permutation test) compares the SRD scores of the solutions to the SRD scores of randomly generated rankings. The second validation option is cross-validation that checks whether the rankings generated from the solutions come from the same distribution or not. For a detailed analysis about the cross-validation process see Sziklai, Baranyi and Héberger (2021) doi:10.48550/arXiv.2105.11939. The package offers a wide array of features related to SRD including the computation of the SRD scores, validation options, input preprocessing and plotting tools.

Changes in 0.2.0

Bug fixes

  • calculateSRDValues, calculateSRDDistribution, utilsRankingMatrix, and calculateCrossValidation now stop with an informative error message when the input data frame contains NA values, non-numeric columns, fewer than 2 columns, or fewer than 2 rows. Previously, NA values could cause an infinite loop in the C++ layer.
  • calculateCrossValidation now stops with an informative error message when number_of_folds is 0 or 1. Previously, these values caused an immediate R session crash.
  • calculateSRDValues, calculateSRDDistribution, utilsRankingMatrix, and calculateCrossValidation now stop with an informative error message when the input data frame contains a constant column or constant columns. Constant columns carry no information for SRD analysis and previously caused undefined behaviour in the C++ layer.
Show all changes in 0.2.0Hide the rest of the changes in 0.2.0
  • plotCrossValidation no longer row-binds the precomputed summary statistics (min, xx1, Q1, median, Q3, xx19, max) together with the fold-wise SRD values into a single sample for ggplot to recompute quartiles and whiskers from. Previously, this mixture of real and derived values distorted the boxplot, especially when number_of_folds was small. The box geometry (min, Q1, median, Q3, max) is now taken directly from boxplot_values, and the mean is computed directly from the fold-wise SRD values; neither is derived from a contaminated sample (see also the related ordering simplification under Improvements).

New features

  • calculateSRDDistribution() and calculateCrossValidation() gain a new seed parameter for reproducible results. When seed = NULL (the default), the original stochastic behaviour is preserved, ensuring full backward compatibility.
  • Three example datasets are now bundled with the package and accessible via system.file("extdata", "<filename>", package = "rSRD"). The following three files use a semicolon separator and include a header row (read.csv(..., header = TRUE, sep = ";")).
  • mep_profiles.csv: voting profiles of Members of the European Parliament.
  • bundesliga20_21.csv: team performance data from the 2020/21 Bundesliga season.
  • movies1994.csv: ratings and rankings of films released in 1994.
  • Added a vignette, "Getting Started with rSRD", covering the full analysis workflow: data preprocessing, computing SRD values, the permutation test for significance, cross-validation, and pairwise visualisation via heatmaps. Access it with vignette("rSRD-introduction", package = "rSRD").

Improvements

  • plotCrossValidation no longer re-derives a column ordering from boxplot_values. The columns of SRD_values_of_different_folds already arrive ordered by median, Q1, Q3, min, and max, since this ordering is performed in the C++ layer (Cross_Validation::Wilcoxon/Alpaydin/ Dietterich) and preserved unchanged by calculateCrossValidation(), which only attaches solution names to the already-ordered columns. The redundant re-ordering step has been removed, simplifying the function and removing its dependency on boxplot_values for this purpose (plotCrossValidation still uses boxplot_values directly for box geometry; see Bug fixes).
  • Deprecated ggplot2::aes_string() calls in plotPermTest replaced with ggplot2::aes() using the .data pronoun.
  • plotPermTest now places solution labels at the interpolated y-value of the distribution curve at each solution's SRD value, replacing a random placement that changed on every call. The x- and y-axes now carry informative labels ("Normalised SRD value" and "Relative frequency" or "Cumulative relative frequency" depending on the densityToDistr argument), replacing auto-generated labels that exposed internal implementation details.
  • Broad import() directives for dplyr, ggplot2, tibble, janitor, rlang, and stringr replaced with specific importFrom() calls. stringr removed as an unused dependency.
  • utilsColorPalette is now generated via grDevices::colorRampPalette() instead of a 250-element hard-coded vector.
  • Added package-level documentation, accessible via ?rSRD.
  • All %>% pipe operators replaced with the native R pipe |>.

Testing

  • Added a comprehensive unit test suite using testthat (edition 3), covering calculateSRDValues(), calculateCrossValidation(), utilsMaxSRD(), utilsTieProbability(), utilsCalculateRank(), utilsCreateReference(), utilsPreprocessDF() and many other functions.

From NEWS.md in the release

Install

Health

CRAN checks
13OK
Slowest check: 4.2 min · r-devel-windows-x86_64
Status changes
  • OK13 OK2026-08-05
  • NOTE12 OK, 1 NOTE2026-08-01
  • OK13 OK2026-06-09
  • ERROR12 OK, 1 ERROR2026-06-08
  • OK14 OK2026-04-22
Show 2 earlier changesHide 2 earlier changes
  • ERROR13 OK, 1 ERROR2026-04-18
  • OK14 OK2026-03-10
Advisories

none known

Documentation

In release 0.2.0
READMENoVignettesYes · 1 · dynamicNEWSYes · 100% structured · mentions 0.2.0Citation fileNoWebsitenot declared in DESCRIPTION
Help pages
21 (5 internal)includes a package overview page
Examples that run
88%14 run in a plain check · 2 only when donttest examples are enabled (R CMD check --as-cran)
Exported functions with examples
100%
Documented parameters
96%
Return-value docs
94%
Help pages citing references
10%

Downloads

7.5K
CRAN downloads in the past year
Rank #7,253 · ~21/day · ~628/mo
Daily download trend is not available in this view yet.
42630 days
1.2K90 days
7.5K1 year
Compare downloads with other packages →
Also on344 r2u1 autocran

Dependencies

Declared dependencies
12 external dependencies (excludes base and recommended)

Installing rSRD adds 40 packages (22 with compiled code, 36 MB of source tarballs). Heaviest: stringi 11 MB, ggplot2 6.0 MB, gplots 4.6 MB.

Depends (1)
R >= 4.1.0
LinkingTo (1)
Enhances (0)
none
Reverse dependencies
0direct
0indirect

Nothing depends on this yet.

Datasets

People & History

People (8)
Author, Copyright holder, Maintainer since 0.1.4 (2022)
Author, Copyright holder
Author, Copyright holder
Contributor · added in 0.1.6
Contributor
Contributor · added in 0.1.6
Contributor · added in 0.1.6
Listed in earlier versions (3)
Show 3 peopleHide 3 people
Balázs R.
no longer listed · 0.1.4 to 0.1.7
Ali, Tugay Sen
no longer listed · 0.1.4 to 0.1.5
Sziklai
no longer listed · 0.1.8 to 0.2.0
Releases (6)

Pick two releases to compare their code metrics.

  • 0.2.0Latest
    record
    2026-06-30 · current release · diff ↗
  • R
    R 4.6.0 released · 2026-04-24
  • R
    R 4.5.0 released · 2025-04-11
  • 0.1.8
    record
    2024-11-01 · diff ↗
  • R
    R 4.4.0 released · 2024-04-24
  • R
    R 4.3.0 released · 2023-04-21
  • 0.1.7
    record
    2023-02-08 · diff ↗
  • unarchivedReturned to CRAN
    2023-02-06
  • 0.1.6
    record
    2023-02-06 · diff ↗ ·
    added 1 export
    utilsColorPalette
  • archivedRemoved from CRAN
    2023-01-10
    issues were not corrected in time
  • 0.1.5
    record
    2022-12-13 · diff ↗ ·
    added 1 export
    utilsCalculateRank
  • 0.1.4
    record
    2022-12-09
  • R
    R 4.2.0 released · 2022-04-22

Package metadata

First published
2022-12-09
Total releases
6 / 4 yrs
License
GPL-3 OSI
Minimum R
≥ 4.1.0
Download size
102 KB

Cite

Cite this package

Run in R for the authors' preferred citation:

citation("rSRD")
Staudacher, J., Gere, A., Horn, D., Hébeger, K., Olsson, L., Pothmann, A., Sen, A. T., & Sziklai, B. R. (2026). rSRD: Sum of Ranking Differences Statistical Test (Version 0.2.0) [Computer software]. https://doi.org/10.32614/CRAN.package.rSRD

This package has no citation file, so citation() returns this entry built from DESCRIPTION.

Cite the R Observatory

For a number measured here: a download total, a coverage figure, an archival date.

APA

Balamuta, J. J. (2026). R Observatory: Metrics for rSRD version 0.2.0 [Data set]. HJJB, LLC. Data release v2026-10-22. https://doi.org/10.5281/zenodo.21843040

From data release v2026-10-22, which the citation names so these numbers can be found later. More on citing and the projects behind them.

Report a problem with this page →

Privacy choices

These apply to this browser and are stored on this device only. Nothing about your choice is sent to us.

Read the privacy policy