Sum of Ranking Differences Statistical Test
Overview
We provide an implementation for Sum of Ranking Differences (SRD), a novel statistical test introduced by Héberger (2010) doi:10.1016/j.trac.2009.09.009. The test allows the comparison of different solutions through a reference by first performing a rank transformation on the input, then calculating and comparing the distances between the solutions and the reference - the latter is measured in the L1 norm. The reference can be an external benchmark (e.g. an established gold standard) or can be aggregated from the data. The calculated distances, called SRD scores, are validated in two ways, see Héberger and Kollár-Hunek (2011) doi:10.1002/cem.1320. A randomization test (also called permutation test) compares the SRD scores of the solutions to the SRD scores of randomly generated rankings. The second validation option is cross-validation that checks whether the rankings generated from the solutions come from the same distribution or not. For a detailed analysis about the cross-validation process see Sziklai, Baranyi and Héberger (2021) doi:10.48550/arXiv.2105.11939. The package offers a wide array of features related to SRD including the computation of the SRD scores, validation options, input preprocessing and plotting tools.
Bug fixes
calculateSRDValues,calculateSRDDistribution,utilsRankingMatrix, andcalculateCrossValidationnow stop with an informative error message when the input data frame containsNAvalues, non-numeric columns, fewer than 2 columns, or fewer than 2 rows. Previously,NAvalues could cause an infinite loop in the C++ layer.calculateCrossValidationnow stops with an informative error message whennumber_of_foldsis 0 or 1. Previously, these values caused an immediate R session crash.calculateSRDValues,calculateSRDDistribution,utilsRankingMatrix, andcalculateCrossValidationnow stop with an informative error message when the input data frame contains a constant column or constant columns. Constant columns carry no information for SRD analysis and previously caused undefined behaviour in the C++ layer.
Show all changes in 0.2.0Hide the rest of the changes in 0.2.0
plotCrossValidationno longer row-binds the precomputed summary statistics (min, xx1, Q1, median, Q3, xx19, max) together with the fold-wise SRD values into a single sample for ggplot to recompute quartiles and whiskers from. Previously, this mixture of real and derived values distorted the boxplot, especially whennumber_of_foldswas small. The box geometry (min, Q1, median, Q3, max) is now taken directly fromboxplot_values, and the mean is computed directly from the fold-wise SRD values; neither is derived from a contaminated sample (see also the related ordering simplification under Improvements).
New features
calculateSRDDistribution()andcalculateCrossValidation()gain a newseedparameter for reproducible results. Whenseed = NULL(the default), the original stochastic behaviour is preserved, ensuring full backward compatibility.- Three example datasets are now bundled with the package and accessible via
system.file("extdata", "<filename>", package = "rSRD"). The following three files use a semicolon separator and include a header row (read.csv(..., header = TRUE, sep = ";")). mep_profiles.csv: voting profiles of Members of the European Parliament.bundesliga20_21.csv: team performance data from the 2020/21 Bundesliga season.movies1994.csv: ratings and rankings of films released in 1994.- Added a vignette, "Getting Started with rSRD", covering the full analysis workflow: data preprocessing, computing SRD values, the permutation test for significance, cross-validation, and pairwise visualisation via heatmaps. Access it with
vignette("rSRD-introduction", package = "rSRD").
Improvements
plotCrossValidationno longer re-derives a column ordering fromboxplot_values. The columns ofSRD_values_of_different_foldsalready arrive ordered by median, Q1, Q3, min, and max, since this ordering is performed in the C++ layer (Cross_Validation::Wilcoxon/Alpaydin/Dietterich) and preserved unchanged bycalculateCrossValidation(), which only attaches solution names to the already-ordered columns. The redundant re-ordering step has been removed, simplifying the function and removing its dependency onboxplot_valuesfor this purpose (plotCrossValidationstill usesboxplot_valuesdirectly for box geometry; see Bug fixes).- Deprecated
ggplot2::aes_string()calls inplotPermTestreplaced withggplot2::aes()using the.datapronoun. plotPermTestnow places solution labels at the interpolated y-value of the distribution curve at each solution's SRD value, replacing a random placement that changed on every call. The x- and y-axes now carry informative labels ("Normalised SRD value" and "Relative frequency" or "Cumulative relative frequency" depending on thedensityToDistrargument), replacing auto-generated labels that exposed internal implementation details.- Broad
import()directives fordplyr,ggplot2,tibble,janitor,rlang, andstringrreplaced with specificimportFrom()calls.stringrremoved as an unused dependency. utilsColorPaletteis now generated viagrDevices::colorRampPalette()instead of a 250-element hard-coded vector.- Added package-level documentation, accessible via
?rSRD. - All
%>%pipe operators replaced with the native R pipe|>.
Testing
- Added a comprehensive unit test suite using
testthat(edition 3), coveringcalculateSRDValues(),calculateCrossValidation(),utilsMaxSRD(),utilsTieProbability(),utilsCalculateRank(),utilsCreateReference(),utilsPreprocessDF()and many other functions.
From NEWS.md in the release
Install
Health
- OK13 OK2026-08-05
- NOTE12 OK, 1 NOTE2026-08-01
- OK13 OK2026-06-09
- ERROR12 OK, 1 ERROR2026-06-08
- OK14 OK2026-04-22
Show 2 earlier changesHide 2 earlier changes
- ERROR13 OK, 1 ERROR2026-04-18
- OK14 OK2026-03-10
Documentation
- Help pages
- 21 (5 internal)includes a package overview page
- Examples that run
- 88%14 run in a plain check · 2 only when donttest examples are enabled (R CMD check --as-cran)
- Exported functions with examples
- 100%
- Documented parameters
- 96%
- Return-value docs
- 94%
- Help pages citing references
- 10%
Downloads
Dependencies
Code & Tests
Datasets
People & History
Show 3 peopleHide 3 people
Pick two releases to compare their code metrics.
- 0.2.0Latest
- RR 4.6.0 released · 2026-04-24
- RR 4.5.0 released · 2025-04-11
- 0.1.82024-11-01 · diff ↗
- RR 4.4.0 released · 2024-04-24
- RR 4.3.0 released · 2023-04-21
- 0.1.72023-02-08 · diff ↗
- unarchivedReturned to CRAN2023-02-06
- 0.1.6
- archivedRemoved from CRAN2023-01-10issues were not corrected in time
- 0.1.5
- 0.1.42022-12-09
- RR 4.2.0 released · 2022-04-22
Package metadata
- First published
- 2022-12-09
- Total releases
- 6 / 4 yrs
- License
- GPL-3 OSI
- Minimum R
- ≥ 4.1.0
- Download size
- 102 KB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("rSRD")This package has no citation file, so citation() returns this entry built from DESCRIPTION.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-22, which the citation names so these numbers can be found later. More on citing and the projects behind them.