Data Validation Based on 'YAML' Rules
Overview
A comprehensive data validation package that allows comparing datasets using configurable validation rules defined in 'YAML' files. Built on top of the 'pointblank' package for robust data validation, it supports exact matching, tolerance-based numeric comparisons, text normalization, and row count validation.
Bug fixes
- A tolerance rule (
abs/rel) on a column that is not numeric in the reference was silently ignored: the column fell back to an exact string comparison whileapplied_rulesstill listedabsandrel, so the report suggested a comparison by value that never happened. {datadiff} never converts data, so the rule cannot be honoured: such a column now gets a dedicated, always failing check,tolerance_on_non_numeric: <column>, visible in the agent and incoverageliketype_mismatch; a warning is raised when it is detected, the column takes part in no other check, andapplied_rulesno longer lists the rule that was not applied. This holds whatever the origin of the rule (by_name, or aby_typeblock such asby_type: character: abs: 0.1, which now fails every character column) and whatever the non-numeric type (character,date,datetime,logical).build_coverage()andsetup_pointblank_agent()gain atolerance_non_numeric_colsargument, default empty (issue #60).
Show all changes in 0.6.1Hide the rest of the changes in 0.6.1
- A comparison whose candidate or reference has no row could not carry a structural failure: on the local path adding the dummy column of a
type_mismatchstep errored ("replacement has 1 row, data has 0"), and on the lazy path withcheck_count: truethe structural and row-count steps interrogated 0 units and passed, soall_passedwasTRUEwhilecoveragesaidFAIL. The structural steps (missing_column,type_mismatch,tolerance_on_non_numeric) are now table-widecol_exists()checks on a reserved, deliberately absent column: one unit, failing on any table, empty or not, which also aligns the agent withcoverage(n = 1); acmpthat carries such a reserved name is refused. The row-count step still needs a row, so both paths seed one for the agent when the table is empty. keys: []in a rules file was read as an empty key and rejected before the data was even looked at ("Parameter 'key' must be a non-empty character vector"), although[]is the natural spelling of "no key" in a hand-written YAML.keys: []and the legacykey: []now mean "no key", exactly like an absent field orkeys: ~: the comparison is positional. A presentkeysfield now wins over a legacykeyfield whatever its value:keys: ~next tokey: [id]used to selectid, it means "no key" likekeys: []does. Thekeyargument ofcompare_datasets_from_yaml()andwrite_rules_template()is unchanged: an empty vector there is still an error,NULLis the way to ask for a positional comparison from R code (issue #59).
From NEWS.md in the release
Install
Health
- OK13 OK2026-08-05
- NOTE12 OK, 1 NOTE2026-08-01
- OK13 OK2026-06-09
- ERROR12 OK, 1 ERROR2026-06-08
- OK7 OK2026-03-19
Documentation
- Help pages
- 14
- Examples that run
- 100%11 run in a plain check
- Exported functions with examples
- 92%
- Documented parameters
- 100%
- Return-value docs
- 100%
- Help pages citing references
- 0%
Downloads
Repository
- PRs merged (all time)
- 100%
- Median days to close, 50 most recently updated closed issues
- 90
- Median days to close, 50 most recently updated closed PRs
- 0
Stars over time
Releases over time
Issues over time
PRs over time
Commits over time
Commit identities over time
4 development-tooling and community-health practices detected across 3 families in the upstream repository
Checks run against github.com/thinkr-open/datadiff on 2026-10-22.
The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel
This repository shows signs of AI coding tools.
- Claude Codefirst seen on or before 2026-03-03Named in .Rbuildignore: .claude, CLAUDE.mdPull request it wrote: opened by a project memberCommits: 5 by its own account · at least 11 crediting it in the message
- Claude Fable 53 commits · Jul 2026
Dependencies
Code & Tests
People & History
Pick two releases to compare their code metrics.
- 0.6.1Latest
- 0.6.0
- 0.5.0
- RR 4.6.0 released · 2026-04-24
- 0.4.42026-03-18
- RR 4.5.0 released · 2025-04-11
Package metadata
- First published
- 2026-03-18
- Total releases
- 4 / 1 yrs
- License
- MIT + file LICENSE OSI
- Minimum R
- ≥ 4.0.0
- Download size
- 169 KB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("datadiff")This package has no citation file, so citation() returns this entry built from DESCRIPTION.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-23, which the citation names so these numbers can be found later. More on citing and the projects behind them.