Grouped and Nested Data Pipelines Built on 'data.table'
Overview
A toolkit for grouped and nested data pipelines built on 'data.table': import many Excel / CSV files (including multi-row headers) into one table, reshape and nest data by trait and group, run reproducible (stratified) k-fold cross-validation inside every group, summarise groups with report-ready descriptive statistics, and write each piece back to its own file or sheet. Developed for animal breeding, where it prepares phenotypic files for 'ASReml-R', 'HIBLUP' or 'DMU', but useful for any multi-group, multi-variable analysis.
New features
- New
desc_stats(): descriptive statistics by group (presets as inrstatix::get_summary_stats(), pluscv,skew,kurt,ci_low/ci_high), an optional total row (counts add up, statistics use all records),min_n, per-variabledigits,labels, report-ready text viafmt = "{mean} ± {sd}"and a wide report layout (shape = "wide"). Computed column by column with data.table's grouped functions: about a second for a million records in 20,000 groups. import_xlsx()reads multi-row headers (header_rows,header_sep), using the merged cells stored in .xlsx files (header_fill), accepts sheet names insheet, and can skip unreadable files / sheets (on_error = "warn").
Show all changes in 0.2.0Hide the rest of the changes in 0.2.0
Breaking changes
top_perc()now computes its statistics with the same engine asdesc_stats(): columns are lower case (n,min,max,mean,median,sd,se,cv), the CV isNAunless all selected values are positive, and a newstatsargument chooses the statistics.split_cv()andnest_cv()no longer depend on 'rsample'. Folds are built with base R / 'data.table'. Thesplitscolumn (rsamplersplitobjects) is replaced bytrain_idx/validate_idx(row indices);...is no longer forwarded. New argumentsseed(reproducible folds without disturbing the RNG stream),materialize(FALSEkeeps only indices) andout_type("df"returnstrain/validateas plain data.frames, e.g. for ASReml-R).export_nest()now returns the written file paths (invisibly) instead of a count, consistent withexport_list()andexport_xlsx().export_nest()stops when the grouping columns do not identify rows uniquely (files would overwrite each other) or name unknown columns.export_nest()/export_list()write missing values asNAand do not quote fields by default (na,quoteand...are passed tofwrite()).r2p_nest()gainsid; it stops instead of silently returning counts whenid+names_fromdo not identify rows uniquely.get_path_info(rm_path = FALSE)keeps the root (/,C:/,//).
Unified argument names
The old argument names have been removed; code written for 0.1.x must be
updated (calls with old names now fail with "unused argument").
| Function(s) | Old | New |
|---|---|---|
| nest_cv(), split_cv(), export_nest(), export_list() | nest_dt, split_dt | data |
| w2l_nest(), w2l_split() | cols2l | cols |
| c2p_nest() | cols2bind | cols |
| r2p_nest() | rows2bind, by | names_from, cols |
| export_nest() | nest_cols, group_cols | cols, by |
| top_perc() | trait | cols |
| w2l_nest(), w2l_split(), c2p_nest(), r2p_nest() | nest_type, split_type | out_type |
| export_nest(), export_list() | export_path | path |
| get_path_info() | paths | path |
| import_csv(), import_xlsx() | rbind | combine |
| import_csv() | rbind_label | file_col |
| import_xlsx() | show_excel_name, show_sheet_name (logical) | file_col, sheet_col (column name, NULL to omit) |
Bug fixes
- Fixed a bug in
export_xlsx(). w2l_nest()/w2l_split()no longer turn the caller's data.frame into a data.table by reference.w2l_split(sep = )works on every data.table version.split_cv()/export_list()reject a single data.frame.import_xlsx(): tables returned by parallel workers get a valid self-reference; duplicated base names are warned about and made unique.import_csv(): dots in directory names are no longer taken for an extension.export_list()/export_nest()sanitise path components.export_xlsx()rejects.xls/.xlsmpaths and sheets above Excel's row limit.top_perc()keeps groups too small to select any record (N = 0, with a warning);format_digits()validatesdigitsbefore using it.
Other
- Regression tests for all functions.
From NEWS.md in the release
Install
Health
- OK13 OK2026-08-05
- NOTE12 OK, 1 NOTE2026-08-01
- OK13 OK2026-06-09
- ERROR12 OK, 1 ERROR2026-06-07
- OK14 OK2026-03-10
Documentation
- Help pages
- 17
- Examples that run
- 100%17 run in a plain check
- Exported functions with examples
- 100%
- Documented parameters
- 94%
- Return-value docs
- 100%
- Help pages citing references
- 0%
- Get started with mintyr HTML
- Cross-validation HTML
- Descriptive statistics HTML
- Import and export HTML
- Reshape and nest HTML
- Utilities HTML
Downloads
Repository
- PRs merged (all time)
- not measured
- Median days to close, 50 most recently updated closed issues
- 2
- Median days to close, 50 most recently updated closed PRs
- not measured
Releases over time
Issues over time
Commits over time
8 development-tooling and community-health practices detected across 4 families in the upstream repository
Checks run against github.com/tony2015116/mintyr on 2026-10-18.
The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel
GitHub Pages site last deployed 2026-10-05 · The repository has a site deploy workflow
pkgdown site last built 2026-10-05
Dependencies
Installing mintyr adds 21 packages (10 with compiled code, 13 MB of source tarballs). Heaviest: data.table 5.7 MB, readxl 2.1 MB, vctrs 1.0 MB.
Code & Tests
Datasets
People & History
Package metadata
- First published
- 2024-12-13
- Total releases
- 5 / 2 yrs
- License
- MIT + file LICENSE OSI
- Minimum R
- ≥ 4.1.0
- Download size
- 552 KB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("mintyr")This package has no citation file, so citation() returns this entry built from DESCRIPTION.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-21, which the citation names so these numbers can be found later. More on citing and the projects behind them.