Grouped Summaries for Large 'arrow' Datasets
Overview
Provides a 'dplyr'-like interface for grouped summaries on large 'arrow' datasets. Complete summaries are attempted in 'arrow' first. When a result can be reconstructed from 'arrow'-computable sufficient statistics, an explicit MapReduce-style reduction and 'R' finalization strategy is available. For arbitrary 'R' functions that require raw group observations, complete groups are materialized in bounded chunks, with optional parallel execution and shared-memory processing. The MapReduce strategy follows the programming model described by Dean and Ghemawat (2008) doi:10.1145/1327452.1327492.
Initial release
- Arrow-first grouped summaries via
summarise_big(). - Explicit Arrow reduction plus R finalization with
strategy = "map_reduce". - Bounded complete-group materialization via
parallel_chunksandshared_chunk. - Optional Mirai/Futurize parallelism and Mori shared memory.
- User functions referenced inside captured summary quosures are identified with
future::getGlobalsAndPackages()and supplied explicitly to futurized workers. - The caller's Future plan is restored after every exit path.
- Default worker count is 2.
Show all changes in 0.1.0Hide the rest of the changes in 0.1.0
- Added a runnable Getting Started vignette covering all four execution paths.
- Added pkgdown configuration and GitHub setup instructions for the project website and continuous
R CMD check.
From NEWS.md in the release
Install
Health
- OK13 OKNow
- OK5 OK2026-09-22
Documentation
- Help pages
- 2 (1 internal)includes a package overview page
- Examples that run
- 100%1 run in a plain check
- Exported functions with examples
- 100%
- Documented parameters
- 100%
- Return-value docs
- 100%
- Help pages citing references
- 0%
Downloads
Repository
6 development-tooling and community-health practices detected across 3 families in the upstream repository
Checks run against github.com/larry77/summarisebig on 2026-10-18.
The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel
GitHub Pages site last deployed 2026-09-11 · The repository has a site deploy workflow
pkgdown site last built 2026-09-11, before release 0.1.0 (2026-09-21)
Dependencies
Code & Tests
People & History
- 0.1.0Latest2026-09-21 · current release
- RR 4.6.0 released · 2026-04-24
Package metadata
- First published
- 2026-09-21
- Total releases
- 1 / 1 yrs
- License
- MIT + file LICENSE OSI
- Minimum R
- ≥ 4.3.0
- Download size
- 30 KB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("summarisebig")This package has no citation file, so citation() returns this entry built from DESCRIPTION.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-20, which the citation names so these numbers can be found later. More on citing and the projects behind them.