A Comprehensive Toolkit for Working with Encrypted Parquet Files
Overview
Utilities for reading, writing, and managing RCDF files, including encryption and decryption support. It offers a flexible interface for handling data stored in encrypted Parquet format, along with metadata extraction, key management, and secure operations using AES and RSA encryptions.
Security fixes
generate_pw(): switched from Mersenne Twister to a cryptographically secure RNG (openssl::rand_bytes).write_parquet(): DuckDB errors no longer leak raw encryption key material in messages.write_rcdf_parquet(): plaintext temp table is now always removed after a shared-connection write.write_rcdf_as(): replacedeval(parse())dispatch with a safe named-list of function references.hex_to_raw(): added strict input validation (type, length, character set).
Performance enhancements
write_parquet()/write_rcdf_parquet(): removedhttpfsoverhead; shared DuckDB connection across writes; default compression changed tozstd.read_rcdf(): newlazy = TRUEoption for DuckDB-backed lazy reads;n_threadsfor parallel I/O; duplicatePRAGMAcalls deduplicated per connection.add_metadata(): label assignment vectorised with batch update.open_duckdb_connection(): no longer forces single-threaded mode.
Bug fixes
Show all changes in 0.1.6Hide the rest of the changes in 0.1.6
collect.rcdf_tbl_db(): fixed infinite-dispatch recursion;NULLmetadata no longer causes an error.read_parquet(): unencrypted early-return path now correctly appliesmetadataandas_arrow_table.normalize_key_value(): hex keys no longer misidentified as base-64.
Documentation
- Main vignette rewritten for general/non-technical users with a step-by-step workflow.
- Merging vignette completed and fixed.
From NEWS.md in the release
Install
Health
ERRORexamplesr-devel-linux-x86_64-fedora-clang
Running examples in ‘rcdf-Ex.R’ failed
The error most likely occurred in:
> ### Name: write_rcdf_csv
> ### Title: Write RCDF data to CSV files
> ### Aliases: write_rcdf_csv
>
> ### ** Examples
>
> dir <- system.file("extdata", package = "rcdf")
> rcdf_path <- file.path(dir, 'mtcars.rcdf')
> private_key <- file.path(dir, 'sample-private-key-pw.pem')
>
> rcdf_data <- read_rcdf(path = rcdf_path, decryption_key = private_key, password = '1234')
duckdb is storing downloaded extensions and secrets under ~/.duckdb:
ℹ /data/localhost/ripley/.duckdb
This persists across sessions and is shared with the DuckDB CLI and other clients.
ℹ Run duckdb(shared_home = FALSE) to use a temporary directory instead.
ℹ See ?duckdb_storage for details and alternatives.
duckdb was built with libc++ (not libstdc++) on Linux, so DuckDB extensions are disabled:
• Loading a prebuilt (libstdc++) extension could crash R (https://github.com/duckdb/duckdb-r/issues/1107).
• INSTALL/LOAD of an extension will error, an
...[truncated]...
..)
11: dbSendQuery(conn, statement, ...)
12: dbSendStatement(conn, statement, ...)
13: dbSendStatement(conn, statement, ...)
14: DBI::dbExecute(conn, sprintf("CREATE TABLE %s AS\n SELECT * FROM read_parquet(%s,\n encryption_config = { footer_key: %s });", sql_ident(conn, record), sql_literal(conn, pq_file), sql_literal(conn, secret$key)))
15: DBI::dbExecute(conn, sprintf("CREATE TABLE %s AS\n SELECT * FROM read_parquet(%s,\n encryption_config = { footer_key: %s });", sql_ident(conn, record), sql_literal(conn, pq_file), sql_literal(conn, secret$key)))
16: process_parquet_file(conn, pq_file, secret, pk, registered_keys)
17: process_rcdf_file(conn = conn, rcdf_file = rcdf_files[i], key = creds$keys[i], password = creds$passwords[i], metadata = metadata, ignore_duplicates = ignore_duplicates)
18: read_rcdf(path = rcdf_path, decryption_key = private_key, password = "1234")
An irrecoverable exception occurred. R is aborting now ...ERRORtestsr-devel-linux-x86_64-fedora-clang
Running ‘testthat.R’
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> # This file is part of the standard setup for testthat.
> # It is recommended that you do not modify it.
> #
> # Where should you do additional test configuration?
> # Learn more about the roles of various files in:
> # * https://r-pkgs.org/testing-design.html#sec-tests-files-overview
> # * https://testthat.r-lib.org/articles/special-files.html
>
> library(testthat)
> library(rcdf)
>
> test_check("rcdf")
duckdb is storing downloaded extensions and secrets under ~/.duckdb:
i /data/localhost/ripley/.duckdb
This persists across sessions and is shared with the DuckDB CLI and other clients.
i Run duckdb(shared_home = FALSE) to use a temporary directory instead.
i See ?duckdb_storage for details and alternatives.
duckdb was built with libc++ (not libstdc++) on Linux, so DuckDB extensions are disabled:
* Loading a prebuilt (libstdc++) extension could crash R (https:/
...[truncated]...
(conditionMessage(cnd), "\n") NULL})
50: with_reporter(reporters$multi, lapply(test_paths, test_one_file, env = env, desc = desc, shuffle = shuffle, error_call = error_call))
51: test_files_serial(test_dir = test_dir, test_package = test_package, test_paths = test_paths, load_helpers = load_helpers, reporter = reporter, env = env, stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning, desc = desc, load_package = load_package, shuffle = shuffle, error_call = error_call)
52: test_files(test_dir = path, test_paths = test_paths, test_package = package, reporter = reporter, load_helpers = load_helpers, env = env, stop_on_failure = stop_on_failure, stop_on_warning = stop_on_warning, load_package = load_package, parallel = parallel, shuffle = shuffle)
53: test_dir("testthat", package = package, reporter = reporter, ..., load_package = "installed")
54: test_check("rcdf")
An irrecoverable exception occurred. R is aborting now ...NOTER code for possible problemsr-devel-linux-x86_64-debian-clang, r-devel-linux-x86_64-debian-gcc, r-devel-linux-x86_64-fedora-clang, r-devel-linux-x86_64-fedora-gcc, r-devel-windows-x86_64
Found calls to structure() using deprecated special names: rcdf/R/read_dot_env.R (.Names: 1) '.Names' should be changed to 'names'.
- ERROR8 OK, 4 NOTE, 1 ERROR2026-10-01
- NOTE8 OK, 5 NOTE2026-09-30
- ERROR8 OK, 4 NOTE, 1 ERROR2026-08-06
- NOTE11 OK, 2 NOTE2026-07-16
- OK13 OK2026-07-14
Show 3 earlier changesHide 3 earlier changes
- ERROR12 OK, 1 ERROR2026-06-29
- OK13 OK2026-06-20
- ERROR13 OK, 1 ERROR2026-03-10
Documentation
- Help pages
- 29 (1 internal)includes a package overview page
- Examples that run
- 64%18 run in a plain check · 10 never run
- Exported functions with examples
- 100%
- Documented parameters
- 100%
- Return-value docs
- 100%
- Help pages citing references
- 0%
- rcdf HTML
- merging-rcdf HTML
Downloads
Repository
- PRs merged (all time)
- 88%
- Median days to close, 50 most recently updated closed issues
- 16
- Median days to close, 50 most recently updated closed PRs
- 0
Stars over time
Releases over time
Issues over time
PRs over time
7 development-tooling and community-health practices detected across 3 families in the upstream repository
Checks run against github.com/yng-me/rcdf on 2026-10-11.
The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel
GitHub Pages site last deployed 2026-05-30 · The repository has a site deploy workflow
pkgdown site last built 2026-05-30
Dependencies
Code & Tests
People & History
Pick two releases to compare their code metrics.
- 0.1.6Latest
- RR 4.6.0 released · 2026-04-24
- 0.1.5
- unarchivedReturned to CRAN2026-03-03
- 0.1.42026-03-03 · diff ↗ ·
removed 1 export
get_data_dictionaryadded 4 exports
collect, get_attr, read_dot_env, read_parquet_tbl - archivedRemoved from CRAN2026-02-19resubmitted with the same issue
- unarchivedReturned to CRAN2026-02-12
- 0.1.3
- archivedRemoved from CRAN2025-12-18issues were not corrected even after re-submission
- unarchivedReturned to CRAN2025-12-03
- 0.1.22025-12-03 · diff ↗
- archivedRemoved from CRAN2025-12-02issues were not corrected in time
- 0.1.12025-10-12 · diff ↗ ·
added 4 exports
generate_pw, generate_rsa_keys, get_data_dictionary, get_rcdf_metadata - 0.1.02025-08-28
- RR 4.5.0 released · 2025-04-11
Package metadata
- First published
- 2025-08-28
- Total releases
- 7 / 1 yrs
- License
- MIT + file LICENSE OSI
- Minimum R
- ≥ 4.1.0
- Download size
- 71 KB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("rcdf")This package has no citation file, so citation() returns this entry built from DESCRIPTION.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-19, which the citation names so these numbers can be found later. More on citing and the projects behind them.