Scaffold and Submit Computational Jobs to HTC Schedulers
Overview
Provides scaffolding tools to help researchers prepare and submit computational jobs to high-throughput computing (HTC) schedulers. Generates the files required to run containerized R analyses on 'HTCondor', including submit files and executable scripts, and wraps the system commands needed to stage files, submit jobs, monitor status, and retrieve results from a CHTC submit node. Provides 'htc_config()' for managing connection details and SSH connection reuse guidance. Works naturally alongside 'containr' for container image management and 'toolero' for dataset splitting and project scaffolding.
Initial release
submitr is the third package in the **From the Notebook to the Cluster**
family, alongside toolero and containr. It provides a workflow for
submitting containerized R analyses to the UW-Madison Center for High
Throughput Computing (CHTC) from inside R.
New functions
Connection management
htc_config()-- create or read a project-levelhtc.cfgconfiguration file. On first use, prompts interactively for username and server, displays ControlMaster SSH setup guidance to reduce Duo MFA prompts, writeshtc.cfg, and adds it to.gitignore. Subsequent calls read the existing file and validate server reachability. Returns a named list withusernameandserver. Errors informatively whenusernameorserverare supplied as empty strings.
Show all changes in 0.1.0Hide the rest of the changes in 0.1.0
Job scaffolding
htc_gen_submit()-- generate an HTCondor.subsubmit file from project parameters. Supports single-job and multiple-job modes. Multiple mode reads a manifest fromtoolero::write_by_group(manifest = TRUE), extracts filenames, writessubdatasets.csv, and emitsqueue file from subdatasets.csv. Resource presets (small,medium,large,custom) are loaded at runtime frominst/extdata/htc-resources.yaml; a local./htc-resources.yamltakes precedence over the package default. GPU support viagpu = TRUEandgpu_options.comments = TRUEannotates each section of the generated file with explanatory text.htc_gen_executable()-- generate the.shexecutable script that HTCondor runs inside the container. Produces a four-element script: shebang,mkdir,Rscript, andtar. In multiple-job mode, passes${1}as a positional argument to the R script.r_scriptmust be supplied explicitly -- there is no default.set_executable = TRUE(default) sets executable permissions viaSys.chmod().
File transfer and job control
htc_upload()-- copy files to the CHTC submit node viascp. Accepts single files, vectors of files, directories (transferred recursively), and glob patterns.remote_pathdefaults to"~/".dry_run = TRUEpreviews the command without executing it.htc_submit()-- runcondor_submiton the remote submit node via SSH from the remote directory where files were uploaded. Returns the cluster ID invisibly for use withhtc_status(). Supportsdry_run = TRUE.htc_status()-- check job progress viacondor_q. Optionally filters by cluster ID.watch = TRUEpolls atintervalseconds (default 60) until the cluster ID leaves the queue. Returnscondor_qoutput invisibly as a character vector. Supportsdry_run = TRUE.htc_download()-- copy result files back from the submit node viascp. Supports single filenames, vectors of filenames, and glob patterns ("*.tar.gz","job.*"). Glob patterns are single-quoted to prevent local shell expansion.local_pathdefaults to".". Supportsdry_run = TRUE.
Package infrastructure
inst/extdata/htc-resources.yamlships with the package and provides default resource presets forhtc_gen_submit().inst/extdata/hello-world.subandinst/extdata/hello-world.shincluded as test files for end-to-end workflow verification.inst/extdata/sample.Rincluded as a sample R script for use in examples.
Testing
The test suite uses a three-layer strategy to handle the fact that end-to-end
testing requires a live HTCondor environment and SSH access. Layer 1 covers
argument validation. Layer 2 covers command construction using dry_run = TRUE
and mocked bindings. Layer 3 integration tests are opt-in via
Sys.setenv(CHTC_USERNAME = "your.netid") and never run on CRAN or CI.
153 tests passing across seven test files.
From NEWS.md in the release
Install
Health
- OK13 OK2026-08-04
- NOTE12 OK, 1 NOTE2026-08-01
- OK7 OK2026-05-20
Documentation
- Help pages
- 8 (1 internal)includes a package overview page
- Examples that run
- 29%2 run in a plain check · 5 only when donttest examples are enabled (R CMD check --as-cran)
- Exported functions with examples
- 100%
- Documented parameters
- 100%
- Return-value docs
- 100%
- Help pages citing references
- 0%
Downloads
Repository
- PRs merged (all time)
- not measured
- Median days to close, 50 most recently updated closed issues
- 16
- Median days to close, 50 most recently updated closed PRs
- not measured
Stars over time
Releases over time
Issues over time
Commits over time
10 development-tooling and community-health practices detected across 7 families in the upstream repository
Checks run against github.com/erwinlares/submitr on 2026-10-22.
The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel
GitHub Pages site last deployed 2026-10-01 · The repository has a site deploy workflow
pkgdown site last built 2026-10-01
Show all practicesHide 3 practices
Dependencies
Code & Tests
People & History
- 0.1.0Latest2026-05-19 · current release
- RR 4.6.0 released · 2026-04-24
Package metadata
- First published
- 2026-05-19
- Total releases
- 1 / 1 yrs
- License
- MIT + file LICENSE OSI
- Download size
- 3.4 MB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("submitr")This package has no citation file, so citation() returns this entry built from DESCRIPTION.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-25, which the citation names so these numbers can be found later. More on citing and the projects behind them.