Test coverage
Overview
Test coverage is measured by downloading each CRAN package's current source tarball, installing its dependencies, building it, and running its test suite under covr.
Test coverage
Line coverage
71%
Expression
68.9%
Tests / Examples
71.4% / 34% ex
Functions
9 4 exported
Complexity
5.6 avg / 11 max
Call network
9 nodes / 9 edges
Call graph
Lowest coverage
9 functions| Function | Cyclo | Coverage |
|---|---|---|
| retry_io | 8 | 0% |
| warn_encoding | 2 | 12% |
| check_locale | 6 | 41% |
| str_trim_ws exp | 4 | 52% |
| str_pad_both exp | 5 | 64% |
| str_wrap exp | 7 | 88% |
An illustrative package with invented numbers. The first three cells come from covr, the last three from static analysis.
Only CRAN packages are measured. Many CRAN packages show no percentage, most often because the package contains no tests. A blank here is a recorded outcome, not missing data.
Measurement process
Collection runs every six hours. The site updates once a day. Tests run with NOT_CRAN=true, so tests that normally skip on CRAN will run here. The resulting number may differ from CRAN's own check results.
Up to three passes are run. The tests pass always runs. Examples and vignettes run only if the tests pass returned ok. Each pass is capped at thirty minutes. A tests pass that exceeds the cap is recorded as a timeout. An examples or vignettes pass that exceeds it is silently dropped.
Only the current version of a package is measured. When CRAN publishes a new version, coverage is re-measured. Older rows are kept but not backfilled.
Outcome codes
Every run records one outcome code. The code page displays the outcome whenever it is not ok.
- ok
- The suite ran successfully under covr. Nearly all of these carry a percentage.
- no_tests
- No .R file found under tests/ in the published tarball.
- test_error
- A test file threw an error. Most of these still carry a partial percentage.
- build_fail
- The package or one of its dependencies failed to install.
- timeout
- The tests pass exceeded the thirty-minute cap.
- covr_error
- covr itself encountered an error.
Only ok and test_error outcomes carry a coverage percentage. On package pages, all non-measured cases display as a dash or not tracked with no further distinction.
Coverage below 1% is possible and displayed as less than 1 percent when it falls below 0.5%.
Coverage figures
Several coverage figures appear on a package page:
- Line coverage is covr's percentage over the full coverage object, including compiled sources. The per-file and per-function tables cover R code only, so the headline and the breakdown may differ for packages with compiled code.
- Expression coverage counts traced expressions rather than lines, using the same scope.
- Tests and Line coverage are the same value rounded differently (0 vs. 1 decimal).
- Examples and Vignettes come from separate passes and appear only on ok runs.
- Per-function coverage merges covr's tally with the static analyzer's function records. Unmatched functions show as n/a rather than zero.
Interpreting coverage
A coverage percentage measures which lines ran, not whether the tests checked the results. A test that calls a function without any assertions contributes the same coverage as one with thorough checks.
Zero means covr traced the code and none of it executed. A dash means coverage was not measured, and the reason appears on the code page.
The color scale (red below 50%, amber below 80%, green at 80% and above) is a reading aid, not an external standard. The call graph uses a continuous ramp, so the same function may appear differently in two places on one page.
Comparing coverage across packages requires care, since differences in compiled code, missing suggested packages, and timeout cutoffs all affect the headline number. Coverage is CRAN only (Bioconductor packages on both registries are measured as CRAN). A test_error percentage reflects only what ran before the failure, and test errors are not retried until the next release.