R Observatory: Metrics for crawlee version 0.1.1

Balamuta, James Joseph

doi:10.5281/zenodo.21843040

crawlee

0.1.1Package record

Tidy Interface for Reproducible Web Crawling

0packages depend

1.2Kdownloads / year

55.7%test coverage

13/13flavors OK

Overview

About

Maintained by Andre LeiteCRAN page ↗GitHub ↗Website ↗Issues ↗

A tidy, pipe-friendly toolkit for reproducible web crawling and structured data collection, inspired by the architecture of the 'Crawlee' library. Provides a unified crawler with a deduplicating, resumable request queue, content-type aware handlers, structured storage backends and rich console logging via 'cli'. Supports crawling HTML pages, sitemaps, RSS and Atom feeds and PDF documents, with optional headless-browser rendering and helpers for retrieval-augmented generation.

Changes in 0.1.1

Removed 'polite' from Suggests. It was archived on CRAN on 2026-09-12 and was never used by the package.
Author metadata: fixed the spelling of Carlos Amorim and Marcos Wasiliew, added ORCID identifiers and added Júlia Nascimento Barreto as an author.

From NEWS.md in the release

Install

R (CRAN)pak (dev)Ubuntu (r2u)Fedora (COPR)openSUSE (OBS)

install.packages("crawlee", repos = "https://cloud.r-project.org/")

Installs the current CRAN release. Prebuilt binaries on Windows and macOS.

if (!requireNamespace("pak", quietly = TRUE))
  install.packages("pak", repos = "https://cloud.r-project.org/")
pak::pak("StrategicProjects/crawlee")

Development version from StrategicProjects/crawlee. Installs pak first if it is not already available.

Release

# enable cranapt (r2u) if it is not already set up, then install
grep -rqs r2u.stat.illinois.edu /etc/apt/sources.list /etc/apt/sources.list.d/ \
  || curl -fsSL https://raw.githubusercontent.com/eddelbuettel/r2u/master/inst/scripts/add_cranapt_noble.sh | sudo bash
sudo apt install -y r-cran-crawlee

Fast Ubuntu binaries via r2u (no compilation). The cranapt setup runs once and is skipped when it is already enabled. We list this channel from recorded r2u download activity rather than from a live index of the repository, so treat it as last known rather than guaranteed.

bspm (used by r2u) also needs make: sudo apt install make

# enable the iucar/cran COPR if it is not already present, then install
ls /etc/yum.repos.d/*iucar*cran*.repo >/dev/null 2>&1 \
  || sudo dnf copr enable -y iucar/cran
sudo dnf install -y R-CRAN-crawlee

RPMs from the iucar/cran COPR, rebuilt from CRAN.

For install.packages() integration in R, also: sudo dnf install R-CoprManager

# add the autoCRAN OBS repo (Tumbleweed) if it is missing, then install
zypper -q lr | grep -qi autoCRAN \
  || sudo zypper addrepo https://download.opensuse.org/repositories/devel:/languages:/R:/autoCRAN/openSUSE_Tumbleweed/devel:languages:R:autoCRAN.repo
sudo zypper install -y R-crawlee

RPMs auto-built from CRAN on openSUSE OBS (Tumbleweed). We list this channel from recorded autoCRAN download activity rather than from a live index of the repository, so treat it as last known rather than guaranteed.

Health

CRAN checks

13OK

CRAN check page ↗

Slowest check: 2.6 min · r-oldrel-macos-x86_64

Status changes

OK13 OK2026-08-05
NOTE12 OK, 1 NOTE2026-08-01
OK7 OK2026-07-04

Advisories

none known

How this is measured·How advisories are found

Documentation

In release 0.1.1

READMEYes · 332 wordsVignettesYes · 5 · dynamicNEWSYes · 100% structured · mentions 0.1.1Citation fileNoWebsiteYes · pkgdown

Help pages: 26 (1 internal)includes a package overview page
Examples that run: 60%9 run in a plain check · 6 never run
Exported functions with examples: 60%
Documented parameters: 93%
Return-value docs: 84%
Help pages citing references: 0%

Vignettes (5)

How these are measured

Downloads

1.2K

CRAN downloads in the past year

Rank #9,996 · ~3/day · ~100/mo

Releases R versions

Daily download trend is not available in this view yet.

41930 days

99890 days

1.2K1 year

Compare downloads with other packages →

Also on23 r2u9 autocran

How downloads are counted

Repository

strategicprojects/crawlee ↗Organization: StrategicProjects

2Stars

0Forks

0Open issues

0Open PRs

0Releases

28Commits

2Commit identities

crawleecrawlergovtechhttr2open-datarrstatsrvest

28 commits · Last activity 2026-09-29 · 0% stars, 30d

Responsiveness

not measured

Stars over time

2026-07-07 · 12026-09-17 · 2

How these are measured

PracticesBeta

7 development-tooling and community-health practices detected across 4 families in the upstream repository

Checks run against github.com/strategicprojects/crawlee on 2026-10-18.

Continuous integration (2)

GitHub ActionsR CMD check workflow

The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel

Docs source (3)

pkgdown

GitHub Pages site last deployed 2026-09-29 · The repository has a site deploy workflow

pkgdown site last built 2026-09-29, before release 0.1.1 (2026-09-30)

CRAN release process (1)

cran-comments.md

Lint, format, editor (1)

RStudio project

How this is detected·Detection ruleset v3 (2026-09-26)

AI coding toolsBeta

This repository shows signs of AI coding tools.

Claude Codefirst seen on or before 2026-06-27
Named in .Rbuildignore: .claude, CLAUDE.md
Named in .gitignore: .claude, CLAUDE.md
Commits: at least 28 crediting it in the message

Models named in its commits

Claude Opus 4.823 commits · Jun 2026
Claude Opus 5.55 commits · Sep 2026

Counts are commits, not lines of code. Commits by a tool's accounts count once across all of its accounts. Commits crediting it are the largest count among its credit lines, a minimum.How this is detected·Detection ruleset v4 (updated 2026-09-27)

Dependencies

Declared dependencies

22 external dependencies (excludes base and recommended)

Installing crawlee adds 23 packages (14 with compiled code, 7.8 MB of source tarballs). Heaviest: openssl 1.1 MB, jsonlite 1.0 MB, vctrs 1.0 MB.

Depends (1)

R >= 4.1.0

Imports (8)

cli httr2 R6 rlang rvest tibble vctrs xml2

LinkingTo (0)

none

Suggests (14)

arrow chromote DBI dplyr duckdb httptest2 jsonlite knitr later nanoparquet pdftools promises rmarkdown testthat

Enhances (0)

none

Dependency license breakdown →

Reverse dependencies

0direct

0indirect

Nothing depends on this yet.

How these are counted

Code & Tests

People & History

People (6)

Andre Leite

Author, Maintainer since 0.1.0 (2026)

Carlos Amorim

Author · added in 0.1.1

Diogo Bezerra

Author

Júlia Nascimento Barreto

Author · added in 0.1.1

Hugo Vasconcelos

Author

Marcos Wasiliew

Author · added in 0.1.1

Listed in earlier versions (2)

Carlos Amorin

no longer listed · 0.1.0

Marcos Wasilew

no longer listed · 0.1.0

Releases (2)

Pick two releases to compare their code metrics.

0.1.1Latest
record
2026-09-30 · current release · diff ↗
0.1.0
record
2026-07-03
R
R 4.6.0 released · 2026-04-24

Package metadata

First published: 2026-07-03
Total releases: 2 / 1 yrs
License: MIT + file LICENSE OSI
Minimum R: ≥ 4.1.0
Download size: 98 KB

Cite

Cite this package

Run in R for the authors' preferred citation:

citation("crawlee")

Format

Leite, A., Amorim, C., Bezerra, D., Nascimento Barreto, J., Vasconcelos, H., & Wasiliew, M. (2026). crawlee: Tidy Interface for Reproducible Web Crawling (Version 0.1.1) [Computer software]. https://doi.org/10.32614/CRAN.package.crawlee

This package has no citation file, so citation() returns this entry built from DESCRIPTION.

Cite the R Observatory

For a number measured here: a download total, a coverage figure, an archival date.

Format

APA

Balamuta, J. J. (2026). R Observatory: Metrics for crawlee version 0.1.1 [Data set]. HJJB, LLC. Data release v2026-10-21. https://doi.org/10.5281/zenodo.21843040

From data release v2026-10-21, which the citation names so these numbers can be found later. More on citing and the projects behind them.

Report a problem with this page →