R Observatory: Metrics for pysparklyr version 0.2.2

Balamuta, James Joseph

doi:10.5281/zenodo.21843040

pysparklyr

0.2.2Package record

Provides a 'PySpark' Back-End for the 'sparklyr' Package

0packages depend

24Kdownloads / year

0.0%test coverage

13/13flavors OK

Overview

About

Maintained by Edgar RuizCRAN page ↗GitHub ↗Issues ↗Declared funder: Posit Software, PBC

It enables 'sparklyr' to integrate with 'Spark Connect', and 'Databricks Connect' by providing a wrapper over the 'PySpark' 'python' library.

Changes in 0.2.2

New

Adds support for tune_grid_spark(). It enables running a Tidymodels tune

grid inside Spark Connect clusters.

Snowflake connections now support all native authenticators via the Snowflake Python SDK, which also picks up on default credentials from Snowflake connections.toml files. Viewer-based credentials on Posit Connect are now supported, too (#181 - @atheriel).

Improvements

Databricks Connect now auto-detects the latest library version from PyPI when

no version parameter is specified. When the auto-detected version differs from

the cluster's DBR version, a warning is displayed with suggestions for ensuring

version compatibility.

Show all changes in 0.2.2Hide the rest of the changes in 0.2.2

Adds profile argument support to the Databricks SDK connection call.
When no cluster version is provided, uses the latest available main library

version from PyPI.

Fixes

Restores compatibility with sparklyr 1.9.5 / dbplyr 2.6.0, which restructured the tbl source slot. Operations on a tbl (e.g. printing, collect()) no longer fail with "no applicable method for 'invoke' applied to an object of class 'c('spark_connection', 'DBIConnection')'". Two-table verbs such as left_join() no longer error with "x and y must share the same source", and the deprecated dbplyr::as.sql() call was replaced with as_table_path() (#185). This raises the minimum requirements to sparklyr >= 1.9.5 and dbplyr >= 2.6.0.
No longer emits spurious reticulate "After Python has initialized, only action = 'add' ..." warnings when opening additional connections in a session that uses an ephemeral (uv) Python environment.
Fixes conversion of Pandas NULL columns and date types (#178 - @tobiasdut)

From NEWS.md in the release

Install

R (CRAN)pak (dev)Ubuntu (r2u)Fedora (COPR)openSUSE (OBS)

install.packages("pysparklyr", repos = "https://cloud.r-project.org/")

Installs the current CRAN release. Prebuilt binaries on Windows and macOS.

if (!requireNamespace("pak", quietly = TRUE))
  install.packages("pak", repos = "https://cloud.r-project.org/")
pak::pak("mlverse/pysparklyr")

Development version from mlverse/pysparklyr. Installs pak first if it is not already available.

Release

# enable cranapt (r2u) if it is not already set up, then install
grep -rqs r2u.stat.illinois.edu /etc/apt/sources.list /etc/apt/sources.list.d/ \
  || curl -fsSL https://raw.githubusercontent.com/eddelbuettel/r2u/master/inst/scripts/add_cranapt_noble.sh | sudo bash
sudo apt install -y r-cran-pysparklyr

Fast Ubuntu binaries via r2u (no compilation). The cranapt setup runs once and is skipped when it is already enabled. We list this channel from recorded r2u download activity rather than from a live index of the repository, so treat it as last known rather than guaranteed.

bspm (used by r2u) also needs make: sudo apt install make

# enable the iucar/cran COPR if it is not already present, then install
ls /etc/yum.repos.d/*iucar*cran*.repo >/dev/null 2>&1 \
  || sudo dnf copr enable -y iucar/cran
sudo dnf install -y R-CRAN-pysparklyr

RPMs from the iucar/cran COPR, rebuilt from CRAN.

For install.packages() integration in R, also: sudo dnf install R-CoprManager

# add the autoCRAN OBS repo (Tumbleweed) if it is missing, then install
zypper -q lr | grep -qi autoCRAN \
  || sudo zypper addrepo https://download.opensuse.org/repositories/devel:/languages:/R:/autoCRAN/openSUSE_Tumbleweed/devel:languages:R:autoCRAN.repo
sudo zypper install -y R-pysparklyr

RPMs auto-built from CRAN on openSUSE OBS (Tumbleweed). We list this channel from recorded autoCRAN download activity rather than from a live index of the repository, so treat it as last known rather than guaranteed.

Health

CRAN checks

13OK

CRAN check page ↗

Slowest check: 2.7 min · r-oldrel-windows-x86_64

Status changes

OK13 OK2026-08-05
NOTE12 OK, 1 NOTE2026-08-01
OK13 OK2026-07-17
WARNING9 OK, 4 WARNING2026-06-09
ERROR8 OK, 4 WARNING, 1 ERROR2026-06-08

Show 1 earlier changeHide 1 earlier change

WARNING9 OK, 5 WARNING2026-03-10

Advisories

none known

How this is measured·How advisories are found

Documentation

In release 0.2.2

READMEYes · 95 wordsVignettesNoNEWSYes · 100% structured · mentions 0.2.2Citation fileNoWebsitenot declared in DESCRIPTION

Help pages: 9 (2 internal)includes a package overview page
Examples that run: no examples
Exported functions with examples: 0%
Documented parameters: 100%
Return-value docs: 100%
Help pages citing references: 0%

How these are measured

Downloads

24K

CRAN downloads in the past year

Rank #3,945 · ~66/day · ~2K/mo

Releases R versions

Daily download trend is not available in this view yet.

84830 days

2.3K90 days

24K1 year

Compare downloads with other packages →

Also on201 r2u8 autocran

How downloads are counted

Repository

mlverse/pysparklyr ↗Organization: mlverse

18Stars

10Forks

15Open issues

4Open PRs

12Releases

1,305Commits

9Commit identities

databrickspysparkrsparkspark-connect

1,305 commits · Last activity 2026-10-02 · +5.9% stars, 30d · Default branch at 0.2.2.9000, ahead of CRAN 0.2.2 (checked 2026-10-11)

Responsiveness

How these are sampled

PRs merged (all time): 99%
Median days to close, 50 most recently updated closed issues: 6
Median days to close, 50 most recently updated closed PRs: 0

Stars over time

2025-04-23 · 162026-10-08 · 18

How these are measured

PracticesBeta

10 development-tooling and community-health practices detected across 6 families in the upstream repository

Checks run against github.com/mlverse/pysparklyr on 2026-10-11.

Continuous integration (2)

GitHub ActionsR CMD check workflow

The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel

Governance and community (1)

Code of conduct

Docs source (1)

README.Rmd

Coverage (1)

Coverage reporting (Codecov)

Show all practicesHide 2 practices

CRAN release process (2)

cran-comments.mdCRAN-SUBMISSION

Lint, format, editor (3)

AirVS CodeRStudio project

How this is detected·Detection ruleset v3 (2026-09-26)

Dependencies

Declared dependencies

34 external dependencies (excludes base and recommended)

Installing pysparklyr adds 53 packages (32 with compiled code, 35 MB of source tarballs). Heaviest: stringi 11 MB, arrow 4.2 MB, sparklyr 2.5 MB.

Depends (1)

R >= 4.1

Imports (22)

arrow cli DBI dbplyr dplyr fs glue httr2 lifecyclemethodsprocessx purrr reticulate rlang rstudioapi sparklyr tidyr tidyselect vctrs uuid withr connectcreds

LinkingTo (0)

none

Suggests (13)

vcr crayon R6 testthat tibble rsconnect rsample workflows tune parsnip dials tailor recipes

Enhances (0)

none

Dependency license breakdown →

Reverse dependencies

0direct

0indirect

Nothing depends on this yet.

How these are counted

Code & Tests

People & History

People (2)

Edgar Ruiz

Author, Maintainer since 0.1.0 (2023)

Posit Software, PBC

Copyright holder, Funder

Releases (13)

Pick two releases to compare their code metrics.

0.2.2Latest
record
2026-07-16 · current release · diff ↗
R
R 4.6.0 released · 2026-04-24
0.2.1
record
2026-04-20 · diff ↗
0.2.0
record
2026-01-27 · diff ↗ ·
removed 1 export
%>%
0.1.9
record
2025-10-07 · diff ↗
0.1.8
record
2025-05-19 · diff ↗
R
R 4.5.0 released · 2025-04-11
0.1.7
record
2025-03-24 · diff ↗ ·
added 1 export
pyspark_config
0.1.6
record
2025-02-14 · diff ↗
0.1.5
record
2024-06-24 · diff ↗
R
R 4.4.0 released · 2024-04-24
0.1.4
record
2024-04-16 · diff ↗
0.1.3
record
2024-01-12 · diff ↗ ·
added 2 exports
deploy_databricks, requirements_write
0.1.2
record
2023-12-06 · diff ↗ ·
added 2 exports
connection_databricks_shinyapp, ml_prepare_dataset
0.1.1
record
2023-11-10 · diff ↗
0.1.0
record
2023-10-04

Show 1 earlier eventHide 1 earlier event

R
R 4.3.0 released · 2023-04-21

Package metadata

First published: 2023-10-04
Total releases: 13 / 3 yrs
License: MIT + file LICENSE OSI
Minimum R: ≥ 4.1
Download size: 147 KB

Cite

Cite this package

Run in R for the authors' preferred citation:

citation("pysparklyr")

Format

Ruiz, E., & Posit Software, PBC. (2026). pysparklyr: Provides a 'PySpark' Back-End for the 'sparklyr' Package (Version 0.2.2) [Computer software]. https://doi.org/10.32614/CRAN.package.pysparklyr

This package has no citation file, so citation() returns this entry built from DESCRIPTION.

Cite the R Observatory

For a number measured here: a download total, a coverage figure, an archival date.

Format

APA

Balamuta, J. J. (2026). R Observatory: Metrics for pysparklyr version 0.2.2 [Data set]. HJJB, LLC. Data release v2026-10-18. https://doi.org/10.5281/zenodo.21843040

From data release v2026-10-18, which the citation names so these numbers can be found later. More on citing and the projects behind them.

Report a problem with this page →