npRmpi
0.70-5Parallel Nonparametric Kernel Smoothing Methods for Mixed Data Types Using 'MPI'
Overview
Nonparametric (and semiparametric) kernel methods that seamlessly handle a mix of continuous, unordered, and ordered factor data types. This package is a parallel implementation of the 'np' package based on the 'MPI' specification that incorporates the 'Rmpi' package (Hao Yu <hyu@stats.uwo.ca>) with minor modifications and we are extremely grateful to Hao Yu for his contributions to the 'R' community. We would like to gratefully acknowledge support from the Natural Sciences and Engineering Research Council of Canada (NSERC, https://www.nserc-crsng.gc.ca/), the Social Sciences and Humanities Research Council of Canada (SSHRC, https://www.sshrc-crsh.gc.ca/), and the Shared Hierarchical Academic Research Computing Network (SHARCNET, https://sharcnet.ca/). We would also like to acknowledge the contributions of the 'GNU GSL' authors. In particular, we adapt the 'GNU GSL' B-spline routine 'gsl_bspline.c' adding automated support for quantile knots (in addition to uniform knots), providing missing functionality for derivatives, and for extending the splines beyond their endpoints.
- Corrected the heteroskedasticity-robust Ichimura index-coefficient covariance returned by
vcov(npindex(..., gradients = TRUE))when the model has more than one free index coefficient. Link-gradient and residual weights are now applied observation by observation. Fits with one free coefficient remain exactly unchanged, as do beta, bandwidth, objective, fitted-value, residual, and gradient results. Serial and MPI results remain exactly equivalent, with no payload change. - Repaired finite-support bounded continuous-kernel normalization for all supported Gaussian, Epanechnikov, uniform, and truncated-Gaussian kernels. The centered normalization now avoids tail cancellation, uses the platform C99 error function where appropriate, and preserves the uniform-density limit as the bandwidth tends to infinity.
Show all changes in 0.70-5Hide the rest of the changes in 0.70-5
- Replaced the bounded distribution operator with the mathematically correct observation-centered truncated-kernel CDF. Analytic centered interval primitives and hoisted invariant work preserve exact support endpoints, numerical accuracy, the finite-support large-bandwidth uniform limit, and serial/MPI numerical equivalence.
- Corrected
transform.bounds = TRUEinitialization for regression initial starts and multistarts and for later conditional-distribution multistarts. External bandwidth starts are now inverse-mapped before transformed Powell search; the public default remainsFALSEand MPI payloads are unchanged. - Clarified optimizer summaries so objective-cache lookups, NOMAD point lookups, family-native R-
optimrefinement, and total function-evaluation accounting are labelled consistently without implying that their denominators are interchangeable. - Clarified bandwidth-object replay documentation across estimator families. Stored bandwidth objects retain search and fit metadata but do not silently materialize or expand training data; callers must provide data again where the documented replay route requires it.
- Hardened MPI lifecycle and protocol handling. Bootstrap fanout timeout or failure now reports a hard failure and instructs the user to restart R before further MPI-backed computation; attach close resets pool state, profile communicator metadata remains distinct, and internal protocol tags are allocated portably.
- Standardized the public
nomad.optscontract across supported bandwidth selectors and hardened native NOMAD callback cleanup and allocation guards without changing MPI message payloads. - Refreshed MPICH build and runtime guidance for current MacPorts layouts and documented the supported session, attach, and profile/manual-broadcast launch modes. Open MPI remains outside the validated 0.70-5 backend matrix.
- Reworked
npindexbw()/npindex()internals after 0.70-4. Ichimura and Klein-Spady single-index objectives now reuse the establishednpregleave-one-out backend where applicable, preserving the public estimator contract while materially improving high-dimensional and local-polynomial objective evaluation. MPI-aware objective services now keep the root-owned optimizer and worker evaluation protocols explicit. - Repaired MPI single-index NOMAD/Powell handoff behavior for
npindex(..., nomad = TRUE)and related bandwidth searches. The Powell refinement stage now uses the same one-active-protocol discipline as the validated MPI search routes, avoiding worker/master desynchronization during the NOMAD-to-Powell transition. - Distributed post-CV
npindex()fitting, evaluation, gradients, and related components across owned evaluation rows in active MPI sessions while keeping the full training data available where kernel smoothing requires it. Focused large-sample checks showed the expected scaling improvement and preserved serial-equivalent numerical results. - Added user documentation for
npindex()optimizer choice and beta interpretation. The documentation now gives practical guidance on when derivative-free Nelder-Mead remains a reasonable low-dimensional default and when BFGS is useful for higher-dimensional index searches, and it explains the relative interpretation of normalized single-index beta coefficients. - Hardened single-index fit, evaluation, plotting, service, and summary behavior. Bounded continuous-kernel options are now carried consistently through objective, fit, evaluation, variance, and bootstrap routes; Klein-Spady confusion-matrix output is guarded against out-of-range fitted values; and large-bandwidth shortcuts are restricted to kernel/order combinations whose constant-weight approximation was validated.
- Repaired single-index plot bootstrap memory usage. Wild-bootstrap self-maps now avoid public-facing
O(n^2)kernel-weight allocations for the ordinary large-sample plot route, while preserving public fitted/plotting contracts and MPI row ownership. - Repaired generalized-nearest-neighbor local-polynomial derivative ownership across
npreghat()and related public routes. Mixed-degree local-polynomial fits now route available derivative components through the correct owner, preserveapply == H %*% ycontracts, and report unavailable derivative components consistently rather than silently applying the wrong operator. - Repaired local-constant derivative ownership in
npreghat()so degree-zero local-polynomial derivative requests use the analytic local-constant derivative contract only when that contract is mathematically available. Scalar, matrix, multi-column, serial-equivalent, and active-pool error routes were validated separately. - Improved conditional mixed-degree local-polynomial gradients for
npcdens()/npcdist()and public conditional-gradient accessors. The partial-availability contract now matches the regression-family policy: available components are returned, unavailable components are represented asNA, and incoherent metadata fails clearly. - Repaired MPI conditional-density native derivative workspace handling so no-gradient and gradient calls can be mixed safely in one session without stale native derivative workspace state affecting later calls.
- Repaired exact nearest-neighbor bootstrap parity in MPI conditional-density and conditional-distribution plot routes. Adaptive and generalized nearest-neighbor exact bootstrap paths now rebuild bandwidth state on the expanded resample consistently with the serial package.
- Repaired formula/data reentry contracts across density, distribution, conditional density, conditional distribution, single-index, smooth coefficient, partially linear, quantile, conditional-mode, copula, and significance-test routes. Explicit estimator
data=now overrides stored bandwidth-object data where that public call shape is supported, formulanewdatais validated against fitted RHS variables, and direct formula calls with numeric smoothing parameters no longer misroute formula objects as native data before autodispatch. - Hardened MPI autodispatch argument transport and bandwidth fingerprints. Named arguments materialized through S3 dispatch are preserved before worker fanout, and bandwidth-object reuse checks are stronger against stale remote object reuse.
- Repaired unconditional density and distribution edge contracts. Ordered kernel-code selection is now consistent between conditional bandwidth selection and fitting where normalization is required, all-NA input is rejected before native calls, training and evaluation omission metadata are retained separately where needed, categorical zero-bandwidth standard-error handling is consistent across categorical configurations, and unsupported bandwidth-selection method codes fail clearly.
- Clarified normal-reference bandwidth documentation for density and distribution routes. The rule-of-thumb formulas are documented as fast exploratory Silverman-style heuristics, not production substitutes for cross-validation or likelihood-based selection.
- Hardened
npcdistbw()normal-reference method handling so the R method code and native C method code stay aligned, avoiding accidental fall-through to an unintended bandwidth-selection branch. - Repaired and hardened public estimator contracts found during adversarial audits of
npscoef,npplreg,npqreg,nplsqreg,npconmode,npcopula,npreg,npcdens,npcdist,npudens, andnpudist. Repairs includenpscoefiterated backfit behavior and MPI service-task handling, partially-linear fit reentry, quantile inversion/clamping contracts, conditional-mode probability/tie handling and orchestrator localization, copula sample reentry, regression tree/large-h predicate alignment, and density/distribution formula/native argument consistency. - Plot-bootstrap memory hardening now covers single-index, conditional-mode, and partially-linear plot routes. The default plot evaluation grids remain linear in the training sample size, and explicit
neval == ntrainstyle requests remain user-controlled. - Native shadow-object and cache lifecycles were hardened. Conditional-density shadow pointers, native objective-cache state, and regression large-h / large-lambda caches are cleared at the appropriate top-level lifecycle boundary so pointer-keyed helper state cannot leak across independent calls, datasets, or MPI sessions.
npksum()numeric-bandwidth dispatch now constructs data-aware bandwidth objects for mixed continuous/categorical data instead of falling back to a default-typed object. The serial and MPI packages now share the same public behavior for numeric smoothing parameters.- The local-polynomial regression CVKS low-support objective path now uses a named bandwidth-method code and aligns the R and C routing contracts, reducing the risk of drift between search metadata and native objective handling.
- Expanded focused tests, demos, benchmarks, and release-protocol sentinels across the public exported surface. The release protocol now requires an explicit public-exported-surface inventory, estimator-family sentinels, documentation/demo/benchmark smoke coverage, MPI scaling canaries, and installed package proof before release-ready claims.
From NEWS.md in the release
Install
Health
ERRORwhether package can be installedr-oldrel-macos-arm64, r-oldrel-macos-x86_64, r-release-macos-arm64, r-release-macos-x86_64
r-oldrel-macos-arm64
Installation failed. See ‘/Volumes/Builds/packages/big-sur-arm64/results/4.5/npRmpi.Rcheck/00install.out’ for details.
r-oldrel-macos-x86_64
Installation failed. See ‘/Volumes/Builds/packages/big-sur-x86_64/results/4.5/npRmpi.Rcheck/00install.out’ for details.
r-release-macos-arm64
Installation failed. See ‘/Volumes/Builds/packages/sonoma-arm64/results/4.6/npRmpi.Rcheck/00install.out’ for details.
r-release-macos-x86_64
Installation failed. See ‘/Volumes/Builds/packages/big-sur-x86_64/results/4.6/npRmpi.Rcheck/00install.out’ for details.
WARNINGwhether package can be installedr-devel-linux-x86_64-debian-gcc
Found the following significant warnings: Rmpi.c:293:9: warning: ‘MPI_Info_get’ is deprecated: MPI_Info_get was deprecated in MPI-4.0; use MPI_Info_get_string instead [-Wdeprecated-declarations] See ‘/home/hornik/tmp/R.check/r-devel-gcc/Work/PKGS/npRmpi.Rcheck/00install.out’ for details. * used C compiler: ‘gcc-16 (Debian 16.2.0-3) 16.2.0’
- ERROR8 OK, 1 WARNING, 4 ERRORNow
- ERROR9 OK, 5 ERROR2026-03-10
Documentation
- Help pages
- 75 (1 internal)includes a package overview page
- Examples that run
- 2%1 run in a plain check · 59 never run
- Exported functions with examples
- 80%
- Documented parameters
- 90%
- Return-value docs
- 98%
- Help pages citing references
- 71%
Also includes 81 demo scripts (run with demo())
Downloads
Repository
- PRs merged (all time)
- 86%
- Median days to close, 50 most recently updated closed issues
- 7
- Median days to close, 50 most recently updated closed PRs
- 0
Stars over time
Forks over time
Issues over time
PRs over time
Commits over time
Checks run against github.com/jeffreyracine/r-package-np on 2026-10-11.
No development-tooling practices detected in the upstream repository.
Dependencies
Code & Tests
Datasets
People & History
Pick two releases to compare their code metrics.
- 0.70-5Latest
- 0.70-42026-06-26 · diff ↗
- 0.70-3
- 0.70-2
- 0.70-12026-05-01 · diff ↗ ·
removed 95 exports
mpi.abort, mpi.allgather, mpi.allgather.Robj, mpi.allgatherv, mpi.allreduce, mpi.any.source, mpi.any.tag, mpi.apply, mpi.applyLB, mpi.bcast.Rfun2slave, mpi.bcast.data2slave, mpi.cancel, mpi.cart.coords, mpi.cart.create, mpi.cart.get, mpi.cart.rank, mpi.cart.shift, mpi.cartdim.get, mpi.comm.c2f, mpi.comm.disconnect, mpi.comm.get.parent, mpi.comm.is.null, mpi.comm.maxsize, mpi.comm.remote.size, mpi.comm.set.errhandler, mpi.comm.spawn, mpi.comm.test.inter, mpi.dims.create, mpi.exit, mpi.finalize and 65 moreadded 12 exports
dimBS, mpi.get.version, npRmpi.init, npRmpi.quit, npudenshat, npudisthat, npreghat, npindexhat, npplreghat, npscoefhat, npcdenshat, npcdisthat - RR 4.6.0 released · 2026-04-24
- unarchivedReturned to CRAN2026-02-16
- 0.60-202026-02-16 · diff ↗ ·
added 8 exports
npRmpi.start, npRmpi.stop, npRmpi.session.info, npuniden.boundary, npuniden.reflect, npuniden.sc, np.pairs, np.pairs.plot - RR 4.5.0 released · 2025-04-11
- RR 4.4.0 released · 2024-04-24
- RR 4.3.0 released · 2023-04-21
- RR 4.2.0 released · 2022-04-22
- RR 4.1.0 released · 2021-05-18
- RR 4.0.0 released · 2020-04-24
- RR 3.6.0 released · 2019-04-26
- RR 3.5.0 released · 2018-04-23
Show 28 earlier eventsHide 28 earlier events
- RR 3.4.0 released · 2017-04-21
- RR 3.3.0 released · 2016-05-03
- RR 3.2.0 released · 2015-04-16
- archivedRemoved from CRAN2015-03-13misuse of '\donttest' was not corrected
- 0.60-22014-06-28 · diff ↗
- 0.60-02014-06-01 · diff ↗ ·
removed 1 export
mpichhostsadded 4 exports
mpi.bcast.Rfun2slave, mpi.bcast.data2slave, npquantile, nptgauss - RR 3.1.0 released · 2014-04-10
- 0.50-1.12014-01-01 · diff ↗
- RR 3.0.0 released · 2013-04-03
- 0.50-12013-03-14 · diff ↗ ·
removed 9 exports
.mpi.serialize, .mpi.slave.apply, .mpi.slave.applyLB, .mpi.slave.exec, .mpi.slave.sim, .mpi.unserialize, bin.nchar, mpi.init.sprng, mpi.setup.sprngadded 8 exports
.mpi.worker.exec, .mpi.worker.apply, .mpi.worker.sim, .mpi.worker.applyLB, mpi.comm.c2f, npudistbw, npcdistbw, npcopula - RR 2.15.0 released · 2012-03-30
- 0.40-13
- 0.40-122011-11-25 · diff ↗
- RR 2.14.0 released · 2011-10-31
- 0.40-112011-10-25 · diff ↗
- 0.40-102011-10-24 · diff ↗
- 0.40-92011-07-30 · diff ↗
- 0.40-72011-06-09 · diff ↗
- 0.40-6
- 0.40-52011-04-26 · diff ↗
- RR 2.13.0 released · 2011-04-13
- 0.40-42011-01-21 · diff ↗
- RR 2.12.0 released · 2010-10-15
- 0.40-32010-07-25 · diff ↗
- 0.40-22010-06-10 · diff ↗
- 0.40-02010-05-26 · diff ↗
- 0.30-92010-05-18
- RR 2.11.0 released · 2010-04-22
Package metadata
- First published
- 2010-05-18
- Total releases
- 23 / 16 yrs
- License
- GPL
- Minimum R
- ≥ 3.5.0
- Bundled data
- 101 KB / 5 files
- Download size
- 1.8 MB
Cite
Cite this package
Run in R for the authors' entry:
citation("npRmpi")The package provides a citation file, so citation() returns the authors' own entry.
It points to 10.18637/jss.v027.i05.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-17, which the citation names so these numbers can be found later. More on citing and the projects behind them.