fastml
0.7.10Guarded Resampling Workflows for Leakage-Aware Machine Learning in R
Overview
Provides a guarded resampling workflow for training and evaluating machine-learning models. When the guarded resampling path is used, preprocessing and model fitting are re-estimated within each resampling split to reduce leakage risk. Supports multiple resampling schemes, integrates with established engines in the 'tidymodels' ecosystem, and aims to improve evaluation reliability by coordinating preprocessing, fitting, and evaluation within supported workflows. Offers a lightweight AutoML-style workflow by automating model training, resampling, and tuning across multiple algorithms, while keeping evaluation design explicit and user-controlled.
Breaking changes
- **
time_varying_coxhas been withdrawn.** It required start-stop interval input, which the evaluation path does not accept, so the model could be fitted but never scored. It no longer appears inavailableMethods("survival"), and a request for it by name now stops with an error naming the reason and pointing tocox_phandstratified_cox. Advertising a learner that cannot produce a result is worse than not offering it. Code that named it previously received a fitted model with no metrics. - **Requesting resampling or tuning for
xgboost_aftis now an error rather than a warning.** parsnip exposes nocensored regressionmode for xgboost, so the family cannot be fitted through a workflow. Previously the request warned and then fitted natively with resampling silently disabled, which returned afastmlobject carrying no fold-level results for a user who had explicitly asked for them. The request is now refused. This is the same contract every other natively fitted survival family already followed. - **Values in
tune_paramsare now candidate values, not the endpoints of a range.** A vector of length two was previously read as an interval and interpolated acrossgrid_levels, while a vector of length three or more was read as an explicit set. Nothing but the length distinguished them, so a single call could containmixture = c(0, 0.5, 1)meaning three values andmin_n = c(20, 50)meaning an interval, which is not what the code appears to say. Every value supplied throughtune_paramsis now searched as written, whatever the number of values, and the parameter's range is widened where a requested value falls outside a package default. The package's own default ranges are unchanged and are still interpolated, so results obtained without atune_paramsargument are unaffected. - **
bootstrap_cinow defaults toFALSE.** The interval it produces resamples the held-out predictions of a single fitted model, so it describes uncertainty from the evaluation sample conditional on that fit and excludes training, tuning and model-selection variability. Read as ordinary uncertainty about model performance, which is how a reader of a.lower/.uppercolumn will read it, it overstates precision. Computing it also dominated run time, accounting for about 85% of the elapsed time of a Cox fit to therotterdamdata, which falls from roughly 19 seconds to roughly 2.9 seconds with it disabled, andsummary()did not display the result unlessshow_ci = TRUEwas passed, so the cost was paid on every fit for output that was not shown. Setbootstrap_ci = TRUEto restore the previous behaviour. With it off,.lowerand.upperareNAand.n_bootis zero, so nothing in the performance table reads as an interval that was not computed. When intervals are displayed,summary()now prints the estimand above the table so the qualification travels with the numbers. - **Regression results now report
rsq_tradalongsidersq.**yardstick::rsq()is squared Pearson correlation and is invariant to location and scale, so it cannot see a prediction that is biased or mis-dispersed.rsq_trad()is one minus the residual sum of squares over the total sum of squares, which is the quantity most readers assume when they see an R-squared in a results table. The two agree when predictions are unbiased and correctly scaled and diverge when they are not, so the gap between them is a calibration diagnostic. Code that reads the regression metrics table by column position will need updating.
New features
- **The tuning grid that was actually searched is recorded and printed.** The fitted object gains a
tuning_gridcomponent listing, per algorithm and engine, each tuned parameter, the values that were searched on the parameter's natural scale, and whether the parameter came from the user or from the package defaults.summary()prints it under the newtype = "grid", which is included in the defaulttype = "all". Reporting only the selected configuration cannot reveal a grid that differs from the one requested, because a selected value looks identical whether the user named it or the package interpolated it, which is how the scale defect below survived.?fastmldocuments the scale and the range semantics oftune_paramsexplicitly rather than leaving them to be inferred. - **
stratified_coxstrata are now named through astrata_colsargument.** Strata were previously identified by a naming convention, any column whose name began withstrata. A convention that determines model structure but appears nowhere in the function signature is not discoverable, and a user who named a columnstratum_sitegot an unstratified fit with no indication.fastml(..., strata_cols = "site")is now the documented interface; thestrataprefix is still recognised, so existing code is unaffected. Naming a column that is not present in the data is an error, and requestingstratified_coxwith no strata at all is an error rather than a silent skip.
Bug fixes
- **Tuning values supplied through
tune_paramsare no longer reinterpreted as logarithms.**dialsstores quantitative parameters on their transformed scale, which forpenalty,learn_rate,loss_reduction,cost,cost_complexityandrbf_sigmais base-10 logarithmic. User-supplied values were passed todials::range_set()on the natural scale without being mapped, solearn_rate = c(0.05, 0.1)was fitted as10^0.05 = 1.122to10^0.1 = 1.259, andpenalty = c(0.01, 0.1)as1.023to1.259. Nothing in the output reported the substitution. For boosting the consequence is severe, since a learning rate above one overshoots at every step: on an NHANES blood-pressure regression the fitted LightGBM predictions had 1.43 times the standard deviation of the outcome and spanned 36 to 294 mmHg, giving an RMSE of 23.1 against 17.3 for predicting the mean. The reportedrsqof 0.19 did not reveal this, becauseyardstick::rsq()is squared Pearson correlation and is invariant to location and scale; the traditional R-squared was -0.79. Values supplied by the user are now mapped through the parameter's own transform, and a value outside a transformed parameter's domain is refused with an error naming the parameter. Values are now mapped through the parameter's own transform before they reachdials, and a negative value for a positive-domain parameter is refused with an error naming the parameter. A requested zero, which is meaningful forloss_reductionbut has no logarithm, maps to the parameter's lower bound and the resulting value is visible in the recorded grid. The same route is taken by the fixed defaults fromget_default_params(), which are also written on the natural scale and were also being exponentiated. A user who supplied anytune_paramsfor LightGBM without naminglearn_ratewas given a learning rate of10^0.1 = 1.2589in place of the intended default of0.1. Only the default *tuning ranges* fromget_default_tune_params()are written on the scaledialsstores, and those are left alone, so results obtained without atune_paramsargument are unaffected. Regression tests are intests/testthat/test-tuning-grid-scale.R.
Show all changes in 0.7.10Hide the rest of the changes in 0.7.10
- **Default
loss_reductionranges no longer search implausible values.** The default range was written asc(0, 5)and read as log10, so the searched values were 1, 316 and 100000 rather than the documented [0, 5]. The corresponding entry in the wider registry,c(0, 10), searched up to 1e10. Both are now written on the scale the registry uses and search the documented range. Selection is by cross-validation, so a degenerate grid point was never chosen; the effect was a wasted search budget and a default grid that never explored small values. - **Candidate values are no longer dropped when they outnumber
grid_levels.**dials::grid_regular()takes at mostlevelsvalues per parameter, andgrid_levelsdefaults to three, so a user supplying four candidate values silently lost one. Levels are now raised per parameter to the number of values requested, leaving parameters at their defaults untouched so the grid does not grow except where asked. - **Default tuning ranges are reachable for survival tasks.**
get_default_tune_params()dropped the outcome columns withsym(), which accepts a single symbol, so a survival label naming two or three columns raised "Can't convert a character vector to a symbol" and no default range was ever returned for any survival family. The columns are now dropped by name. Note thatpenalized_coxhas no default range and fits at a fixed penalty of 0.01 with a pure lasso mixture; its penalty is not selected from the data, and a user wanting selection should supply candidates throughtune_paramsor callglmnet::cv.glmnet()directly. - **Probability metrics no longer return
NAinside a metric set.**yardstick::metric_set()calls its members withna_rmandcase_weights, and passesevent_levelto metrics that accept it. The wrappers forroc_auc,logloss,brier_scoreandecedid not declare those as formals, so they arrived in..., where the wrappers read...as a selection of probability columns. Each metric then returnedNArather than erroring. All four failed together while the six class metrics, whose wrappers already declared these arguments, were unaffected. The four wrappers now declarena_rm,event_levelandcase_weights, and anevent_levelsupplied by the caller takes precedence over the one captured when the metric was configured. Results obtained without tuning are unchanged, including every figure in the article's case studies, because the affected call path is the onetune_grid()uses. - **Nested cross-validation no longer degrades silently when tuning is requested.** Because
roc_aucis the default selection metric and probability metrics were empty under tuning (above),select_best()could not choose a configuration. The unfinalized workflow was then fitted anyway, sotune()placeholders reached the engine and the failure surfaced as a message about a hyperparameter, for example "Invalid value for num.trees". Every outer fold failed, the run fell back to ordinary resampling, and the returned object recordedselection$sourceas"none"with no indication that no nested estimate had been produced. Nested cross-validation consequently worked only when it was not tuning, which is the one case in which it has no purpose. Three changes fix this. Probability metrics compute (above); an outer split whose inner tuning selects nothing is now skipped with a warning naming the selection metric, rather than fitting a workflow that still containstune()markers; and configuration selection now identifies the configuration by.configand recovers its hyperparameter values from rows that carry them, sincetuneattaches those columns only to the class-metric rows. - **The nested selection rule is documented and reported.** The configuration carried into the returned model is the one with the best mean inner-fold score for the primary metric, pooled across all outer folds, with ties broken by smaller standard error and then a deterministic ordering. It is neither the modal winner across outer folds nor the configuration attached to the best outer score.
summary()now prints, alongside a nested estimate, that the figure describes the tuning and fitting procedure rather than the returned model. - **A numeric stratification column no longer discards the entire sample.** The
stratified_coxpath read the stratifier's *values* from the preprocessed frame but its *levels* from the training frame. Any recipe step that rewrites the column's values,step_normalize()being the ordinary case, therefore produced a factor whose levels matched nothing, every row becameNA, andcoxph()stopped with "No (non-missing) observations". A factor stratifier is untouched by normalisation and so was unaffected, which is why the defect survived hand testing. The stratifier is now read from the training frame, since a grouping label is not a modelled covariate and must not be transformed. If preprocessing changes the number of training rows, the alignment is now refused with an error rather than attempted. - **
royston_parmarno longer fails whenrstpm2is not attached.**rstpm2::stpm2()resolves an internal helper from the search path, so calling it through the namespace failed with an object-not-found error whenever the user had not attached the package. The fit now runs insidewithr::with_package("rstpm2", ...), which attaches the package for the duration of the call and restores the search path afterwards. The attach is a no-op when the package is already attached. This is a side effect on global state, and it is documented as such; the fit is native and unresampled, so it does not execute in a parallel worker. - **A survival registry test suite.**
tests/testthat/test-survival-registry.Rasserts that every family inavailableMethods("survival")fits on a canonical right-censored frame under a holdout, and that every combination the documentation records as unavailable raises an error rather than degrading silently. All four defects above were found by hand and would all have been caught by these tests. - **XGBoost AFT interval bounds are now supplied on the original time scale (breaking for results, not for the API).** The native
xgboost_aftpath passedlog(time)aslabel_lower_boundandlabel_upper_bound. XGBoost's AFT objective modelslog(Y) = T(x) + sigma * Zand applies the logarithm to those bounds internally, so the previous code applied the transformation twice and optimised the wrong likelihood. The effect was silent on data whose survival times all exceed one, where it degraded the fit, and fatal below one, where the doubly-logged bound is undefined and every risk score came back non-finite. On a simulated dataset with 351 of 400 times below one, the previous code produced 0 of 400 usable risk scores; it now produces 400 of 400 with a concordance of 0.92. Concordance on thelung,rotterdamandflchainbenchmarks changes by roughly 0.03, 0.03 and 0.10 respectively. Any previously reportedxgboost_aftresult should be regenerated. Regression tests pinning the bounds against a direct xgboost reference are added intests/testthat/test-xgboost-aft-bounds.R. - **The redundant
seedparameter is no longer passed toxgb.train().** The R xgboost package ignores aseedentry inparamsand warns about it on every fit, so survival runs emitted a spurious warning. Seeding is and was handled byfastml_with_seed(), which callsset.seed()around the fit, so predictions are unchanged. - **
base_scoreis now anchored to the observed time scale for XGBoost AFT.** XGBoost initialises the AFT margin atlog(base_score), whose default of0.5sits far below survival times measured in days or months. With the logistic AFT loss the objective saturates at that distance, the gradients underflow, no tree splits, and every prediction is identical. The previouslog(time)bounds masked this by placing the labels near the default starting point.base_scoreis now set to the median observed time, which starts the optimiser on the scale of the data. Without it, concordance onlungandrotterdamcollapses to exactly 0.5. - **Grouped and time-ordered holdout constraints are now enforced together.** Supplying both
block_colandgroup_colspreviously warned that grouping would be ignored and cut the holdout purely by time, so a group could contribute rows to both the training and the test set — the exact contaminationgroup_colsis meant to prevent. The holdout is now cut at the admissible point nearest the requestedtest_size, where a point is admissible only if no group spans it, so every training row precedes every test row *and* no group appears on both sides. Because such a point need not fall at the requested proportion, the realized test fraction is reported through a warning when it differs by more than five percentage points. When groups are interleaved in time no admissible point exists, andfastml()now stops with an explanatory error instead of silently dropping a guarantee. Fold construction underblocked_cvand `
The NEWS section is longer than shown here.
From NEWS.md in the release
Install
Health
- OK13 OK2026-08-05
- NOTE12 OK, 1 NOTE2026-08-01
- OK13 OK2026-06-09
- ERROR12 OK, 1 ERROR2026-06-08
- OK13 OK2026-05-02
Show 2 earlier changesHide 2 earlier changes
- ERROR11 OK, 1 ERROR2026-04-25
- OK14 OK2026-03-10
Documentation
- Help pages
- 74 (26 internal)
- Examples that run
- 18%2 run in a plain check · 3 only when donttest examples are enabled (R CMD check --as-cran) · 6 never run
- Exported functions with examples
- 20%
- Documented parameters
- 98%
- Return-value docs
- 93%
- Help pages citing references
- 0%
Downloads
Repository
- PRs merged (all time)
- 98%
- Median days to close, 50 most recently updated closed issues
- 7
- Median days to close, 50 most recently updated closed PRs
- 0
Stars over time
Forks over time
Releases over time
Issues over time
PRs over time
Commits over time
9 development-tooling and community-health practices detected across 7 families in the upstream repository
Checks run against github.com/selcukorkmaz/fastml on 2026-10-11.
The R CMD check workflow runs on Linux, macOS and Windows, and on R-devel
Show all practicesHide 3 practices
This repository shows signs of AI coding tools.
- Codexfirst seen 2025-06-05Pull request it wrote: opened by a project member
- Claude Codefirst seen 2026-01-19Seen in the repository: .claudeNamed in .Rbuildignore: .claudeNamed in .gitignore: .claudeCommits: at least 3 crediting it in the message
- Claude Opus 53 commits · Sep 2026
Dependencies
Installing fastml adds 106 packages (50 with compiled code, 77 MB of source tarballs). Heaviest: stringi 11 MB, ggplot2 6.0 MB, data.table 5.7 MB.
Code & Tests
People & History
Pick two releases to compare their code metrics.
- 0.7.10Latest
- 0.7.92026-08-19 · diff ↗
- RR 4.6.0 released · 2026-04-24
- 0.7.8
- 0.7.72026-01-27 · diff ↗ ·
added 16 exports
compare_defaults, estimate_tuning_time, explain_stability, get_default_differences, get_default_params_with_warnings, get_expanded_tune_params, get_parsnip_default_engine, get_parsnip_default_params, get_tuning_complexity, get_tuning_params_for_complexity, print_default_differences, print_tuning_presets, recommend_tuning_config, reset_default_warnings, validate_defaults_registry, warn_default_override - 0.7.62026-01-10 · diff ↗
- 0.7.52025-12-22 · diff ↗ ·
removed 1 export
evaluate_modelsadded 2 exports
fastml_compute_holdout_results, predict_model.model_fit - 0.7.02025-10-29 · diff ↗ ·
added 9 exports
counterfactual_explain, explain_ale, explain_dalex, explain_lime, interaction_strength, plot_ice, predict_risk, predict_survival, surrogate_tree - 0.6.22025-07-24 · diff ↗
- 0.6.12025-06-11 · diff ↗
- 0.6.0
- RR 4.5.0 released · 2025-04-11
- 0.5.02025-03-07 · diff ↗ ·
added 11 exports
availableMethods, flatten_and_rename_models, get_best_model_idx, get_best_model_names, get_best_workflows, get_default_engine, get_default_params, get_default_tune_params, get_engine_names, get_model_engine_names, process_model - 0.4.0
- 0.3.0
- 0.2.0
Show 2 earlier eventsHide 2 earlier events
- 0.1.02024-11-12
- RR 4.4.0 released · 2024-04-24
Package metadata
- First published
- 2024-11-12
- Total releases
- 15 / 2 yrs
- License
- MIT + file LICENSE OSI · relicensed from GPL (>= 2) in 0.6.0
- Minimum R
- ≥ 4.1.0
- Download size
- 387 KB
Cite
Cite this package
Run in R for the authors' preferred citation:
citation("fastml")This package has no citation file, so citation() returns this entry built from DESCRIPTION.
Cite the R Observatory
For a number measured here: a download total, a coverage figure, an archival date.
From data release v2026-10-17, which the citation names so these numbers can be found later. More on citing and the projects behind them.