STATassist

draw_dim_reduction_plot() is what that shared axis was for: it paints the

labels one of them found onto the coordinates the other placed, beside the

grouping that was known before either ran.

And the pair of features, which nothing had asked about. Every function so far

reduced one feature at a time or every feature at once; the question of how two

of them move together had no entry point, and a correlation matrix computed by

hand had no p-value beside it and no way of being drawn.

And the summary that had a table but no picture of its own.

summarize_descriptive_stats() has reduced every feature and group to a row of

locations and spreads since the first release, and seeing one of those columns

across a set of groups meant reshaping the table and calling barplot() by hand,

with the interval either side of each mean worked out again on the way.

New features

• New summarize_association_stats() reduces every pair of features to the association between them. Pearson, Spearman and Kendall come back side by side on the same pairs, the way a comparison reports a parametric, a rank-based and a robust test side by side, so that a linear coefficient and a monotonic one disagreeing is a fact about the data rather than about which call was made. Each method gets a slot of four features-by-features matrices: corr from stats::cor(), pvalue from stats::cor.test(), adj_pvalue adjusted across the pairs, and n, the observations the pair shared. Beside them design records the call. The adjustment runs over the pairs that produced a p-value rather than over every cell of the triangle. A pair stats::cor.test() refused, having fewer than three shared observations or no variance on one side, is not a test that was performed, and counting it would shrink the other pairs for a comparison that never happened. Such a pair comes back NA rather than aborting the screen. The diagonal is a property of the matrix and not an estimate: corr is 1 there and both p-values are NA. It is not an sa_comparison. This is a screen, the pairwise counterpart of summarize_descriptive_stats(), and it returns a plain list for the same reason that one returns a plain data.frame.
• New draw_corrplot() draws that matrix, and adds three decisions to draw_heatmap() rather than a second drawing engine. Nothing is standardised, a coefficient already being on a common scale. The colours are fixed at -1 to 1, so the same colour means the same strength from one plot to the next. And the two axes hold the same features, so they are clustered once and permuted together; a symmetric matrix clustered twice can come back with its rows and columns in different orders, and the diagonal then wanders off the diagonal. Given the p-values, the pairs that did not clear sig_level are drawn as blank cells. The blanking happens after the clustering: a cell removed first would change the tree, and the picture would no longer be the matrix the reader is being shown. The diagonal is never blanked, a feature not being tested against itself, which is why the diagonal carries no p-value in the first place. The distance is 1 - cor(), the same one cluster_hclust() and draw_heatmap(dist_method = "correlation") mean, so a corrplot and a heatmap of the same features group them the same way.
• New draw_grouped_barplot() draws a column of summarize_descriptive_stats() as clusters of bars, one cluster per feature and one bar per group level. It is the summary counterpart of draw_grouped_boxplot(): a box shows the distribution a group's observations have, a bar shows one number standing for them, which is what a figure wants when the point is about a location rather than a spread. The heights are that function's own column rather than a second pass over the data, so a bar and a row of the table are the same number. errorbar is read under mainbar rather than independently of it, because only two of the summary columns are locations an interval either side says anything about. A mean takes "se", "sd" and Student's "ci" at conf_level. A median takes "ci" alone, the notch interval median +/- 1.58 * IQR / sqrt(n), which is the interval draw_grouped_boxplot() already returns as median_confidence_stats, so a bar and the notch of the box beside it are the same width on the same data. Every other height — a count, a spread, a shape — has no second quantity to be uncertain by and takes "none". Asking for one of the others is an error rather than a silently dropped interval, the alternative being a figure that answers a question other than the one it was asked. A bar is read against the zero it stands on, so a derived ylim always holds zero and puts its headroom on the side the bars run to; heights that go both ways, as "skewness" does, get the baseline drawn as well. A height the summary could not compute leaves its bar blank rather than shifting the ones beside it, which is what makes a group too small for a shape estimate visible instead of absent.
• New evaluate_regression_models() scores one or more fitted regressions on the same held-out rows. $metrics reports cor, r_squared, rmse, mae, bias and the calibration line, and $comparisons reports each model less the baseline. cor and r_squared are both there because they answer different questions: r_squared is 1 - SSE/SST on the scored rows, so it is the variance the predictions actually removed and can be negative, while cor^2 is what that would be after rescaling by a line fitted to those same rows. The gap between them is what calib_slope and calib_intercept report. There is no p-value beside the differences. A difference of held-out errors has no null this package is in a position to state, and the two numbers it is made of are in $metrics.
• New evaluate_classification_models() does the same for a two-class outcome. $metrics reports auc with a DeLong interval, brier, and accuracy, sensitivity and specificity at a stated threshold; $curves holds the ROC operating points as a rectangular table rather than a curve object. $comparisons asks three questions of each model against the baseline rather than picking one. delta_auc asks whether the ranking improved and is tested by DeLong's paired test. idi asks how much further apart the two classes' predicted probabilities moved, which an AUC is blind to, since reordering nothing leaves it untouched. nri asks how often a probability moved the right way, counting direction only, so a model that helps many rows slightly and hurts a few badly scores well here and badly on the IDI. The class order is the fits'. outcome_lv and control_label are checked against the order the models were fitted with rather than used to change it, since a fitted classification predicts the probability of its own second level and cannot be re-pointed after the fact.
• Both return the new sa_performance contract, a plain list of analysis, models, design, parameters, predictions, metrics, comparisons, curves and metadata. No engine object is stored: the calibration line is two numbers rather than the lm that produced them, so the whole object writes out as JSON. Every model is scored on the rows **all** of them could predict. predict.sa_model() answers NA for a row incomplete across that model's predictors, so models fitted on different predictor sets otherwise come back with different rows filled in, and DeLong's test, the IDI and the NRI are paired statistics with no meaning across different rows. One message reports how many rows went and why.
• New draw_prediction_plot() and draw_roc_curve(), with plot.sa_performance() dispatching to whichever the result calls for. The scatter draws the identity line and the calibration line from the two numbers $metrics holds rather than fitting again, so the picture and the table cannot drift apart. Its type is "panel" past one model, because two clouds of points on shared axes make a third that belongs to neither; type = "overlay" with points = FALSE compares the calibration lines alone.
• The AUC, ROC, DeLong, IDI, NRI and Brier kernels are written out in the package rather than taken from pROC. pROC covers the first three and none of the last three, and DeLong's test has no counterpart in scikit-learn or scipy for the Python port to call, so depending on it would buy the easy half and leave the hard half to be written twice against two sets of defaults. pROC is in Suggests as the oracle the tests check these against. Runtime dependencies are unchanged.
• New cluster_hclust(), cluster_kmeans(), cluster_dbscan() and cluster_snn(), the second half of the unsupervised family. The reductions ask where each point sits; these ask which points belong together. There are four because they disagree about what a cluster is, and the disagreement is the information: the first two are told how many groups to find and will always find that many, while the other two are told how dense a group has to be and derive the count from that, so they can return two clusters, or nine, or none, and they can refuse to place a point. cluster_hclust() returns the tree as well as the cut, so cutree($fit, k = 5) asks for another cut without rebuilding the distance matrix. It is also the only one of the four with a choice of distance, because it is the only one where the choice is real: k-means minimises squared Euclidean distance by construction and the two density methods measure a radius in it. cluster_kmeans() runs nstart = 25 rather than stats::kmeans()'s own 1, which makes the answer a search rather than a draw; the override is reported in engine. seed makes a run repeatable and restores the random stream afterwards.
• All four return the new sa_cluster contract, a plain list of analysis, points, design, parameters, assignments, clusters, engine, fit and metadata. The row axis is points, the same one sa_reduction uses, and the four read their input through the same helper the three reductions do — so the rows are the same rows and an assignment can be painted straight onto a set of scores. cluster_scale sits where embedding_scale does and chooses the same way between clustering the samples and clustering the features. Noise is cluster 0, which is dbscan's own convention. clusters never has a row for it, so nrow(clusters) is the number of groups found and design$n_noise is what did not join one; a partitioning method reports n_noise = 0 and both shapes read down the same columns. Every point being noise is a possible answer rather than an error, and it says so. assignments$silhouette is the one number that compares across all four, since coordinates from two methods share no scale but a ratio of distances does. It is Rousseeuw's definition written out in the package rather than taken from cluster: a singleton scores 0, noise scores NA and takes no part in anyone else's arithmetic, and a single cluster scores NA throughout.
• cluster_dbscan() derives both of its arguments when they are not given, and says what it derived. min_pts follows the textbook d + 1, floored at 4 and capped at half the points, since this package's usual input is wider than it is tall and a threshold above half can only ever return one cluster. eps is a radius in the units of the data and so can have no constant default; it is taken as the 95th percentile of the k-distance curve, which reads as "assume about one point in twenty is noise". parameters$eps_source records whether the value was supplied or derived. cluster_snn() is shared nearest neighbour clustering: each point keeps its k nearest neighbours and two points are linked when their neighbour lists overlap. Its eps counts shared neighbours where cluster_dbscan()'s is a distance. Both engines call the argument eps, so renaming one here would put this documentation at odds with theirs; the two say so instead.
• dbscan is a new dependency, the engine behind those two. draw_heatmap() now measures its dendrogram distances through the same function cluster_hclust() does, so the tree it draws and the tree that gets cut are one tree.
• New draw_dim_reduction_plot(), which plot() on an sa_reduction now calls. Two coordinates of a reduction as a scatter, with a clustering of the same points taking the colours and a grouping that was known already taking the shapes. Two channels rather than one, because the question a clustering usually raises is whether it recovered a grouping nobody told it about, and one channel would answer that by hiding it: either the clustering or the grouping would have to be left out, or the two crossed into a single set of labels whose count is the product of theirs. On two channels it is read directly — one colour per shape is a clustering that found the groups, and a shape split across colours is a group the data does not see as one thing. Give one and it takes the colours on its own; give neither and the points are drawn in the foreground colour. The two objects have to be about the same points, which is checked rather than assumed: both contracts carry points and both read their input through the same function, so a mismatch means two different frames were reduced and clustered and is refused instead of lined up by position. Points that joined no cluster are grey rather than given a colour beside the others, since a point left out is the absence of a cluster and not one more of them, and a PCA's axes carry the share of the variance they explain while an embedding's do not. group_lv here orders the levels and selects no rows, which is the opposite of what it does in draw_grouped_boxplot(). Dropping a row at this point would erase a point the reduction had already placed, so a level group uses and group_lv leaves out is an error.
• draw_dim_reduction_plot() gains cluster_lv, one label per cluster for the legend. NULL keeps #1, #2, and so on; noise is still listed as noise (n) when present.
• draw_dim_reduction_plot()'s col now names group levels when only group is given, and its new pch argument names group shapes or, with no group, the points themselves. Defaults are unchanged: shapes from the built-in sequence and 16 on a plain scatter.
• Clustering on a reduction's $scores now reads sample names from its points column when the table has no row names, so a clustering of PC1 and PC2 lines up with the reduction it is drawn on.
• make_block_cor() gains against, which splits a block in two: the indices in features and the indices in against each correlate at cor among themselves and at -cor with the other side. A block that shares one value has a floor on how negative that value can be — -1/(k - 1) for k predictors, so three of them could not disagree past -0.5 and four past -0.333 — which left a strong negative correlation unavailable at the sizes worth simulating. A split block is o