onls

MAJOR UPDATE: one-pass (ODRPACK-style) algorithm and multivariateness:

• replaced the nested two-step algorithm (inner foot-point optimization for every observation, outer optimization of the parameters, using L-BFGS-B for foot points when p > 1) by a one-pass algorithm in the manner of ODRPACK: the parameters and all foot-point corrections delta_i are estimated simultaneously by a single Levenberg-Marquardt run (minpack.lm::nls.lm) on the joint problem, minimizing the same objective as ODRPACK, sum[ Qyy_i * (y_i - f(x_i + delta_i, theta))^2 + delta_i' * Qx_i * delta_i ], with Qyy and Qx corresponding to ODRPACK's WE and WD
• the same code path now handles any number of predictors (single predictor, multivariate, correlated predictor errors); the foot points are no longer found by a separate inner optimizer, and orthogonality of the foot points is now the stationarity condition of the joint problem
• the Jacobian of the joint problem is now supplied to the solver exactly, using symbolic derivatives of the model formula (deriv()) and falling back to central finite differences if the formula cannot be differentiated (e.g. calls to user-defined functions). MINPACK's internal forward-difference Jacobian is unsuitable here, because its step is proportional to the size of each unknown and vanishes for small foot-point corrections, which stalled foot points at their observations (non-orthogonal points, e.g. for the last point of the odrpack guide example 2.C.i)
• control$outer_max is now the total budget of Levenberg-Marquardt iterations (default 5000). As minpack.lm silently caps a single nls.lm call at 1024 iterations, the solver is restarted from its last iterate until it converges, a restart brings no further improvement, or the budget is used up
• new control elements ftol and ptol (convergence tolerances, default 1e-10 instead of sqrt(.Machine$double.eps), which left points with very small residuals slightly non-orthogonal), diag (optional parameter scaling, MINPACK's adaptive scaling is now the default) and lower/upper as alternatives to the arguments of the same name
• extend and window now default to NULL, i.e. foot points are unbounded as in ODRPACK. They remain available as optional foot-point bounds for single-predictor models (window only together with extend)
• the covariance matrix of the parameters is now the ODRPACK (Gauss-Newton) covariance, i.e. the parameter block of the inverse of J'J of the joint problem, evaluated with effective-variance weights w_i = 1/(1/Qyy_i + g_i' * Qx_i^-1 * g_i) with g_i = df/dx at the foot point. This is valid for any number of predictors (the previous multivariate version ignored the predictor error in the standard errors)
• the NLS warm start now substitutes fixed parameters into the model formula instead of fitting them freely and resetting them afterwards, and no longer fails silently when weights are supplied (the weights were not found by nlsLM, so the raw starting values were used instead)
• onls() now warns if some free parameters have no measurable influence on the fitted model (e.g. a sigmoid that has collapsed to a constant because its exponent underflows), a degenerate stationary point at which the solver may report convergence
• the model formula is now evaluated in its own environment, so that user-defined functions and constants used in the model are found; the model is evaluated once at the starting values and errors are reported instead of being swallowed
• extended the core algorithm from single-predictor to n-dimensional (multivariate) models