Streaming Koenker studentized Breusch-Pagan test¶

Source: R/performance.R · Help: ?performKoenkerTestStreaming

Evaluates Koenker's studentized variant of the Breusch-Pagan test (the \(n R^2\) statistic from regressing the squared residuals on the regressors) chunk by chunk so that large datasets can be processed without allocating the full auxiliary regression matrix.

Usage¶

performKoenkerTestStreaming(
  model,
  data = NULL,
  chunk_size = 10000,
  progress = interactive()
)

Arguments¶

model: A fitted stats::lm object supplying residuals and fitted values for the diagnostic.
data: The data frame model was fitted to, or NULL, the default, read as performWhiteTestStreaming() reads it: the response, design, weights and offset come from the fit itself, so a model that transforms its variables, has an offset() term, or takes its weights or offset from a column of the data needs no data (since 0.16.0), and data that are given are matched to the observations of the fit.
chunk_size: Positive integer giving the number of observations processed per streaming chunk.
progress: Logical flag indicating whether a textual progress bar should be shown while processing the chunks. Defaults to interactive().

Value¶

A htest object equivalent to performKoenkerTest with metadata recording the streaming configuration.

Details¶

Each chunk of the regressors, centred at their means, is reduced by a QR decomposition stacked on the factor of the chunks before it, as in performWhiteTestStreaming(), since 0.17.0. Since 0.18.0 the factor leaves out what is left of a regressor that is a combination of those before it in the rows so far, rounding error, as its details say. Up to 0.16.0 the cross-products of the regressors were accumulated: those of a regressor 1,000 standard deviations from zero looked collinear with the intercept, and the test stopped ("Koenker auxiliary regression became rank deficient"); and squared residuals of about \(10^{-10}\) were taken for constant ("requires variation in squared residuals"). It gives the value of performKoenkerTest() on both. Sparse cross-products from Matrix are no longer used.

The degrees of freedom are the rank of the auxiliary regression less its intercept, as in performKoenkerTest() (see its Degrees of freedom). Since 0.18.0 the test gives them, and the value of performKoenkerTest(), on a model without an intercept whose columns span the constant, such as the dummies of every level of a factor in lm(y ~ 0 + group + x1): 10.52 on 4 degrees of freedom, where it stopped ("Koenker auxiliary regression became rank deficient").

Since 0.18.0 the test refuses an auxiliary regression that fits the squared residuals exactly, as on a paired design, by the rule and with the message and class of performKoenkerTest() (see its Exact fit), where it gave 60 on 30 degrees of freedom on 30 subjects of two observations. It refuses it when the design forces it, decided from the squared pseudo-residuals it carries in the same pass, as performKoenkerTest() decides it; a fit the data make exact gives a value.

Since 0.16.0 the chunks are rows of the design of the fit, as in performWhiteTestStreaming(), whose details say what changed.

Supported models¶

It accepts an lm and a glm with the Gaussian family and the identity link, unweighted or with positive weights, a fitWLS() fit and, since 0.16.0, a rank-deficient fit, which it tests on the estimable part of its design: it gives the value of the same model fitted without the aliased columns. It refuses a glm of another family or link, a survey::svyglm() fit, a fit with a zero weight, a coefficient that is not finite for another reason, data that lack rows or variables of the fit, and, since 0.16.0, a multivariate lm, a fit changed after it was fitted and a fit with model = FALSE whose data are gone, unless data gives its response and fitted values. Which model each test accepts, and why, is set out in test_results, under Supported models.

Examples¶

data(mtcars)
mod <- lm(mpg ~ wt + qsec, data = mtcars)
performKoenkerTestStreaming(mod, mtcars, chunk_size = 16, progress = FALSE)