Streaming Breusch\–Pagan test for large datasets¶

Source: R/performance.R · Help: ?performBPTestStreaming

Computes the classical Breusch\–Pagan statistic using chunked cross-products so that large datasets can be evaluated without materialising the full auxiliary regression in memory. The streamed result is algebraically identical to performBPTest.

Usage¶

performBPTestStreaming(model, data = NULL, chunk_size = 10000, progress = interactive())

Arguments¶

model: A fitted stats::lm object describing the mean structure whose residual variance is to be assessed.
data: A base::data.frame containing the variables referenced in model. The data must include all observations used to fit the model. When NULL, the default, the model frame of model is used. It holds the terms of the model rather than its variables (log(x) rather than x), so a model that transforms a variable needs data, as does a model with an offset() term, and so may a fit whose weights or offset argument is a column of the data, which the model frame holds as (weights) or (offset).
chunk_size: Positive integer giving the number of observations processed per streaming chunk. Smaller values reduce peak memory usage at the expense of additional iteration overhead.
progress: Logical flag controlling whether a textual progress bar is displayed while chunks are processed. Defaults to interactive().

Value¶

A htest object mirroring performBPTest and reporting the chi-squared statistic, degrees of freedom, p-value, and metadata describing the chunked computation.

Details¶

Each chunk contributes to the cross-product matrices \(X'X\) and \(X'y\) for the auxiliary regression of squared residuals on the original regressors. The chunks are aggregated to recover the exact Breusch\–Pagan statistic while keeping memory usage bounded. When Matrix is installed, sparse cross-products are used automatically for large chunks.

Examples¶

data(mtcars)
mod <- lm(mpg ~ wt + qsec, data = mtcars)
performBPTestStreaming(mod, mtcars, chunk_size = 16, progress = FALSE)