Releases¶
The changes in each version, from the package's NEWS.md.
0.17.0¶
0.17.0 completes the model context, in the last four of its eleven steps
(The model context, under 0.16.0), and fixes the failures of the additional
check of 0.16.0 on CRAN with Intel MKL as BLAS. The resampling tests,
bootstrapTestStatistic() and the robust tests are built on the context, and
each replicate is refitted by least squares on the stored design, where the
White bootstrap and bootstrapTestStatistic() refitted the formula on the
rows of data; a Gaussian glm is bootstrapped as the lm it is (step 8).
A fit with na.action = na.exclude, which the tests refused, is read as the
same fit with na.omit (step 9). A grouped run of a fitWLS() fit is
refused, the runners take the data a tidymodels workflow was fitted to, the
recommendations and generics::augment() match the rows of data to the
observations of the fit as the tests do, and the preparation of data the
tests used before the context is gone (step 10). A MASS::rlm() fit is
refused; a penalised mgcv::gam() fit is tested on the basis of its smooths
by the tests that do not need least squares and refused by those that do, and
a gam that is the least-squares fit of its design is taken by every test
(step 11). Beside the steps, White's streaming test drops the collinear terms
of its auxiliary regression, and the three streaming tests no longer take the
terms of a regressor located far from zero for collinear ones and compare the
variation of the squared residuals with their scale (N3, The streaming
tests). The MKL check failed 179 expectations that compared two computations
of the same numbers bit for bit: on CRAN most such comparisons are now made to
a tolerance of 1e-10, a few to 1e-8 or on another scale, and six blocks whose
outcome turns on the last bits run off CRAN only; off CRAN they are still made
bit for bit. A CI workflow runs R CMD check --as-cran as the
additional checks of CRAN do, under MKL, the other BLAS libraries they use and
R without long doubles, and under a BLAS that moves its results by up to one
unit in the last place, on every pull request that changes the package
(Packaging). No export, argument or default changes; hardhat joins the
suggested packages.
The changes in bold change what calls return. The streaming tests: White's
drops the collinear terms of its auxiliary regression and gives the value of
performWhiteTest(), where it stopped on any model with a factor; the three
no longer take the terms of a regressor located far from zero for collinear
ones, compare the variation of the squared residuals with their scale, so
that residuals of about 1e-5 give a value, and refuse squared residuals
that agree to about eight digits, at every scale and in chunks of any size;
and White's, in more than one chunk, gives the value of performWhiteTest()
on a regressor named like a term it generates. Step 8: the White bootstrap
computes its observed statistic and its replicates on the stored design;
bootstrapTestStatistic() refits each replicate on it, no longer refuses a
fit whose model frame cannot be rebuilt from data, and bootstraps a
Gaussian glm with the identity link; the resampling and robust tests take a
model without data, and the model frame as data; the White and wild
bootstraps and the robust tests refuse a fit changed after it was fitted, the
White bootstrap a fit with model = FALSE whose data are gone given a tibble
in another order, and a gam with smooth terms, and
bootstrapTestStatistic() data that lack a variable of the formula; every
test built on the context refuses a fitWLS() fit of a gam with smooth
terms; and a multivariate lm, a fit with model = FALSE whose data are gone
and not given, and, in bootstrapTestStatistic(), a Gaussian glm with a
zero weight or with data the tests refuse are refused with a message that says
why. Step 9: the tests and the runners take a fit with na.exclude, and give
the value of the same fit with na.omit; the panel tests and the scatter
diagnostic read it at the rows it kept; fitWLS() pads its estimated
variances; and a gam with
na.exclude is read through its design with na.omit, and the functions
outside the tests that read such a fit, the influence and VIF diagnostics, the
recommendations, autoCompareRemediations(), analyzeMLResiduals(), the
plots and augment(), read it without the padding. Step 10: a grouped run of
a fitWLS() fit is refused; the runners take the data a workflow or a parsnip
fit of fit_xy() was fitted to, a transformed outcome, a lazy dtplyr table
and the data of a recipe step that dropped rows at the fit among them, with
their case weights compared, and refuse the data of a fit_xy() fit with case
weights; a refusal names a recipe step skipped on new data that changed the
data; their results on a workflow or a parsnip fit carry
attr(, "tidymodels"), a recipe step with the environment it was written in;
recommendRemediationStrategies(), generateHeteroRecommendations() and
augment() match the rows of data as the tests do, and refuse what the
tests refuse, data that lack the variables of the model and a model the tests
refuse among them; augment() binds the rows as broom does, binds the first
copy of data given twice, and refuses the data of a preprocessing that
reorders the rows or makes a row from other rows, values that were right
among them; and the report profiles data as given. Step 11: a MASS::rlm()
fit is refused by every function built on the context, in a grouped run and
by autoCompareRemediations() too, and the call of a fitRobust() result
names what was given; a penalised gam is refused by the tests that need
least squares and by fitWLS(), and the streaming tests read the basis of its
smooths; a gam that is the least-squares fit of its design is taken by every
test, and one without a penalised term on an ill-conditioned design, where
mgcv's fit is not that fit, is refused by the tests that need least squares
and by fitWLS(); and a refusal of the model context in a group stops a
grouped run. The clean-up of step 10, in bold too, changes no value.
The streaming tests¶
White's streaming test drops the collinear terms of its auxiliary
regression, as performWhiteTest() does (N3 of the design of the model
context). performWhiteTestStreaming() stopped with "Auxiliary regression
became rank deficient" whenever terms of its auxiliary regression were
collinear: the square of a dummy, which is the dummy, so on any model with a
factor; the square of the first column of an orthogonal poly() basis; x
beside I(x^2). It now counts the terms by the rank performWhiteTest()
takes, that of the intercept and the auxiliary design of the centred
regressors at the tolerance of qr(), drops those that are a combination of
the intercept and the terms before them, and gives the degrees of freedom and
the value of performWhiteTest(). On lm(y ~ x1 + group) in the design of
contract_data() (tests/testthat/helper-test-recipes.R), with group a
factor of four levels:
| Call | 0.16.0 | Now |
|---|---|---|
performWhiteTestStreaming(m, d), in chunks of any size |
Error: "Auxiliary regression became rank deficient" | 17.83 on 8 (p = 0.0225), the value of performWhiteTest() |
runHeteroTests(m, d, chunk_threshold_mb = 0) |
White's test without cross products, 13.79 on 5 (p = 0.017), tagged as a fallback | White's streaming test, 17.83 on 8 |
The same holds on 30 subjects of two observations and one of three, where
performWhiteTest() gives 59.75 on 30, on a model of I(x > median(x)) or
cut(x, quantile(x)), on a factor with contr.sum, Helmert or polynomial
contrasts, without an intercept, with weights and in a Gaussian glm: in
chunks of 7, 40 and 10,000, with and without cross products, the degrees of
freedom are those of performWhiteTest() and the statistic agrees with it to
4e-14 relative. A fit whose auxiliary regression fits the squared residuals
exactly is refused with the message of performWhiteTest(), as since 0.16.0.
The three streaming tests no longer take terms of a regressor located far
from zero for collinear ones. They accumulated the cross-products of the
auxiliary design, decided their rank by qr() of them, which squares the
condition of the design, and solved the normal equations. White's streaming
test stopped on the squares of a regressor 100 standard deviations from
zero, and on the terms of lm(y ~ x1 * x2) with x1 between 0.5 and 2;
performBPTestStreaming() and performKoenkerTestStreaming() stopped
("auxiliary regression became rank deficient") on a regressor 1,000 standard
deviations from zero beside the intercept, such as a date written as
YYYYMMDD. Each chunk is now reduced by a QR decomposition stacked on the
triangular factor of the chunks before it, which holds the cross-products of
every row without forming them, of the auxiliary design of the regressors
centred at their means, as performWhiteTest() builds it. On lm(y ~ x1 +
x2) with x1 moved 1,000 standard deviations from zero, White's streaming
test gives 11.14 on 5, the Breusch-Pagan test 12.45 on 2 and Koenker's 9.04
on 2, the values of the exact tests and of the model at zero, where the three
stopped. From 100 to 10,000 standard deviations, on linear terms, factors,
weights and a Gaussian glm, the degrees of freedom are those of the exact
tests and the statistics agree with them to about 2e-10 relative, the
rounding being the exact tests', which fit the uncentred design; on a product
or a power of a variable located far from zero inside the model, which
performWhiteTest() itself computes less precisely, to 3e-9 at 1,000
standard deviations. Where they gave a value, the values change in their
last bits; where the normal equations lost more digits, as on an ns() term
(White's test, 5.7e-11 relative) or a regressor 1,000 standard deviations from
zero inside a factor model (Breusch-Pagan and Koenker, up to 5e-9), the new
value is that of the exact test, to 6e-14. The sparse cross-products of
Matrix are no longer used.
The streaming tests compare the variation of the squared residuals with
their scale. They refused squared residuals whose total sum of squares
about their mean was at most .Machine$double.eps, whatever their scale
("requires variation in squared residuals"), so residuals of about 1e-5:
with set.seed(11); n <- 120; d <- data.frame(x1 = runif(n, 0.5, 2), x2 =
rnorm(n)); d$y <- 1e-5 * rnorm(n, sd = d$x1); m <- lm(y ~ x1 + x2, data = d),
the three stopped, where performWhiteTest() gives 12.787 on 5 degrees of
freedom, performBPTest() 12.997 on 2 and performKoenkerTest() 10.289 on
2, and the same data times 1e5 streamed to 12.787. They now give those
values. The total is compared with the sum of squares of the squared
residuals: they are taken for constant when it is at most
.Machine$double.eps of it, when they agree to about eight digits, at any
scale.
The streaming tests refuse squared residuals that agree to about eight
digits, at every scale and in chunks of any size. The absolute threshold let
such residuals through at some scales, where the statistic was computed from
the rounding: residuals of +1 and -1 orthogonal to the regressors gave, in
chunks of 40 or more at a scale of 1, White's 0, Koenker's 40 (the number of
observations) and the Breusch-Pagan test 3.6e-15, and were refused at other
scales; and squared residuals that differ by a few parts in 1e9, such as those
of lm(y ~ x) with x <- rep(1:30, each = 4) and
y <- 2 + 0.5 * x + s * 1e4 * (1 + 3e-9 * v), s <- rep(c(1, -1, -1, 1), 30),
v <- rep(c(1, 0, 1, 0), 30) * x / 10, gave White's 60 on 2 and Koenker's 60
on 1, noise, where performWhiteTest() and performKoenkerTest() give
23.05. performWhiteTest() and performKoenkerTest() compute a statistic
from the rounding of the first case too (18.82 and 18.36).
White's streaming test, in more than one chunk, on a regressor named like a
term it generates, gives the value of performWhiteTest(). It matched the
columns of every chunk after the first to those of the first by name, and on
a regressor a beside a regressor a_sq the square of a, also named
a_sq, took the place of that of a_sq: 27.76 in chunks of 7 and 30.65 in
chunks of 40, where performWhiteTest() and the test in one chunk give
27.64. Every chunk has the columns of the design of the fit since 0.16.0, and
they are taken in their order.
No export or argument changes.
The model context, step 8: the resampling tests¶
The eighth of the eleven steps of the model context (The model context,
under 0.16.0; the design is on the site) builds the resampling tests on it:
performWhiteTestBootstrap(), performWildBootstrapTest(),
bootstrapTestStatistic() and its deprecated name
rbootstrap_test_statistic(), and the robust tests performWhiteTestRobust()
and performBPTestRobust(), with and without bootstrap = TRUE, and so the
registered "white_bootstrap" and "wild_bootstrap" and
runAdvancedDiagnostics(). They read the response, the design, the residuals,
the weights and the offset from the fit, match the rows of data to its
observations as every test built on the context does, and refit each
replicate by least squares on the stored design, where the White bootstrap
and bootstrapTestStatistic() refitted it from the formula on the rows of
data. The random numbers are drawn as before, in the same order, so a call
made after set.seed() returns what it returned wherever the stored design
is the design rebuilt from data, and leaves the generator where it left it.
No export or argument changes.
The examples below use the data of the example of
?performWhiteTestBootstrap, with a weight, an offset and a second regressor,
and draw 199 replicates after set.seed(1):
set.seed(42)
n <- 120
sim <- data.frame(x = runif(n, 1, 5), z = rnorm(n))
sim$y <- 1 + 2 * sim$x + rnorm(n, sd = 0.3 + 0.6 * sim$x)
sim$w <- 1 / sim$x
sim$o <- 0.5 * sim$z
The White bootstrap computes its observed statistic and its replicates on
the stored design. Both were computed on a design rebuilt from the rows of
data, through the formula and the call of the fit, and rwhite_test_rebuilt_design(),
the internal function that did it, is gone. Each replicate, the fitted values
plus the residuals drawn, is refitted by least squares on the design stored
with the fit, with its weights and its total offset, and its statistic is
White's on that design, the one performWhiteTest() reads. Where the design
rebuilt from data was the stored one, every value is identical() to
before. Where it was not:
| Call | 0.16.0 | Now |
|---|---|---|
performWhiteTestBootstrap(lm(y ~ x + z, data = sim, weights = w), B = 199), without data |
Error: "object 'w' not found" | 6.210 (p = 0.275), the value with data |
performWhiteTestBootstrap(lm(y ~ x + z + offset(o), data = sim), B = 199) |
Error: "The following variables are missing from input data: o" | 14.684 (p = 0.015), the value with data |
performWhiteTestBootstrap(lm(y ~ cut(x, quantile(x)) + z, data = sim), sim, B = 199) |
Error: "factor cut(x, quantile(x)) has new levels ..." | 12.206 (p = 0.135), the value of the factor computed beforehand |
I(x > median(x)) on a fit that dropped rows 3, 7 and 11 (z missing), given its data |
7.822 (p = 0.115), on the median of the rows the fit kept | 7.407 (p = 0.140), the value of the column computed beforehand on all the rows |
mgcv::gam(y ~ x + z, data = sim) given sim[sample(n), ] (set.seed(3)) |
14.684 (p = 0.020): the draws were added to the fitted values of other rows, the rows of data taken by position |
14.684 (p = 0.015), the value of the data in the order of the fit |
A fit to a subset given by index, which stopped with "White test could not
align the design matrix with model residuals", and the tibble of a fit that
dropped rows, which stopped with "Missing rows", give the values of the rows
of the fit. A model that transforms a variable, without data or given its
model frame, gives the value it gives with data.
bootstrapTestStatistic() refits each replicate on the stored design, and
no longer refuses a fit whose model frame cannot be rebuilt from data.
Each replicate was refitted from the formula on the rows of data, regenerated
or drawn, so a fit whose model frame could not be rebuilt from them so was
refused ("data does not reproduce the fit"): a term computed from the rows
it is given, cut(x, quantile(x)), or I(x > median(x)) on a fit that dropped
rows; and the model frame given as data for a model that transforms a
variable or has an offset() term. Each replicate is now an lm built from
the stored design and the model frame of the fit in the rows of the
replicate, which the statistic given as test_function reads as it reads a
fit. performWhiteTestRobust(bootstrap = TRUE) and
performBPTestRobust(bootstrap = TRUE), which call it, follow.
| Call | 0.16.0 | Now |
|---|---|---|
bootstrapTestStatistic(performKoenkerTest, lm(y ~ cut(x, quantile(x)) + z, data = sim), sim, B = 199) |
Error: "data does not reproduce the fit" |
11.080 (p = 0.020), the value of the factor computed beforehand |
The same with I(x > median(x)), rows 3, 7 and 11 dropped |
Error: "data does not reproduce the fit" |
7.346 (p = 0.035), the value of the column computed beforehand |
bootstrapTestStatistic(performKoenkerTest, m, model.frame(m), B = 199), m <- lm(y ~ x + z + offset(o), data = sim) |
Error: "data does not reproduce the fit" |
14.468 (p = 0.005), the value with sim |
performWhiteTestRobust(lm(y ~ cut(x, quantile(x)) + z, data = sim), sim, bootstrap = TRUE, B = 199) |
Error: "data does not reproduce the fit" |
bootstrap p = 0.135 |
Pairs resampling draws the rows of the stored design, so a term computed from
the rows it is given keeps the values of the fit in every replicate, where the
refit of the formula computed it again on the rows drawn: on
lm(y ~ splines::ns(x, 3) + z, data = sim) the percentile interval of the
replicates of Koenker's statistic was (10.34, 28.75) and is (10.40, 28.74);
on the test matrix each such replicate is that of the same fit with the
columns of the term computed beforehand. A
level of a factor that no row drawn holds is left out of the replicate, as
before, and so is a value of a character regressor; the contrasts of the fit,
given by contrasts = or as an attribute of the factor, are kept, and a
contrasts matrix with a row for each level gives way to the default contrasts
in a replicate that leaves its factor a level short, as in a refit of the
formula: with a factor of four levels, two of them on two rows each, every
replicate of each of these fits is that of before this step, identical(),
but for rounding on the factor coded by contrasts =, which was refitted
with the default contrasts. The statistic given as
test_function is given the rows drawn of data with the columns of the
model frame data does not hold under their own names, such as
I(x > median(x)) or (weights).
bootstrapTestStatistic() bootstraps a Gaussian glm with the identity
link (N8 of the design of the model context). It refused every glm, as its
refits by least squares would drop a family and a link; a Gaussian glm
with the identity link is the least-squares fit of lm(..., weights = w) with
its prior weights, and its replicates are those of that lm:
| Call | 0.16.0 | Now |
|---|---|---|
bootstrapTestStatistic(performKoenkerTest, glm(y ~ x + z, data = sim, weights = w), sim, B = 199) |
Error: "bootstrapTestStatistic() supports Gaussian lm models only" |
5.245 (p = 0.065), the value of lm(y ~ x + z, data = sim, weights = w) |
performWhiteTestRobust(<that glm>, sim, bootstrap = TRUE, B = 199) |
the same error | bootstrap p = 0.270, the value of the lm |
A glm of another family or link, a survey::svyglm() fit and an
mgcv::gam() fit are refused, as before, with a message that says why. So is
a Gaussian glm whose rank, decided at its tolerance of 1e-11, is not that of
lm() at 1e-7, as on a column within 1e-9 of a combination of the others:
the replicates are refitted at the tolerance of lm(), and such a fit is
refused as an lm fitted with a smaller tol is, "data does not reproduce
the fit".
The resampling tests and the robust tests take a model without data,
and the model frame as data. The wild bootstrap and the robust tests
prepared data, or the model frame, themselves, and a model that transforms
a variable or has an offset() term stopped without data: on
lm(log(y) ~ log(x) + z, data = sim), performWildBootstrapTest(m, B = 199)
and performWhiteTestRobust(m) stopped with "The following variables are
missing from input data: y, x", and give 2.003 (p = 0.375) and 3.469
(p = 0.628), the values with data. The complete rows of an na.omit fit
numbered again, and the tibble in fit order of a fit to named data, which they
refused ("Missing rows"), give the values of the rows of the fit. The wild
bootstrap refitted its replicates on the stored design already: every call
that gave a value gives it, identical().
New refusals. The White and wild bootstraps, and the robust tests,
refuse a fit whose coefficients, residuals or weights were changed after it
was fitted, as every test built on the context does: on lm(y ~ x + z, data =
sim) with 0.5 added to the coefficient of x, the White bootstrap returned
14.684 (p = 0.015) and the wild bootstrap 14.468 (p = 0.005), the values of the
fit before the edit. The White bootstrap refuses a fit with model = FALSE
whose data are gone, given a tibble in another order, on which it returned the
value of other rows. The White bootstrap refuses an mgcv::gam() fit with
smooth terms, as each replicate refits its design by least squares, which is
not the fit of a penalised smooth (decision 23 of the design); its refit of
the formula stopped on the smooth term ("could not find function \"s\""), and
runHeteroTests(tests = "white_bootstrap") put White's test under its name,
where it stops with the refusal. bootstrapTestStatistic() refuses data that
lack a variable of the formula, as the tests refuse them: with offset(o)
and o a vector of the workspace, a statistic of the caller's own that reads
nothing from data was bootstrapped on data in the order of the fit; give the
variable as a column of data, where it now follows the rows drawn, in any
order and under pairs resampling, which refused it ("does not come from
data").
A fitWLS() fit of an mgcv::gam() fit with smooth terms is refused.
fitWLS() refits such a fit as an lm whose terms do not rebuild its
design: they give a column for each variable of the formula, 3 for
y ~ s(x) + z, for the 11 coefficients of the basis of the smooth. The White
and wild bootstraps, which rebuilt a design from the terms, returned values
computed on the variables of the formula rather than on the fit, and the
tests built on the model context stopped with an internal error. Every test
built on the context now refuses it, as it refuses a fit whose design does
not reproduce its residuals (M8 of the design of the model context); a value
on the stored design is not defined, as the terms of the fit give no design
of its 11 coefficients. As refitting gives the same fit again, the message
points to testing the gam itself, or to giving fitWLS() a fit with a
parametric term such as splines::ns() in place of each smooth. With
g <- fitWLS(mgcv::gam(y ~ s(x) + z, data = sim), sim):
| Call | 0.16.0 | Now |
|---|---|---|
performWhiteTestBootstrap(g, sim, B = 199) |
3.279 (p = 0.715) | Error: "The fitted model does not reproduce its residuals from its design and coefficients: its terms rebuild a design of 3 columns for its 11 coefficients, ..." |
performWildBootstrapTest(g, sim, B = 199) |
2.152 (p = 0.380) | the same error |
performKoenkerTest(g, sim), performWhiteTest(g, sim), performWhiteTestRobust(g, sim, bootstrap = TRUE, B = 199) and bootstrapTestStatistic(performKoenkerTest, g, sim, B = 199) |
Error: "(subscript) logical subscript too long" | the same error |
Messages. A multivariate lm, given to any of them or to
bootstrapTestStatistic(): "needs a model with one response", where they
stopped with "'length = 4' in coercion to 'logical(1)'". A fit with
model = FALSE whose data are gone, without data: the message of the model
context, "cannot use the model frame of model", where they stopped with
"Expected a data.frame". bootstrapTestStatistic() on a Gaussian glm with a
zero weight, or with data the tests refuse: the refusal of the weight or of
the rows, where it refused the glm.
Unchanged: every other value of a valid call; a transformed response under
null-imposed resampling, refused by bootstrapTestStatistic() ("Null-imposed
resampling needs a plain response variable"); a MASS::rlm() fit, refused by
bootstrapTestStatistic() ("data does not reproduce the fit"), until a later
step; data = NULL given to bootstrapTestStatistic(), which needs data; a
fit with na.action = na.exclude, until step 9 (below); and every message the
test suite pins. The
White bootstrap no longer runs the checks of the model on each replicate, and
no longer gives their warnings on the residuals of a replicate, such as
"Residual outliers detected". On a factor with a contrasts matrix as its
attribute, the bootstraps no longer give the warning "contrasts dropped from
factor", which the refits from the formula gave once for each model frame
they rebuilt (20 times in a White bootstrap of 19 replicates, once in
bootstrapTestStatistic() under null-imposed resampling), and, under pairs
resampling, on each replicate that left a level out ("... due to missing
levels"). A corpus of 37,194 calls (scripts/model-context-identity.R),
every test and the bootstraps above on 378 combinations of fits, offsets,
terms, codings of a factor, forms of data, dropped rows and row names, with
the Mammen multipliers, four statistics bootstrapped, pairs resampling, the
bootstraps of a fitWLS() fit of each model and the runner with the
resampling tests, returns what it returned before this step, identical(),
and leaves the generator as it left it, but for the 1,229 calls of the changes
above, each change to a value verified against a reference; on a factor coded
by contrasts =, whose replicates were refitted without its contrasts, the
values of before but for rounding, within 7e-13 relative.
The model context, step 9: na.exclude¶
The ninth of the eleven steps of the model context reads a fit with
na.action = na.exclude as the same fit with na.omit. Such a fit drops the
rows na.omit drops, and residuals(), fitted(), weights() and
hatvalues() hold an NA at each of them. The checks of the model read
those residuals and refused the fit, as one whose residuals are not finite,
and so did every test that runs them and the runners; RESET stopped on its
fitted values; and the panel tests and the scatter diagnostic, which have no
such check, read its padded residuals against the rows of data by position.
The checks and every test built on the context now read one value per
observation of the fit, so a fit with na.exclude gives what the same fit
with na.omit gives on the same data: the same statistic, p-value, refusal
and warnings, and for the resampling tests the same draws of the generator.
An output with one value per row is padded with NA at the rows the fit
dropped, as residuals() of such a fit is. No export or argument changes.
The examples below use these data, z missing in rows 3, 7 and 11, with
m <- lm(y ~ x + z, data = sim, na.action = na.exclude); the bootstraps draw
199 replicates after set.seed(1):
set.seed(42)
n <- 120
sim <- data.frame(id = rep(1:12, each = 10), time = rep(1:10, 12),
x = runif(n, 1, 5), z = rnorm(n))
sim$y <- 1 + 2 * sim$x + rnorm(n, sd = 0.3 + 0.6 * sim$x)
sim$z[c(3, 7, 11)] <- NA
The tests and the runners take a fit with na.exclude. Each gives the
value of the same fit with na.omit, with its warnings, "Removed 3
observations due to missing values in z" where a test given sim gives it:
| Call | 0.16.0 | Now |
|---|---|---|
performKoenkerTest(m, sim) |
Error: "Model residuals must be finite before running Koenker." | 13.93 (p = 0.00095) |
performWhiteTest(m, sim) |
Error: "Model residuals must be finite before running White." | 14.20 (p = 0.014) |
performGQTest(m, sim, order_by = "x") |
Error: "Model residuals must be finite before running Goldfeld-Quandt." | 5.270 (p = 1.2e-07) |
performArchLMTest(m) |
Error: "Model residuals must be finite before running ARCH LM." | 0.141 (p = 0.707) |
performRESETTest(m) |
Error: "NA/NaN/Inf in 'x'" | 1.392 (p = 0.253) |
runHeteroTests(m, sim) |
Error: "Diagnostic execution failed for White: Model residuals must be finite before running White." | White 14.20, Koenker 13.93 |
performWhiteTestBootstrap(m, sim, B = 199) |
Error: "Model residuals must be finite before running Bootstrap White." | 14.20 (p = 0.040) |
bootstrapTestStatistic(performKoenkerTest, m, sim, B = 199) |
Error: "Model residuals must be finite before running Bootstrap helper." | 13.93 (p = 0.005) |
So do the other tests built on the context, their registered names and
test_factory entries, runDiagnostics(), runTimeSeriesTests() and
runAdvancedDiagnostics(), and, through the checks of the model they share,
rvalidate_against_reference(), rcalculateEffectSize() and
restimate_test_power(); rvalidate_against_reference() hands
lmtest::bptest() and car::ncvTest() the fit read as na.omit, as lmtest
stops on the padded weights of a weighted fit ("incompatible dimensions"). A
call the same fit with na.omit refuses is refused with its message, where it
was refused as one whose residuals are not finite: data the tests cannot
match, a variable missing from data or from the model frame, and the other
refusals of each test. RESET returned F = NA where the dropped rows were the
last ones of the data, and gives the value of the same fit with na.omit.
The panel tests and the scatter diagnostic read a fit with na.exclude at
the rows it kept. They read its residuals, padded with NA, against the rows
of data by position, as before 0.16.0: the random-effects statistic and the
correlations were NA, Pesaran's statistic was computed with the NA of the
rows the fit dropped, data in another order were refused, and so was an
unbalanced panel of the rows of data. They read the residuals of the fit and
the id, time and vars of the rows that hold its observations, as for the
same fit with na.omit:
| Call | 0.16.0 | Now |
|---|---|---|
performPesaranTest(m, sim, "id", "time") |
-0.315 (p = 0.753), computed with the NA |
-0.702 (p = 0.482), that of plm::pcdtest() on the rows of the fit |
performBPRandomEffectsTest(m, sim, "id") |
NA |
1.973 (p = 0.160) |
performScatterDiagnostic(m, sim, c("x", "z")) |
NA, NA |
0.377, 0.073 |
performBPRandomEffectsTest(m, sim[sample(n), ], "id"), after set.seed(3) |
Error: "... it reads data by position, the i-th row with the i-th observation, and the rows of data, matched by name, are not in the order of the fit." |
1.973 (p = 0.160) |
performPesaranTest(<m on sim[-(1:5), ]>, sim[-(1:5), ], "id", "time") |
Error: "performPesaranTest() reads a fit with na.action = na.exclude by position ... the panel of the rows of data is unbalanced ..." |
-0.597 (p = 0.551) |
runPanelTests() follows. A unit or a period missing only at rows the fit
dropped is not read, as for the same fit with na.omit: Pesaran's test
refused a pair of id and time of a dropped row, or a period missing there,
as at an observation of the fit.
fitWLS() pads its estimated variances. Its fit keeps the na.action of
the model, so residuals(), fitted() and weights() of a fit with
na.exclude are padded, as before; the estimated variances it attaches as the
attribute "variance_model" had one value per observation of the fit, 117 on
fitWLS(m), and have one per row, 120, NA at rows 3, 7 and 11. The record
of the variance function ("variance_function"), which the bootstraps read
by observation, keeps one row per observation, as decided by the author.
An mgcv::gam() fit with na.exclude, and the functions outside the
tests. From the review of the step: model.matrix() of a gam with
na.exclude pads its design with NA rows, and the tests refused such a fit
("Internal error: the parts of the model context differ in length."); it is
read through the design of the same gam with na.omit. And the functions
that run the checks of the model, or read a fit past them, read its residuals
and fitted values without the padding too. With
g <- mgcv::gam(y ~ x + z, data = sim, na.action = na.exclude):
| Call | 0.16.0 | Now |
|---|---|---|
performWhiteTest(g, sim), performKoenkerTest(g, sim) |
Error: "Model residuals must be finite ..." | 14.20 and 13.93, those of the same gam with na.omit |
performPesaranTest(g, sim, "id", "time") |
Error: "Internal error: the parts of the model context differ in length." | -0.702 (p = 0.482) |
performInfluenceDiagnostics(g)$cooks_distance[4] |
0.00143, with the warning "longer object length is not a multiple of shorter object length": cooks.distance() of the gam recycled the padded residuals |
0.00338, the third distance of the same gam with na.omit; NA at row 3 |
generateHeteroRecommendations(m) |
Error: "... Model residuals must be finite ..." | the recommendations of the same fit with na.omit |
autoCompareRemediations(m, sim)$metrics$RMSE |
NA, NA, 2.28 |
2.277, 2.284, 2.280 |
analyzeMLResiduals(m, sim)$rmse_reduction |
NA |
0 |
plotBubbleVariance(m), and so plotDiagnosticSuite(m) and generateDiagnosticReport(m) |
Error: "replacement has 120 rows, data has 117" | the plot of the same fit with na.omit |
plotResidualsFitted(m), plotSpreadLevel(m), plotResidualDensity(m), plotResidualQQ(m), plotBeforeAfter(m, fitWLS(m)) |
the plot, with the warning "Removed 3 rows containing non-finite outside the scale range" | the plot of the same fit with na.omit, without it |
dim(generics::augment(runHeteroTests(m)$white)) |
Error: "Diagnostic execution failed for White: ..." | 120 rows, 7 columns: the 117 rows of the same fit with na.omit, with a row of NA at rows 3, 7 and 11 |
performVIFDiagnostic() on such a gam read the padded design too, and
fitWLS(g) refitted it, stopping with "incompatible dimensions"; it returns
the fit of the same gam with na.omit, its weights padded. Given the
data of the fit, augment() binds its padded columns to the rows of the data,
120 rows; given data of another number of rows, on a fit with na.exclude or
na.omit, it says what it binds ("augment() binds one row of data to each
fitted value of the model, 117 of them, and data has 120 rows."), where it
stopped with "replacement has 117 rows, data has 120", or recycled the fitted
values (below). The Cook's distances of
performInfluenceDiagnostics() are padded as naresid() pads them: on a
weighted lm with na.exclude, whose cooks.distance() left the names of the
padded positions NA, they are named as the rows the fit dropped, as on an
unweighted one.
New refusals, on a fit without na.exclude too. generics::augment() of
the results of the runner, given data whose number of rows is a multiple of
the number of fitted values, recycled the fitted values, the residuals and the
standardised residuals over those rows, binding them to rows of other
observations: generics::augment(runHeteroTests(lm(y ~ x1 + x2, d), d,
use_cache = FALSE, progress = FALSE), data = rbind(d, d)) returned 480 rows,
each fitted value twice, and was refused in this step with the message above
("... 120 of them, and data has 240 rows."); step 10 binds the 120 rows of
the first copy (below). Given the data of the fit, or its model frame
without data, it returns what it returned. And
recommendRemediationStrategies(results, model, data) reads one row of data
for each observation of the fit: given the full data of a fit that dropped
rows, it stopped on the same fit with na.omit with "incompatible dimensions",
and read the padded residuals of a fit with na.exclude against the rows by
position, returning a value; both are refused with a message that names the
two counts ("recommendRemediationStrategies() reads one row of data for each
observation of the fit, 117 of them, and data has 120 rows."). Given the
model frame, both give the value; generateHeteroRecommendations(), which
calls it, refused such data before, and still does. Step 10 (below) matches
such data to the rows of the fit, and both give the value of those rows.
Unchanged, but for these two refusals: every call on a fit without
na.exclude; the spread-level and Davidian-Carroll tests on such a fit, which
left out the NA themselves and gave the value of the same fit with na.omit
(3.959, p = 0.00013, for the spread-level test on m); and fitWLS()'s
coefficients and weights of an lm. A corpus of 41,604 calls
(scripts/model-context-identity.R) returns what it returned in the previous
step, identical(), and leaves the generator as it left it, but for 2,590
calls on 30 fits with na.exclude, of the response, a regressor, the group,
the units and periods of a panel, the variable of a second formula, in every
form of data, weighted, with an offset, a Gaussian glm, a gam with and
without a smooth term, and a balanced and an unbalanced panel, each of which
gives what it gives on the same fit with na.omit, with its warnings and its
draws.
The model context, step 10: the runners and the clean-up¶
The tenth of the eleven steps of the model context. A grouped run of a
fitWLS() fit is refused; the runners take the data a tidymodels workflow was
fitted to; the functions outside the tests that read data row by row match
its rows to the observations of the fit, as the tests do; and the preparation
of data the tests used before the model context is gone. No export or
argument changes; hardhat, which workflows imports, joins the suggested
packages. The examples use the data of step 9, sim, with z missing in
rows 3, 7 and 11, sim0, the same data before z was removed, with a
column half, and sim2, sim0 with a response whose variance grows faster
with x:
set.seed(42)
n <- 120
sim0 <- data.frame(id = rep(1:12, each = 10), time = rep(1:10, 12),
x = runif(n, 1, 5), z = rnorm(n))
sim0$y <- 1 + 2 * sim0$x + rnorm(n, sd = 0.3 + 0.6 * sim0$x)
sim <- sim0
sim$z[c(3, 7, 11)] <- NA
sim2 <- sim0
set.seed(5)
sim2$y <- 1 + 2 * sim2$x + rnorm(n, sd = 0.1 * sim2$x^2)
sim0$half <- rep(c("first", "second"), each = 60)
set.seed(3)
perm <- sample(n)
New refusal: a grouped run of a fitWLS() fit. Given grouped data,
runHeteroTests() refits the model within each group by evaluating its call
on the rows of the group. The call of a fitWLS() fit is that of the model it
refitted, without the weights it estimated, so each group was tested as the
unweighted fit of its rows, without an error. Such a run is refused, with the
class of the refusals of the model context, before any group is refitted.
With fw <- fitWLS(lm(y ~ x + z, data = sim0), sim0):
| Call | Before | Now |
|---|---|---|
runHeteroTests(fw, dplyr::group_by(sim0, half)) |
White 8.335 (p = 0.139) and 8.567 (p = 0.128), Koenker 7.286 and 6.508: the values of lm() on each half |
Error: "runHeteroTests() cannot run a grouped analysis of a fitWLS() fit: each group is refitted by evaluating the call of the model on its rows, and that call does not hold the weights fitWLS() estimated, ..." |
fitWLS() fitted within each half gives White 4.924 and 6.755: fit it within
each group and test each fit, or pass the data ungrouped to test the fit as a
whole (White 4.562). A grouped run of a weighted lm, whose weights are a
column of the data that the refit of each group reads at its rows, of an
unweighted lm and of a formula, is unchanged, and so is runDiagnostics(),
which runs no grouped analysis.
The runners take the data a tidymodels workflow was fitted to. A workflow
is tested on the fit of its engine, which names its response ..y and its
predictors as the workflow's preprocessing made them, and the runners refused
the data the workflow was fitted to; the tests refused the data of a parsnip
fit of fit_xy(), which names its response ..y too. The runners map such
data to the variables of the engine by the workflow's own preprocessing
(hardhat::forge()), the outcome as the preprocessing makes it among them, or
the outcome of fit_xy() to ..y, match their rows to the observations of the
fit, every column of its model frame compared, and test the model frame of the
engine's fit, which gives the value without data. The results keep what
generics::augment() needs of the workflow, so that it maps its data in the
same way (attr(, "tidymodels"), below). With
wf <- workflows::fit(workflows::add_model(workflows::add_formula(workflows::workflow(), y ~ x + z), parsnip::linear_reg()), data = sim0):
| Call | Before | Now |
|---|---|---|
runHeteroTests(wf, sim0) |
Error: "A tidymodels workflow is tested on the fit of its engine, ... and data lacks ..y. Pass data = NULL, which tests the model frame of that fit." |
White 14.68 (p = 0.012), Koenker 14.47 (p = 0.00072), as runHeteroTests(wf) |
the same with a recipe, recipes::step_log(recipes::recipe(y ~ x + z, data = sim0), x) |
the same error | White 14.95 (p = 0.011), Koenker 13.78 (p = 0.0010), as without data |
the workflow fitted to sim, given sim |
the same error | White 14.20 (p = 0.014), Koenker 13.93 (p = 0.00095), as without data |
runDiagnostics(wf, sim0) |
the same error | White 14.68, RESET 1.273, as without data |
runHeteroTests(parsnip::fit_xy(parsnip::linear_reg(), x = sim0[, c("x", "z")], y = sim0[, "y", drop = FALSE]), sim0) |
Error: "Data quality issue detected for White: The following variables are missing from input data: ..y. ..." | White 14.68, Koenker 14.47, as without data |
The engine's fit of a workflow numbers its observations in the order of the
data it was fitted to, so data in another order are refused, with the class of
the refusals of data, and so is a subset of them ("Missing rows: 101, 102,
103"), with the same advice. From the review of the step:
- A transformed outcome is mapped too (decided by the author). A recipe
with
step_log()of the outcome, or a formula preprocessorlog(y) ~ x + z, maps the outcome ofdatato the outcome it makes, which is the response of the engine's fit, and the value is that withoutdata: White 3.495 and Koenker 2.825 on a recipe withstep_log()ofyand on a formula preprocessorlog(y) ~ x + z, fitted tosim0and given it, where both were refused ("datalacks ..y"). The first version of the step refused such data. - Case weights are compared. The case weights of a workflow
(
workflows::add_case_weights()) are mapped under the name the engine's call gives them, so other weights are refused, as the weights of a fit are for the tests: withcw <- hardhat::importance_weights(1 / x), the data of the workflow give White 6.21 and Koenker 5.245, as withoutdata, and the same data with the weights reversed are refused ("(weights) differs from the model frame"). - Recipe steps that drop rows. A step that drops rows at the fit and is
skipped on new data, as
step_naomit()andstep_filter()are by default, numbers the rows it kept afresh. The rows ofdatacomplete in the variables of the engine are taken when they are as many as the rows it kept, and confirmed: a recipe withstep_naomit()of the predictors, fitted tosimand given it, gives White 14.20 and Koenker 13.93, as withoutdata, where it was refused. Otherwise the data are refused with a message that names such a step: withstep_filter(x > 1.5), which keeps 106 rows ofsim0, "... its preprocessing kept 106 rows of the data it was fitted to, numbered afresh, where it maps the 120 rows ofdatawithout dropping any: a step it skips on new data, such asstep_naomit()orstep_filter()withskip = TRUE, dropped rows at the fit only, ...". A step applied to new data (skip = FALSE) maps the rows it keeps. - A lazy
dtplyrtable is mapped too. It is collected first:runHeteroTests(wf, dtplyr::lazy_dt(sim0))gives White 14.68 and Koenker 14.47, as withoutdata, where it was refused, the names of the lazy object read for those of the data ("datalacks ..y, x, z"), as in the first version of the step. - Grouped data given with a workflow are refused, as before, with a message
that says that only an ungrouped data frame is mapped. Data that cannot be
mapped are refused with a message that says why: data that lack a column the
preprocessing needs, the package hardhat not installed, and the outcome of
fit_xy()given as a vector, which the fit does not record.
From the second and third reviews of the step, with d0 <- sim0[c("x", "z", "y")]:
- New refusal:
augment()of a workflow whose recipe reorders the rows.step_arrange(), which a recipe applies to new data, sorts the rows it maps, andaugment()bound each row ofdatathe fitted value and the residual of the row the sort put in its place. The rows a workflow's preprocessing maps are traced to the rows ofdatathey were made from, and the data of a preprocessing that reorders them, or makes a row from other rows, are refused byaugment(). The runners, which test the model frame of the engine once its rows are confirmed, give the value withoutdata. Withwa, a workflow ofrecipes::step_arrange(recipes::recipe(y ~ x + z, data = d0), x)fitted tod0:
| Call | Before | Now |
|---|---|---|
generics::augment(runHeteroTests(wa)$white, data = d0) |
120 rows, each with the fitted value of another: rows 1 and 2 with 3.244 and 3.223, where theirs are 10.589 and 10.611, and 8.079 off at most (7.496 with dplyr::desc(z)); so in the first version of the step |
Error: "augment() cannot tell which rows of data hold the observations of the fit: the workflow's preprocessing reorders the rows of data it maps, or makes a row from other rows, as step_arrange() does, so the row of data each observation comes from cannot be told. Pass data = NULL, which binds the model frame of the fit." |
runHeteroTests(wa, d0) |
Error: "... data lacks ..y. ..." |
White 14.68, Koenker 14.47, as without data |
- A step a recipe skips on new data is named.
step_log(y, skip = TRUE), the idiom recipes recommends for the outcome, transforms the outcome at the fit only, so the data the workflow was fitted to, mapped by its preprocessing, hold it untransformed. They were refused ("datalacks ..y"), and in the first version of the step with the advice to pass the data the workflow was fitted to, in its order, which they were ("... ..y differs from the model frame"); the refusal now says what stands between them: "runHeteroTests() cannot mapdatato the variables of the fit of the workflow's engine: the workflow's preprocessing has a step it skips on new data,step_log()withskip = TRUE, which changed the outcome, a predictor or the order of the rows of the data it was fitted to, sodata, mapped by that preprocessing (hardhat::forge()), do not hold the values the engine was fitted to. Passdata = NULL, which tests the model frame of that fit." Withoutdatasuch a workflow gives White 3.495 and Koenker 2.825. The steps named are those without which the rows are not confirmed: with astep_filter(x > 0)before it, skipped on new data too and keeping every row, the second version of the step named "step_filter()andstep_log()", and namesstep_log()alone (from the third review). - Case weights are compared on the rows a recipe keeps. The first version
of the step mapped the case weights only when the preprocessing kept every
row of
data, and took other weights when a recipe dropped rows of new data: withcw <- hardhat::importance_weights(1 / x)andstep_filter(x > 1.5, skip = FALSE), which keeps 106 rows, the data with the weights reversed gave White 4.102 and Koenker 2.986, the values of the fit with its own weights. A recipe gives the weights of the rows it keeps, in the order it keeps them, and those data are refused ("(weights) differs from the model frame"); weights that cannot be told for each row are refused. - New refusal: the data of a
parsnip::fit_xy()fit with case weights.fit_xy()takes its case weights as a vector, and the fit records the column of its outcome but not that of its weights, so the weights ofdatacannot be compared with those of the fit. Withdw,d0with a columnw = 1 / x, andpx <- parsnip::fit_xy(parsnip::linear_reg(), x = dw[, c("x", "z")], y = dw[, "y", drop = FALSE], case_weights = hardhat::importance_weights(dw$w)),runHeteroTests(px, dw)was refused ("missing from input data: ..y"), and the second version of the step gave White 6.210 and Koenker 5.245 on these data and on the data withwreversed alike, the values of the fit with its own weights. Both are refused, from the third review: "runHeteroTests() cannot compare the case weights of a parsnip fit made withfit_xy()with those ofdata:fit_xy()takes them as a vector, and the fit does not record which column ofdataheld them, sodatacannot be confirmed to be the data the fit was made from. Passdata = NULL, which tests the model frame of that fit." Withoutdatait gives White 6.210 and Koenker 5.245. A parsnip fit offit(), given its case weights as a vector too, is matched todataby the variables of its formula, as anlmwhose weights are a vector of the workspace is, and its weights are not compared. - The results of the runners on a workflow or a parsnip fit carry
attr(, "tidymodels"). Each result keeps whatgenerics::augment()needs to map the data of the workflow as the runners map them: the blueprint of its preprocessing, the number of rows it kept and the column of its case weights, or the outcome of a parsnip fit offit_xy().runHeteroTests(wf)andrunDiagnostics(wf), with or withoutdata, return what they returned with this attribute on each result, and are no longeridentical()to the results of 0.16.0; two calls on the same workflow giveidentical()results. The blueprint is kept as the workflow keeps it, and no fit (decision 36, decided by the author on the fourth review). A recipe step keeps the environment it was written in, so that a saved result carries whatever that environment holds, as saving the workflow does. Serialized, the results ofrunHeteroTests(wf)with three tests on 5,000 rows take 1,523 KB, where they took 1,522 KB; those of a workflow whose recipe was written in a function whose frame held a 50 MB data frame take 51,245 KB, where they took 1,894 KB, the attribute 49,347 KB of them, andsaveRDS()of that workflow takes 50,272 KB. The first version of the step kept the whole workflow, 2,794 KB on 5,000 rows. The third rewrote the expressions of the steps, writing the variables they read of the frame into them as values: that changed what some of them mean or broke them (a variable a later step makes a column, a variable named as a column a step removed, a function called by its name as a string, a variable holding a symbol or a call, where the runners gave White 14.6845, or 14.8861 for the removed column, andaugment()each row its own value, as they do), and copied the values into each result: 114,196 KB for a step reading one value of a local vector of 5,000,000 numbers, against 39,418 KB as the step keeps it. - New refusal, of values that were right:
augment()of a recipe step that makes a row from other rows. The rows a workflow's preprocessing maps are traced to the rows ofdataby mappingdatain two other orders, and a step that makes a row from other rows, ascumsum(),dplyr::row_number()or a mean overdplyr::lag()instep_mutate()do, orstep_arrange()given data already in its order, does not mapdatain another order to the same rows in that order, and is refused, though on data in the order of the fit 0.16.0 bound each row its own values: the refusal is conservative (decision 14), and never gives a wrong value. Withwc, a workflow ofrecipes::step_mutate(recipes::recipe(y ~ x + z, data = d0), cs = cumsum(x))fitted tod0:
| Call | Before | Now |
|---|---|---|
generics::augment(runHeteroTests(wc)$white, data = d0) |
120 rows, each with its own fitted value: rows 1 and 2 with 10.396 and 10.431 |
Error: "augment() cannot tell which rows of data hold the observations of the fit: the workflow's preprocessing reorders the rows of data it maps, or makes a row from other rows, as step_arrange() does, ..." |
runHeteroTests(wc, d0) |
Error: "... data lacks ..y, cs. ..." |
White 16.14, Koenker 15.37, as without data |
The recommendations and augment() match the rows of data as the tests
do. recommendRemediationStrategies() correlates the residuals with the
columns of data row by row, generateHeteroRecommendations() hands data to
the tests and to that advice, and generics::augment() of the results of the
runner binds the columns of the fit to its rows. They read the i-th row of
data with the i-th observation, and since step 9 refused data of another
number of rows. They now read the rows of data that hold the observations of
the fit, found as the tests find them: named data in any order, numbered data
in the order of the fit and the full data of a fit that dropped rows give the
value of the rows of the fit, and on a fit with na.exclude the value of the
same fit with na.omit. With m <- lm(y ~ x + z, data = sim),
me <- lm(y ~ x + z, data = sim, na.action = na.exclude),
res <- runHeteroTests(m, sim), m0 <- lm(y ~ x + z, data = sim0) and
m2 <- lm(y ~ x + z, data = sim2):
| Call | Before | Now |
|---|---|---|
recommendRemediationStrategies(res, m, sim) |
Error: "recommendRemediationStrategies() reads one row of data for each observation of the fit, 117 of them, and data has 120 rows. ..." |
the advice of the 117 rows of the fit ("No dominant variance pattern detected."), with na.exclude too |
generateHeteroRecommendations(m, sim) |
Error: "Input validation for recommendation_engine failed: Model reports 117 residuals but the supplied data has 120 rows. ..." | the report of the 117 rows of the fit: White 14.20, Koenker 13.93, with na.exclude too; its profile is that of sim, 120 rows with their missing values |
recommendRemediationStrategies(runHeteroTests(m2, sim2), m2, sim2[perm, ])$pattern |
"Variance appears to increase with fitted values.", from the residuals read against other rows | "Variance is strongly associated with x higher values.", as on sim2 |
generateHeteroRecommendations(m2, sim2[perm, ])$remediation$pattern |
"Variance appears to increase with fitted values." | "Variance is strongly associated with x higher values.", as on sim2 |
augment() binds as broom does (decided by the author). The rows of
data are matched to the observations and bound in the order of data, each
with its own fitted value, residual and standardised residual: on a fit with
na.omit the rows the fit used, and on a fit with na.exclude every row,
with NA at the rows it dropped, as broom::augment() binds the full data of
such a fit in their order:
| Call | Before | Now |
|---|---|---|
generics::augment(res$white, data = sim) |
Error: "augment() binds one row of data to each fitted value of the model, 117 of them, and data has 120 rows. ..." |
117 rows, the rows the fit used, 1, 2, 4, ..., in the order of sim, each with its fitted value; shuffled, sim[perm, ], the same rows in that order |
generics::augment(runHeteroTests(me, sim)$white, data = sim[perm, ]) |
rows 5, 58 and 12 with the fitted values of observations 1, 2 and 3 by position, 10.563, 10.580 and NA (row 3 is one the fit dropped), and NA at rows 12, 104 and 20 |
rows 5, 58 and 12 with theirs, 8.379, 3.368 and 8.833, and NA at rows 3, 7 and 11, the rows the fit dropped |
head(generics::augment(runHeteroTests(m0, sim0)$white, data = sim0[perm, ]), 2) |
rows 5 and 58 of sim0 with the fitted values of observations 1 and 2, 10.589 and 10.611 |
rows 5 and 58 with theirs, 8.398 and 3.417 |
Given the full data of a fit with na.exclude in their order, augment()
binds what it bound. Given the rows of its observations only, as its model
frame without data, such a fit binds them in the order of the fit with a row
of NA at each row it dropped, as in step 9. The first version of the step
refused, on augment() of the results of a workflow, the data it was fitted
to ("missing from input data: ..y"), where 0.16.0 bound them by position. A
preprocessing that drops or adds rows of new data, such as step_filter()
with skip = FALSE, does not say which rows of data it kept, nor one that
reorders them, such as step_arrange(), which row of data each observation
comes from, and augment() refuses their data (above).
augment() given the data of the fit twice binds the rows of the first
copy. generics::augment(runHeteroTests(m0, sim0)$white, data = rbind(sim0,
sim0)) binds the 120 rows of the first copy, 1 to 120, each with its own
fitted value, as the tests take the rows of the fit and leave out the rows
after them; 0.16.0 bound the 240 rows, each fitted value twice, and step 9
refused them ("... 120 of them, and data has 240 rows.").
The report profiles data as given. generateHeteroRecommendations()
reads the rows of the fit for the checks, the tests and the advice, which read
the residuals, and profiles data as given, its size and its missing values,
so its advice on missing data fires: with simna, sim0 with z missing in its first 40 rows, generateHeteroRecommendations(lm(y ~ x + z, simna), simna[, c("y", "x", "z")]) was refused ("Model reports 80 residuals but the supplied data has 120 rows") and profiles 120 rows with 11% of their values missing, advising multiple imputation; the first version of the step profiled the 80 rows of the fit, with none missing. Grouped data keep their
groups on the rows of the fit, so the analysis of a grouped tibble is grouped
whether or not the fit dropped rows; the first version of the step returned
them ungrouped when it did.
New refusals. What the tests refuse of data, these functions refuse
too. Where they computed a value from other rows: numbered data in another
order than the fit (dplyr::as_tibble(sim0)[perm, ], a shuffled data.table,
dplyr::arrange()), repeated row names, and data in the order of the fit
whose rows tie on every column of the model frame (the tibble of a fit to
repeated rows: they read every column of data, as a test that reads a
column beyond the model frame does); and, by augment() of the results of a
workflow, its data in another order, bound by position before. And, from the
review of the step, where they computed a value from the model or from the
data without matching them:
| Call | Before | Now |
|---|---|---|
recommendRemediationStrategies(res, m0, sim0[, c("id", "time")]), data without the variables of the model |
"No dominant variance pattern detected.", from id and time read against the residuals by position |
Error: "recommendRemediationStrategies() matches the rows of data to the observations of the fit by the variables of the model, and data lacks y, x, z. Pass the data the model was fitted to, with the columns to examine beside them, such as cbind(model.frame(model), columns) ..." |
recommendRemediationStrategies(res, glm(I(y > median(y)) ~ x + z, data = sim0, family = binomial()), sim0), a model the tests refuse |
"No dominant variance pattern detected." | Error: "recommendRemediationStrategies() needs a model fitted by stats::lm(): a glm with the binomial family and the logit link is not a least-squares fit. ..." |
The models the tests refuse before matching data, a glm of another family
or link than the Gaussian with the identity, a multivariate lm and a fit with
model = FALSE whose frame cannot be rebuilt or was rebuilt from data changed
after the fit, are refused by the three, as by every function built on the
model context; augment() bound the values of such a glm, and the advice
computed its correlations. Each refusal has the message and class of the
tests' refusal, naming the function; data that lack the variables of the
model, which augment() and the advice read by position, are refused with a
message that says what to pass.
The clean-up. prepare_model_data_for_test(), rdata_or_model_frame(),
rcheck_data_rows() and rcheck_rows_after_cleaning(), internal functions
no test has called since step 8, are gone; no value changes.
Unchanged, but for the changes above: every test, and every other call of the
runners; their results on a workflow or a parsnip fit carry the attribute
tidymodels (above), and are otherwise what they were. A corpus of 43,226
calls (scripts/model-context-identity.R), the three functions above on every
fit and form of data of the corpus among them, and the attributes of the
results of the runners compared by name, returns what it returned in the
previous step, identical(), and leaves the generator as it left it, but for
856 calls of these changes, each change to a value that of the same
call on the rows of the fit, or without data, and for the results of the
runners on a workflow without data the value of the previous step but for
their attribute; and each value of
augment() of a workflow checked row by row against the prediction of its
engine on that row alone.
The model context, step 11: MASS::rlm() and the penalised mgcv::gam()¶
The last of the eleven steps of the model context, in two parts. A
MASS::rlm() fit, which every test took for an lm, is refused. A penalised
mgcv::gam() fit with smooth terms is tested on the basis of its smooths by
the tests that do not need least squares, the streaming tests included, and
refused by those that do; a gam that is the least-squares fit of its design,
without smooth terms or with fixed ones, is taken by every test (decisions 37
and 38 of the design of the model context, decided on 2026-10-29). No export
or argument changes. The examples use the design of the Supported models
page:
set.seed(20261015)
n <- 120
d <- data.frame(x1 = runif(n, 0.5, 2), x2 = rnorm(n),
group = factor(rep(letters[1:4], length.out = n)),
id = rep(1:12, each = 10), time = rep(1:10, 12))
d$y <- 1 + d$x1 - d$x2 + rnorm(n, sd = d$x1)
m <- MASS::rlm(y ~ x1 + x2, data = d)
g <- mgcv::gam(y ~ s(x1) + x2, data = d) # penalised
f <- mgcv::gam(y ~ s(x1, fx = TRUE) + x2, data = d) # a fixed smooth
New refusal: a MASS::rlm() fit. An rlm fit is an lm by class and an
M-estimate: its coefficients minimise a robust loss of the residuals, by
iteratively reweighted least squares with weights that depend on them, so its
residuals are not least-squares residuals, and the null distributions of the
tests assume least squares. 37 of the 39 tests that take a model tested it as
an lm, its M-estimation residuals as if they were least-squares ones; the
quantile regression and variance form tests refused it as a weighted fit, its
case weights being all 1. The studentized Breusch-Pagan statistic refitted its
design by least squares, and so tested another fit, the lm() of the same
formula. Every function built on the model context refuses it now, before
data is matched, with the class of the refusals of the model context,
heteroTests_model_context_error, and a message that says what to test
instead; the runners keep the refusal final:
| Call | Before | Now |
|---|---|---|
performStudentizedBPTest(m, d), performBPTestRobust(m, d) |
9.04 (p = 0.011), the value of lm(y ~ x1 + x2, data = d) |
Error: "performStudentizedBPTest() cannot test a MASS::rlm() fit: an rlm fit is an M-estimate, whose residuals are not least-squares residuals, and the null distributions of the tests assume least squares. Test the lm() fit of the same formula instead, lm(y ~ x1 + x2, data = d)." (performBPTestRobust() names itself) |
performKoenkerTest(m, d) |
8.74 (p = 0.013), on the residuals of the M-estimate | Error: "performKoenkerTest() cannot test a MASS::rlm() fit: ..." |
performWhiteTest(m, d) |
11.01 (p = 0.051) | Error: "performWhiteTest() cannot test a MASS::rlm() fit: ..." |
performBPTest(m, d) |
12.42 (p = 0.0020) | Error: "performBPTest() cannot test a MASS::rlm() fit: ..." |
runHeteroTests(m, d) |
White 11.01, Koenker 8.74 | Error: "Model cannot be tested for White: performWhiteTest() cannot test a MASS::rlm() fit: ..." |
runHeteroTests(m, d, tests = "quantile_regression") |
0.341: the rank permutation test under the name quantile_regression, the quantile test having refused the fit as a weighted one, which is not final |
Error: "Model cannot be tested for Quantile Regression: performQuantileRegressionTest() cannot test a MASS::rlm() fit: ..." |
fitWLS(m) |
a fit of class rlm refitted by weighted least squares, its variance function fitted to the residuals of the M-estimate, which summary() could not read |
Error: "fitWLS() cannot refit a MASS::rlm() fit: an rlm fit is an M-estimate, whose residuals are not least-squares residuals, and fitWLS() fits its variance function to least-squares residuals. Pass the lm() fit of the same formula instead, fitWLS(lm(y ~ x1 + x2, data = d))." |
So do the other tests, with and without data and in every form of it, their
robust, resampling and streaming forms, bootstrapTestStatistic() and its
deprecated name, their registered names, test_factory entries and
cachedTest(), runHeteroTestsParallel(), runDiagnostics(),
runTimeSeriesTests(), runPanelTests(), runAdvancedDiagnostics(),
rvalidate_against_reference() and generateHeteroRecommendations(), and
recommendRemediationStrategies() and generics::augment() of a result given
data, which they match through the model context ("cannot take").
bootstrapTestStatistic() refuses it before it calls test_function; it
computed the observed statistic and then refused the fit as one its
least-squares refit does not reproduce ("data does not reproduce the fit"),
and the quantile regression and variance form tests refused it as a weighted
fit ("does not support weighted fits", "weights that did not come from
fitWLS()"): each now gives the refusal above. fitRobust() returns such a fit
for estimation, and its help page says that the tests refuse it. The functions
that run the tests through the runners stop with the refusal of the first
test: compareTestResults(m, d), which gave White's 11.01 and Koenker's 8.74,
summary() and test() of HeteroDiagnostic(m, d), and
generateDiagnosticReport(m, d), which wrote its report; and
compareModelDiagnostics(list(m), d) gives the fit a row of NA, with the
warning "Diagnostics for model 1 failed: Model cannot be tested for White:
...", where it gave 11.01 and 8.74. augment() of a result around such a fit,
which the runners no longer return, binds the columns of the fit to its model
frame without data, as before.
The lm() call the message names is built from the call of the fit: its
formula, and its data, weights, subset, na.action, offset and
contrasts, without the arguments only rlm() takes (psi, method,
maxit and the others), so that
MASS::rlm(y ~ x1 + x2, data = d, weights = w, subset = x1 > 0.6, psi = MASS::psi.bisquare)
is told to test lm(y ~ x1 + x2, data = d, weights = w, subset = x1 > 0.6).
An rlm of the matrix interface, MASS::rlm(X, y), which has no formula, is
told to test "the lm() fit of the same model".
The call of a fitRobust() result names what was given. It named the
variables of fitRobust(), rlm(formula = form, data = data): print() and
summary() showed that call, update() failed on it (it could not find
rlm() without MASS attached, and form and data are not the caller's
either), and the refusal of the tests suggested
lm(mpg ~ wt + qsec, data = data), which stops at top level, or fits
whatever object data names there. The call names the formula and the data
the caller gave now, MASS::rlm(formula = mpg ~ wt + qsec, data = mtcars), with the
other arguments as written (psi = MASS::psi.bisquare, where it recorded
..1), and, for an lm, the data, subset and na.action of its call:
print() and summary() show it, update() refits it, and the tests suggest
lm(mpg ~ wt + qsec, data = mtcars). The fit itself is unchanged.
New refusal: a grouped run of a MASS::rlm() fit. runHeteroTests() on
grouped data refits the model in each group from its call: with MASS
attached, runHeteroTests(m, dplyr::group_by(d, group)) refitted an rlm in
each group and returned its values (White's 5.65, p = 0.342, in group a).
It refuses the fit now before any group is refitted, as it refuses a
fitWLS() fit (since step 10), with the class of the model context:
"runHeteroTests() cannot test a MASS::rlm() fit: ...".
New refusal: autoCompareRemediations() of a MASS::rlm() fit. Given an
rlm fit, it compared the fit, fitWLS() of it and a robust refit by their
AIC and recommended fitWLS() of it (best "WLS", AIC 396.5 against
417.0). It refuses the fit now with the message of the tests,
"autoCompareRemediations() cannot take a MASS::rlm() fit: ...", and the
class of the model context. Given an lm, it still compares the rlm fit of
fitRobust() by its AIC, as before.
New refusals: a penalised mgcv::gam() fit, in the tests that need least
squares. A smooth term is penalised unless it is fixed
(s(x, fx = TRUE)), and the residuals of a penalised fit are not those of the
least-squares fit of its design, the basis of its smooths. The tests that fit
that design again by least squares, those whose replicates do, and those whose
null distribution is derived for least-squares residuals of the design refuse
it now with the class of the model context, and the runners keep the refusal
final:
| Call | Before | Now |
|---|---|---|
performHarrisonMcCabeTest(g, d, order_by = "x1") |
0.315 (p = 0.0025), from the null distribution of the least-squares residuals of the basis | Error: "performHarrisonMcCabeTest() cannot test a penalised mgcv::gam() fit (s(x1)): its residuals are not those of the least-squares fit of its design, the basis of its smooths, for which the null distribution of the test is derived. Fix its smooths, s(x, fx = TRUE), so that it is that fit, or refit the model with lm() and a parametric term such as splines::ns() in place of each smooth." |
performSzroeterTest(g, d, order_by = "x1") |
3.029 (p = 0.0012); the registered "szroeter", ordered by the fitted values, 1.634 (p = 0.051) |
Error: "performSzroeterTest() cannot test a penalised mgcv::gam() fit (s(x1)): ..." |
performVarianceFormTest(g, d) |
1.053 on 49 and 60 degrees of freedom (p = 0.42), the test of a weighted least-squares refit of the basis without the penalty | Error: "performVarianceFormTest() cannot test a penalised mgcv::gam() fit (s(x1)): ..., which the test fits again. ..." |
fitWLS(g, d) |
an lm of 11 coefficients, the weighted least-squares refit of the basis without the penalty, which every test then refused |
Error: "fitWLS() cannot refit a penalised mgcv::gam() fit (s(x1)): ..., which fitWLS() refits by weighted least squares, without the penalty. ..." |
performWildBootstrapTest(g, d) |
Error: "lm object does not have a proper 'qr' component", on every gam |
Error: "performWildBootstrapTest() cannot test a penalised mgcv::gam() fit (s(x1)): ..., which each replicate of the bootstrap fits again. ..." |
runHeteroTests(g, d, tests = "wild_bootstrap") |
46.05 on 60: White's test under the name wild_bootstrap, tagged as a fallback, the error of the wild bootstrap not being final |
Error: "Model cannot be tested for Wild Bootstrap: performWildBootstrapTest() cannot test a penalised mgcv::gam() fit (s(x1)): ..." |
bootstrapTestStatistic(performKoenkerTest, g, d), and performWhiteTestRobust() and performBPTestRobust(studentize = FALSE) with bootstrap = TRUE |
Error, without the class of the model context: "bootstrapTestStatistic() supports Gaussian lm models only: ... An mgcv::gam() fit is not taken: ...", on every gam |
Error: "bootstrapTestStatistic() cannot test a penalised mgcv::gam() fit (s(x1)): ..." (the robust tests name themselves) |
RESET, Goldfeld-Quandt, the quantile regression test, the studentized
Breusch-Pagan statistic and the White bootstrap refused such a fit already,
since 0.16.0 or step 8, and refuse it with this message, where theirs said
that a later step would settle it. The other tests read only the residuals,
the fitted values, columns of data or the columns of the design as
auxiliary regressors, and test such a fit on its design, as before;
runDiagnostics() refuses it as a whole, through RESET, as before, and
runHeteroTests() returns White's and Koenker's tests.
New refusals: a refusal of the model context in a group stops a grouped
run. runHeteroTests() on grouped data records the failure of a test in a
group and goes on, and it recorded a refusal of the model by the model
context so too, as a failed test whose statistic is NA, where the run on
ungrouped data stops with it (decision 6). It stops the grouped run now, with
the class and the message of the refusal; the failures of a test for another
reason are recorded in their group, as before. On the data of the examples
with a factor of two levels, d$group <- factor(rep(letters[1:2], length.out = n)),
and grouped <- dplyr::group_by(d, group):
| Call | Before | Now |
|---|---|---|
runHeteroTests(g, grouped, tests = "szroeter"), g a penalised gam |
0.687 and 0.777, in its two groups | Error: "Model cannot be tested for Szroeter: performSzroeterTest() cannot test a penalised mgcv::gam() fit (s(x1)): ..." |
runHeteroTests(g, grouped, tests = "student_bp") |
a failed test of NA in each group |
Error: "Model cannot be tested for Student Bp: performStudentizedBPTest() cannot test a penalised mgcv::gam() fit (s(x1)): ..." |
runHeteroTests(lm(cbind(y, x2) ~ x1, data = d), grouped) |
a failed test of NA for White's and Koenker's tests in each group |
Error: "Model cannot be tested for White: performWhiteTest() needs a model with one response; model has 2." |
The streaming tests read the basis of the smooths of a gam. They read
the design of the variables of its formula, which its model frame holds; they
read its stored design now, chunk by chunk, as the exact tests do, and give
the value of the exact test in chunks of any size, so runHeteroTests() gives
the same values with and without chunk_threshold_mb:
| Call | Before | Now |
|---|---|---|
performBPTestStreaming(g, d) |
11.19 on 2 (p = 0.0037) | 23.99 on 10 (p = 0.0076), the value of performBPTest(g, d) |
performKoenkerTestStreaming(g, d) |
8.54 on 2 (p = 0.014) | 18.31 on 10 (p = 0.050), the value of performKoenkerTest(g, d) |
performWhiteTestStreaming(g, d) |
10.61 on 5 (p = 0.060) | 46.05 on 60 (p = 0.91), the value of performWhiteTest(g, d) |
the same on f, the fixed smooth |
9.59 on 2, 6.55 on 2 and 8.22 on 5 | 21.49, 14.70 and 43.88, on 10, 10 and 60 |
Refusals become values: a gam that is the least-squares fit of its
design. A gam without smooth terms, or whose smooths are all fixed, is as
a rule the least-squares fit of its design (on an ill-conditioned design
mgcv's fit is not, below), and every test takes such a fit now as the lm()
of that design, those that fit it again included. Whether a gam is that fit
is decided from the fit: its design refitted by least squares, with its
weights and offset, gives its residuals to 1e-8 of the largest residual of
that refit, plus 1e3 times the machine epsilon of the largest value of the
response and of the offset, the rounding of a fit on that scale. So the
verdict does not depend on how the model is written: an offset or a trend
leaves it as it is, up to the scale at which that rounding hides a penalised
fit, which passes once the largest value of the response or of the offset is
about 1e12 times its residuals; and a fixed smooth on a response of 1e8 is
taken as least squares. A penalised fit whose smoothing parameter is near
zero passes that check (on g's formula, sp = 1e-10 does and sp = 1e-9
does not). A gam without a penalised term that fails it is refused with a
message of its own, not as a penalised fit (below):
| Call | Before | Now |
|---|---|---|
performStudentizedBPTest(f, d), performBPTestRobust(f, d) |
Error: "cannot test an mgcv::gam() fit with smooth terms (s(x1)): a smooth term is usually penalised, ..." |
14.70 on 10 (p = 0.14) |
performGQTest(f, d, order_by = "x1") |
Error, as above | 2.96 on 38 and 38 (p = 0.00058) |
performRESETTest(f, d) |
Error, as above | 2.61 on 2 and 107 (p = 0.078) |
performQuantileRegressionTest(f, d) |
Error, as above | 1.40 on 10 and 230 (p = 0.18) |
performWhiteTestBootstrap(f, d, B = 19) |
Error, as above | 43.88 (p = 0.95 after set.seed(7)) |
performWildBootstrapTest(f, d, B = 19) |
Error: "lm object does not have a proper 'qr' component" | 14.70 (p = 0.15 after set.seed(7)) |
bootstrapTestStatistic(performKoenkerTest, f, d, B = 19) |
Error: "bootstrapTestStatistic() supports Gaussian lm models only: ..." |
14.70 (p = 0.10 after set.seed(7)) |
performWildBootstrapTest() and bootstrapTestStatistic(performKoenkerTest) on mgcv::gam(y ~ x1 + x2, data = d), B = 19 |
Error, as above | 9.04 (p = 0.05 after set.seed(7)), each the value of the same call on lm(y ~ x1 + x2, data = d) |
performWhiteTestRobust(bootstrap = TRUE, B = 19) on the same gam |
Error, as above | White's 11.14 (bootstrap p = 0.20 after set.seed(7)), the value of the lm |
performBPTestRobust(studentize = FALSE, bootstrap = TRUE, B = 19) on the same gam |
Error, as above | 12.45 (bootstrap p = 0.05 after set.seed(7)), the value of the lm |
Each is the value of the same call on the lm() of the basis of the gam,
with the same replicates; the wild bootstrap reads the leverages of a gam
from its design, where hatvalues() stopped on it. fitWLS() of such a gam
refits its design, but returns an lm whose terms rebuild the variables of
the formula, so that the tests refuse it when the gam has smooth terms, as
since step 8.
New refusal: a gam without smooth terms on an ill-conditioned design.
mgcv's fit of a gam without a penalised term is not the least-squares fit
of its design when the design is ill-conditioned, as with a regressor far
from zero:
set.seed(20261015)
n <- 120
dd <- data.frame(x1 = runif(n, 0.5, 2), x2 = rnorm(n))
dd$y <- 1 + sin(3 * dd$x1) - dd$x2 + rnorm(n, sd = 0.3 * dd$x1)
dd$x3 <- 1e4 + dd$x2
far <- mgcv::gam(y ~ x1 + x3, data = dd) # residual sum of squares 169.1
lm(y ~ x1 + x3, data = dd) # 41.9, on the same design
The tests that fit the design again took such a fit when it has no smooth
terms: RESET compared mgcv's fit with the least-squares fit of the augmented
design, and Goldfeld-Quandt and the studentized statistic gave the values of
the lm. They refuse it now, and so do the bootstraps, the Harrison-McCabe
and Szroeter tests, the variance form test and fitWLS(). Of these,
performWhiteTestBootstrap() (63.38), the Harrison-McCabe and Szroeter
tests, the variance form test and fitWLS() took it;
bootstrapTestStatistic(), which the robust tests' bootstraps call, refused
it as not an lm, and performWildBootstrapTest() stopped with an error of
its own (decision 14):
| Call | Before | Now |
|---|---|---|
performRESETTest(far, dd) |
476.2 (p = 2.3e-56), where lm(y ~ x1 + x3, data = dd) gives 0.31 |
Error: "performRESETTest() cannot test this mgcv::gam() fit: its residuals are not those of the least-squares fit of its design, the basis of its smooths, which the test fits again, although none of its terms is penalised. mgcv's fit is not that fit on an ill-conditioned design, such as one with a regressor far from zero: centre that regressor, or fit the model with lm(). Otherwise the fit was changed after it was fitted: refit it." |
performGQTest(far, dd, order_by = "x1"), performBPTestRobust(far, dd) |
2.65 and 12.84, the values of the lm, from the least-squares refit of the design |
Error, as above |
The same holds with x3 = 1e6 + x2 and with a fixed smooth,
s(x1, fx = TRUE) + x3, which these tests refused before as a gam with
smooth terms. With x3 = 1e3 + x2 mgcv's fit is the least-squares fit, and
is taken. The tests that read only the residuals of the fit test mgcv's
residuals, as they test those of a penalised fit (Koenker's 2.55, where the
lm gives 12.84).
Unchanged: every call on any other fit. A corpus of 44,156 calls
(scripts/model-context-identity.R), the rlm fit in five forms of data
and in two grouped runs, the penalised gam in seven cells and a grouped
run, a gam that is the least-squares fit of its design in eighteen, a fixed
smooth among them, and a grouped run of a multivariate lm, through every
recipe, route and runner among them, returns what it returned in the previous
step, identical(), with the same warnings, and leaves the generator as it
left it, but for 1,071 calls on those cells: 492 on the rlm fit, each now
the refusal above with its class (287 that returned a value, 205 that refused
the fit with another message); 319 on the penalised gam, 253 of them now
the refusal above (79 that returned a value, 174 that refused it with another
message) and 66 of the streaming tests and the runner that streams them, each
now the value of the exact test of the same call; 2 grouped runs of the
multivariate lm, now the refusal its first group recorded; and 258
on a gam that is the least-squares fit of its design, each now what the
same call gives on the lm() of its basis, to 1e-8, with the same draws
of the generator (200 refusals that became values, 48 values of the streaming
tests and the runner that streams them, and 10 calls of
bootstrapTestStatistic() on a shuffled tibble that refuse its rows, as for
any fit, where it refused every gam).
Packaging¶
- On CRAN the tests no longer require bit-identical numbers under other BLAS libraries, on which the check of 0.16.0 with Intel MKL failed 179 of them. There they compare two computations of the same numbers to a tolerance of 1e-10, as waldo applies it: the mean absolute difference of the elements that differ, relative to their mean expected size, or absolute where that size is at most 1e-10. Off CRAN they still compare them bit for bit. No function changes.
- A CI workflow,
.github/workflows/cran-blas.yml, checks the package as the additional checks of CRAN do: it builds the tarball and runsR CMD check --as-cranon it with R-devel andNOT_CRANunset, so that the tests compare as on CRAN, in R-hub containers that reproduce those checks. Its jobs areMKL(Intel MKL 2026.2, the version of the MKL check of CRAN),OpenBLAS,BLISandnoLD(R built without long doubles), andNoisy BLAS 1to3, a BLAS that moves its results by up to one unit in the last place (tools/noisy-blas.c) on three seeds. The vignettes are neither built nor checked there. It runs on the pull requests tomainand the pushes to it that change the package (R/,tests/,inst/,data/,man/,DESCRIPTION,NAMESPACE,.Rbuildignore, the noisy BLAS or the workflow), weekly, and by hand.
0.16.0¶
0.16.0 moves the tests onto a model context and corrects the statistics found
wrong on the way to 1.0.0. In the first seven of the eleven steps of the model
context the tests of a model, but the resampling tests, read the response, the
design, the residuals, the weights, the offset and the rank from one internal,
validated object per call, built from the stored fit, and match the rows of
data to the observations of the fit before they read anything from them,
named data in any order and numbered data in the order of the fit, where they
rebuilt the design from data or read data by position (The model
context; the other four steps are planned for 0.17.0). White's test takes its
degrees of freedom from the rank of its auxiliary regression with the
intercept and refuses one that fits the squared residuals exactly; the caches
follow a contract, set out on ?cachedTest; print() and summary() of the
results of runHeteroTests() lay them out as summary.lm() does; the
streaming tests run only when asked for; White's, Koenker's and the
Breusch-Pagan tests warn on fixed effects for small groups; and the
Fligner-Killeen, Box's M, Szroeter, Pesaran and quantile regression tests,
runAdvancedDiagnostics(), the Szroeter entry of test_factory and the
advice for a small sample are corrected. No export or argument changes; the
default of chunk_threshold_mb becomes Inf, and eight S3 methods are added
for the suites. The documentation site moves to Material for MkDocs, and CI
runs the suite once per job and the full matrix of the model context in one
job.
The changes in bold change what calls return. Through the model context a term
computed from all the rows of the data, such as I(x > median(x)) or
cut(x, quantile(x)), is tested as the fit computed it, in every chunk of the
streaming tests too; on a model with an offset the studentized Breusch-Pagan
statistic, Goldfeld-Quandt and the quantile regression test give the value of
that model, and RESET the nested F test of it; Goldfeld-Quandt and the
quantile regression test fit the design of the fit; Harrison-McCabe takes the
rank of the fit, at its tolerance; the tests that read a column of data
beyond the model match a gam to its data by row name; the panel tests and
the scatter diagnostic match data by name, and the random-effects test and
the scatter diagnostic give the value of a fit that dropped rows; the tests
computed from the model alone match data as every other test does, and a
second formula is evaluated on the rows of the fit, its variables never read
from the workspace; without data, runHeteroTests() runs White's and
Koenker's tests on a fit whose weights or offset are a column of the data; the
registered "szroeter" orders a gam with an offset argument by its fitted
values with the offset, and runHeteroTests(tests = "spatial_hetero") runs on
data it stopped on; and the tests refuse a fit whose coefficients, residuals
or weights were changed after it was fitted, a fit with model = FALSE whose
data changed after the fit, or are gone and are not given in a data that
holds its rows, a multivariate lm in RESET and the Davidian-Carroll test, a
fit with smooth terms in RESET, Goldfeld-Quandt, the quantile regression test
and the studentized statistic, a fit whose model frame cannot be rebuilt under
its levels in bootstrapTestStatistic() and the robust tests with
bootstrap = TRUE, and a repeated pair of id and time in Pesaran's test,
where they returned a value. Of the models the tests accept, a rank-deficient
fit is tested on the estimable part of its design; a Gaussian glm with a
zero prior weight, and a near-perfect one, are refused as the same lm;
performWhiteTestStreaming(), performHarrisonMcCabeTest() and
performVarianceFormTest() accept a Gaussian glm with the identity link;
fitWLS() refits one as the lm it is and refuses any other glm;
performScatterDiagnostic() refuses a glm that is not a least-squares fit
and data that lack a variable of the model; the registry name "ncv" checks
the data it is given; runHeteroTests() on a tidymodels workflow without
data runs the tests on the model frame of the engine's fit; and the calls
that go through the tests change with them. White's test counts one degree of
freedom fewer where its auxiliary columns span the constant, keeps the columns
lm() keeps on a nearly collinear auxiliary design, no longer depends on
where its regressors are located, and refuses an auxiliary regression that
fits the squared residuals exactly, where it returned the number of
observations. A cached result is the result of its own call; the checks of the
model are those of the model; a test that draws random numbers is never
cached, and the check of normality draws its sample on every call;
runHeteroTests() stores no call that fell back on a diagnostic the caches
may not hold, and returns with the cache the suite it returns without it; and
the names of the estimates of the quantile regression test do not depend on
the options. runHeteroTests() runs the exact tests on data of any size by
default. White's, Koenker's and the Breusch-Pagan tests warn when a term of
the model that holds a factor has more levels than observations per level,
with their values unchanged. And of the single tests, performPesaranTest()
computes the statistic of an unbalanced panel from the periods each pair of
units shares, as plm::pcdtest() does; performFlignerKilleenTest() ranks
the deviations of repeated observations as tied, as in exact arithmetic;
performBoxMTest() applies Box's correction factor with three groups or more,
drops a level of group with no observation, and refuses a group with no more
observations than variables, a group whose covariance matrix is singular and
data with fewer than two groups; performSzroeterTest() takes the null mean
and variance of its statistic from the design of the fit, and refuses a fit
with one residual degree of freedom; the Breusch-Pagan entry of
runAdvancedDiagnostics() is Koenker's studentized statistic with
auto_enhance = FALSE, and tests = "bp" runs it; and
performQuantileRegressionTest() gives a value on a model with a single
regressor.
Pesaran's test on an unbalanced panel¶
performPesaranTest(), and so runPanelTests(), now compute the statistic
of an unbalanced panel, whose units are not all observed at the same periods,
from the periods each pair of units shares, as plm::pcdtest() computes it
except where a correlation is undefined (below), where 0.16.0's development
version refused such a panel and 0.15.0 stopped on it or returned a wrong
value. 0.15.0 filled a matrix of periods by units,
and a unit observed at fewer periods than the panel holds stopped it with
"number of items to replace is not a multiple of replacement length", or,
when its number of periods divides that of the panel, had its residuals
recycled over the periods it lacks. The development version refused such a
panel with "needs a balanced panel" (The model context, below). On the 120
observations of contract_data() (tests/testthat/helper-test-recipes.R)
without the first five rows, unit 1 keeps 5 of its 10 periods:
performPesaranTest(lm(y ~ x1 + x2, data = d), d, "id", "time") |
Result |
|---|---|
| 0.15.0 | 0.234 (p = 0.815), the residuals of unit 1 recycled over the periods it lacks |
| 0.16.0's development version, before this change | Error: performPesaranTest() needs a balanced panel, every unit observed at the same periods, and the panel of the fit is unbalanced: 1 of its 12 units is observed at fewer of its 10 periods, id = 1 at 5 of them. ... |
| Now | -0.373 (p = 0.709), the value of plm::pcdtest(test = "cd", model = "pooling") |
The statistic is that of Pesaran (2004, Section 9, equation 31),
CD = sqrt(1 / P) sum_{i<j} sqrt(T_ij) rho_ij, where T_ij is the number of
periods at which units i and j are both observed, rho_ij the
correlation of their residuals over those periods, each centred at its mean
over them, and P the number of pairs that share two periods or more,
N (N - 1) / 2 when every pair does. A pair that shares fewer than two
periods has no correlation, and is left out of the sum and of P with a
warning, as plm::pcdtest() leaves it out; a panel in which no pair shares
two periods is refused, of class heteroTests_model_context_error. A
correlation that is not defined, the residuals of a unit constant over the
periods of a pair, adds nothing to the sum and the pair stays in P, with a
warning, as on a balanced panel, where plm::pcdtest() returns NA: on
four units at periods 1 to 4 without the first and the last observation,
with the response of unit 2 constant, lm(y ~ 1) gives 0.758, plm NA
(the response drawn by set.seed(4); sample(1:9, 16, TRUE)). Compared
with plm::pcdtest() on 252 panels of 5 to 30 units and 4 to 12 periods,
228 of them unbalanced, the statistic and the p-value agree to 1e-10 in
each of 5,040 comparisons, by at most 3.6e-15 and 3.9e-16: units that enter
late, leave early or skip periods, 1, 3, 5, 10 or 20 rows missing at random,
a unit observed at one period, and balanced panels; an lm, a weighted
lm and a Gaussian glm, weighted or not, against plm::pcdtest() on their
Pearson residuals, and the lm against its pooled fit too; the full data of a
fit with na.omit; and each through runPanelTests(). The full data of a
fit that dropped some rows of some units, which 0.15.0 refused for their
number of rows, give the value of the fit to the complete rows: with x2
missing in rows 3, 7 and 11 of contract_data(), 0.189 (p = 0.850),
plm::pcdtest()'s on the rows of the fit.
On a balanced panel the statistic is computed as before, and every value is
identical() to before. A fit with na.action = na.exclude that dropped
rows is read as before, by position, until the step of the model context
that moves na.exclude: the panel of the rows of data it reads must be
balanced, and an unbalanced one stays refused, with a message that now
points to na.omit, on which the statistic of an unbalanced panel is
computed. A fit with na.exclude that dropped no row is tested as any
other, its panel balanced or not.
The Fligner-Killeen test and its ties¶
On data with repeated observations, observations with the same values of
the model's variables such as integer responses or responses recorded with
few digits, performFlignerKilleenTest() now ranks their deviations as
tied, as in exact arithmetic, which can change its value a lot: on
lm(extra ~ group, data = sleep) the statistic was 0.154 and is 0.213 (the
p-value 0.695 and 0.645), and on lm(breaks ~ tension, data = warpbreaks)
it was 3.44 and is 3.90 (0.179 and 0.142), the values of exact ties. On
continuous data it changes much less: in simulated fits by up to about 1e-2
of the statistic at 20 observations, where the statistic is often near zero
(by at most about 0.005 in absolute terms, and 2.2e-4 in the p-value), and
by about 1e-5 of it at 100 or more, up to 2.4e-5 with a group whose
residuals are far from zero (below).
The test ranks the absolute deviations of the residuals from their group
medians, and averages the ranks of equal values, but rounding broke ties in
two places. It read the residuals an lm stores, which its QR decomposition
rounds differently from one observation to the next: two observations with
the same response and design row, equal in exact arithmetic, had residuals
that differed in their last bits, and the test ranked them apart. In 20 fits
of rpois(n, 3) on four groups the gaps reached 17 times
eps * max(abs(r)) at 60 observations and 6,900 times at 6,000, with
eps = .Machine$double.eps and r the residuals. The test now computes the
residuals observation by observation, sqrt(w) * (y - X b - o) with X b
summed in the same order for every row, as the Gaussian glm computes them;
for an mgcv::gam() fit X is its linear predictor matrix. Observations
with the same values of the model's variables, weight and offset are then
given one residual, that of the first of them, where their residuals agree
to within rounding (64 times eps of the size of the numbers they are
computed from), since a basis such as poly(), built with a QR
decomposition of its own, can give them design rows that differ in their
last bits: on 70 observations that repeat 30, lm(y ~ poly(x, 3)) gave
0.8001 before 0.16.0 and gives 0.91385, the value of
lm(y ~ x + I(x^2) + I(x^3)). A term of the row position, such as
seq_along(x) or cumsum(x), gives such observations residuals that differ
by much more, and keeps them apart. This needs the model's variables: it is
done when data is given, or when the model frame holds them, as it holds a
variable the model uses under its own name (x, not poly(x, 3)); on those
70 observations lm(y ~ poly(x, 3) + g) gives 1.21547 with data and
1.22684 without. And in a group of even size the two
residuals in the middle lie at the same distance from the median, half their
difference, a tie that the centring breaks by up to eps * s, with s the
largest absolute residual of the group: deviations closer than
8 * eps * s, with the smaller s of the two groups for deviations of
different groups, are given the mean of their ranks. Where neither changes a
rank, the residuals of the fit are passed to stats::fligner.test() as
before, and the value is identical() to before.
An lm and the Gaussian glm of the same model now give the same value,
identical(). On lm(y ~ x1 + x2) with 120 observations in four groups of
30, the lm gave 9.3670076662104975, with a p-value of 0.0247890868, and
the glm 9.3670316853686657 and 0.0247888156; both now give the second, the
value the gam gave already.
- On 480 fits of
lm(y ~ g)with integer responses (rpois(n, 3),rpois(n, 300)and1e4 + rpois(n, 3), four groups, balanced and unbalanced, 60 to 60,000 observations), the statistic equals that of exact ties,stats::fligner.test()of the responses, in all 480, where it did in none before. Onsleep,PlantGrowth,warpbreaks,InsectSpraysandiris[-c(50, 100, 150), ], each fitted on its groups, it equalsstats::fligner.test()of the responses scaled to integers, and oniriswith every row twice thelmand theglmgive the same value. A tie between different observations that exact decimal arithmetic would give, as when two pairs of observations of a regressor recorded with few digits differ by the same amounts, is restored only when rounding leaves it within the tolerance: oniriswith every row twice, regressed onPetal.Length, one of 106 sets of such ties stays broken, and the statistic is 8.14863 where exact arithmetic gives 8.14287. Two forms of one model can then differ by several percent: on the 70 observations above,y ~ poly(x, 3) + ggives 1.21547, andy ~ x + I(x^2) + I(x^3) + gandy ~ poly(x, 3, raw = TRUE) + ggive 1.30280, 7% more, sincepoly()breaks in its last bits the ties between repeated values ofxin different groups (recorded under 0.16.0 in the roadmap). - On 3,000 continuous fits of 20 to 10,000 observations (normal,
t3, Cauchy, exponential and scale-mixture errors, with and without weights, with an intercept of 0 or 1e6, or with a group whose residuals are far from zero), no two deviations that differ in exact arithmetic are tied, and thelmand theglmgive the same value, unless the residuals of a group are 1e8 times their spread. Then the tolerance is a few units in the last place of those residuals, and it tied such deviations in 41 of the 200 fits of 1,000 and 10,000 observations, and thelmand theglmdiffered in 4. - Over a corpus of 55 calls, the 20 calls in which no rank changes are
identical()to before. Of the 17 with repeated observations, 14 change, by up to 38% of the statistic (sleep); of the 38 with continuous responses, 21 change, by 7.9e-7 to 3.2e-5 of the statistic and at most 6.1e-6 in the p-value. In the identity harness of the model context, 128 of the 351 calls of the test change, by 1.1e-6 to 4.4e-6 of the statistic. The test is not run byrunHeteroTests(), the registry ortest_factory, so nothing else changes.
The wild bootstrap in the advice for a small sample¶
suggestDiagnosticsForProfile() recommended the wild bootstrap for a sample
of fewer than 100 observations with the rationale "Bootstrap inference
performs well in small samples.", and step 1 of
buildDiagnosticDecisionTree() said "Emphasise wild bootstrap and rank
permutation tests.". With centred exponential errors the wild bootstrap
rejects a true null hypothesis 8.2%, 7.3% and 6.1% of the time at the 5%
level at 30, 50 and 100 observations, with the Rademacher multiplier
(inst/validation/wild-multiplier-size.R). Both now say what the warning of
warnDiagnosticAssumptions() says since 0.15.0. The rationale reads:
In the package's validation study, at 30 to 100 observations, the wild
bootstrap rejected a true null hypothesis 4.3% to 5.1% of the time at the 5%
level with normal and t5 errors, but 6% to 8% of the time with skewed errors.
Check the residuals for skewness and read p-values near the level with
caution.
and step 1 reads "Add the wild bootstrap and rank permutation tests;" followed by the caution of the warning, "with skewed errors the wild bootstrap does not hold its level here: ...". Only the text changes: the same tests are recommended, and the same steps given.
Fixed effects for small groups¶
White's, Koenker's and the Breusch-Pagan tests warn when a term of the
model that holds a factor has more levels than observations per level. On
such a fit, fixed
effects for small groups as in lm(y ~ x + subject) with each subject
observed a few times, the three tests reject a true null hypothesis far too
often, and the more so the more levels and the fewer observations each. In
2,000 homoscedastic draws of lm(y ~ x + subject) with normal x, subject
effects and errors (inst/validation/fixed-effects-size.R), White's,
Koenker's and the Breusch-Pagan tests rejected at the 5% level in 50.7%,
57.3% and 39.6% of them with 30 subjects of three observations, in 22.2%,
23.9% and 28.7% with 30 of four, in 23.8%, 23.8% and 25.2% with 100 of
eight, and in 97.8%, 97.6% and 82.5% with 200 of four. The excess comes from
the subject dummies in the auxiliary regression: under normal errors the
squared residuals of a subject of T observations vary between subjects
T / (T - 1) times as much as the regression assumes, Koenker's statistic has
mean its degrees of freedom plus (G - 1) / (T - 1) for G subjects, and the
squared residuals regressed on x alone were rejected in 4.6% to 5.5% of the
draws by Koenker's statistic and in 4.3% to 5.6% by the Breusch-Pagan
statistic (inst/validation/fixed-effects-dummies.R). At a fixed number of
observations per subject the size goes to 1 as the subjects grow.
The rule: a term of the model that holds a factor or a character variable,
with L levels among the n observations of the fit, warns when L > n / L,
that is L^2 > n. L counts the distinct combinations of the factors of the
term: a factor such as subject; an interaction of factors by its cells,
such as a:b of a crossed a * b or school:class of a nested
school / class; or a slope for each level, x:subject. A variable left
out of the formula, as in y ~ . - id, does not count, and the terms are
matched to the model frame by position, so that a name in backticks works.
On a balanced design of G subjects of T it is G > T. In the study, at most as many subjects as observations per subject
kept the three tests within 2.5% to 7.5% at the 5% level, and more put at
least one of them outside, in 68 of 69 designs; the exception is the
Breusch-Pagan test at 8.0% on 20 subjects of 20, inside the rule. On 10
subjects of 10 they reject in 6.4%, 6.2% and 7.3%, and on 5 subjects of 4,
over the rule, White's test in 1.0%, too rarely. Unbalanced groups, two
regressors and a regressor correlated with the subject effects followed the
same pattern. A slope for each subject, lm(y ~ x + x:subject), is counted
too, but there the error goes the other way: on 30 subjects of 3 and of 4 and
on 100 of 8 the three tests rejected in 0.1% to 2.8% of the draws
(inst/validation/fixed-effects-variants-size.R), as White's test with the
products of a few regressors does on 5 subjects of 4. The warning names the
term, gives its numbers, and says in which direction the p-value erred in
the simulations:
The term `subject` has 30 levels for 120 observations, 4 per level: with
fixed effects for small groups, more levels than observations per level, the
p-value of Koenker's test is unreliable; in simulations it was usually far
too small, and too large when the factor enters only as a slope per level
(see 'Fixed effects for small groups' in ?performKoenkerTest).
Values are unchanged: the statistic, the degrees of freedom and the
p-value of every call are those before this change, identical(), with the
warning added on a fit over the rule, and nothing changes on a fit below it.
The
warning has class heteroTests_fixed_effects_warning;
suppressWarnings(expr, classes = "heteroTests_fixed_effects_warning"), or a
handler for that class in withCallingHandlers() that calls
invokeRestart("muffleWarning"), muffles it and no other warning. The
condition carries the terms, their levels and the number of observations
($factors, $levels, $observations).
The routes that compute the same statistics, and over-reject as the three
tests do, warn too (inst/validation/fixed-effects-variants-size.R, 2,000
replications on 30 subjects of 4 and 100 of 8, 1,000 for the resampling
routes on 100 of 8):
performWhiteTestRobust(), performBPTestRobust() with studentize = TRUE
or FALSE, performBPTestStreaming(), performKoenkerTestStreaming(), the
deprecated performStudentizedBPTest() and performBreuschPaganTest(), and
the registered "white", "koenker", "breusch_pagan" and "student_bp".
On those designs they reject in 21.6% to 27.6% of the samples, each
as the test it wraps, and give one warning per call.
performWhiteTestBootstrap(), performWildBootstrapTest() and
bootstrapTestStatistic(), whose p-values are bootstraps', held their level
there, at 3.7% to 4.6% with B = 199, and give none.
performWhiteTestStreaming() refuses a factor that is
a main effect, as before, and gives the warning where it computes White's
statistic, on a slope for each level. autoTransform(), which runs the
Breusch-Pagan test on each candidate transformation, gives it once.
The warning is given on every call: a result that cachedTest() or
runHeteroTests() answers from its cache gives it again, where it gives no
other warning of the call that computed it, so a call answered from a cache
warns as a call that computes does. runHeteroTests() and runDiagnostics()
give it once for each test that gives it: a test that is refused and replaced
by a fallback warns once, for the fallback, because the tests warn after they
have computed. They, and cachedTest(), hold the warnings while the tests run
and give them once the result is complete and stored, so that under
options(warn = 2) the call stops at the end with the right values stored,
where a warning turned into an error inside the runner would have made it
take a fallback, and store that, under the name of the test. A robust form with bootstrap = TRUE
warns once, for the asymptotic p-value it reports: bootstrapTestStatistic()
computes its observed statistic and replicates without the warning. On the
paired design of White's degrees of freedom below, where Koenker's and the
Breusch-Pagan tests return 60 and 63.25 on 30 degrees of freedom, they now
warn; refusing that fit is still to do. The help pages of the three tests
have a section Fixed effects for small groups with the mechanism, the rule,
a table of the study and the reference to Juhl and Sosa-Escudero (2014), who
derive a test for this setting. Testing against the regressors without the
dummies, and that test, are recorded in the roadmap for after 1.0.0. No
argument changes.
Box's M test¶
performBoxMTest() applies Box's correction factor with three groups or
more, and its statistic grows. Box (1949) refers M (1 - c1) to a
chi-square distribution, with
c1 = (sum 1 / nu_i - 1 / nu) (2 p^2 + 3 p - 1) / (6 (p + 1) (g - 1)) for
g groups of nu_i + 1 observations of p variables. The package wrote
the factor with g - 1 in the numerator as well, where it cancelled the
denominator's, so it used (g - 1) c1: Box's with two groups, too large
with more, which made the statistic too small and the test conservative.
performBoxMTest(iris[, 1:4], iris$Species) gave 135.2229 and gives
140.9430, on 20 degrees of freedom either way, the value of
biotools::boxM(), heplots::boxM() and rstatix::box_m();
performBoxMTest(mtcars[, c("mpg", "hp", "wt")], mtcars$cyl) gave 20.3998
(p = 0.0599) and gives 25.5351 (p = 0.0125). With three or more groups the
statistic grows by the factor (1 - c1) / (1 - (g - 1) c1), 1.01 to 2.11
on 23 data sets of 3 to 5 groups, and the p-value falls; with two
groups every value is identical() to 0.15.0's, on 28 data sets.
runMultivariateTests(), whose "box_m" is this test, returns the new
values too, and stops on the refusals below as the test does.
inst/validation/box-m-approximations-size.R measured the effect: under
Gaussian data the test of 0.15.0 rejected at 5% less often than the release
gate allows in 29 of the 38 cells of three or five equal groups, down to 0
with small groups and four variables or more, and Box's chi-square is inside
the gate in 60 of 67 cells. The full sweep's size of 0.012 was not the
approximation's: in its design, with x uniform and so lighter-tailed than
normal data, Box's chi-square and his F approximation both reject 0.017 of
the time at 5000 replications. The help page now gives the sizes: liberal
when the groups are small for the number of variables (0.062 to 0.106 in four
cells of groups of 10 with 4 or 6 variables), not robust to heavy tails
(t5 coordinates, 0.26 to 0.48) and conservative with light tails. The
statistic keeps its name, M, and parameter its df.
performBoxMTest() drops a level of group with no observation, which
made the result NA. A factor keeps its levels when rows are taken out, so
sub <- subset(iris, Species != "virginica") has a virginica level with no
row, whose covariance matrix was all NA:
performBoxMTest(sub[, 1:4], sub$Species) gave NA on 20 degrees of
freedom, and gives the test of the two species observed, 66.8105 on 10
(p = 1.8e-10), the value of droplevels(sub) and of biotools::boxM(). A
character group, which has no empty level, and a factor without one give
what they gave, identical().
performBoxMTest() refuses a group with no more observations than
variables, and data with fewer than two groups. A group of n_i <= p
observations of p variables has a singular covariance matrix, whose
log-determinant is -Inf, and the statistic is not defined, whatever the
data; rounding made it a finite number. On
d <- rbind(subset(iris, Species != "virginica"), iris[101:103, ]),
performBoxMTest(d[, 1:4], d$Species) gave 133.74 on 20 degrees of freedom
(p = 7.7e-19), and stops with
Box's M test needs more observations than variables in each group: with 4
variables, group 'virginica' (3 observations) has no more. The covariance
matrix of such a group is singular, so its log-determinant is -Inf and the
statistic is not defined, whatever the data. Each group needs at least 5
observations; use more data, fewer variables or fewer groups.
A group of one observation, which gave NA, is refused with the same
message, and a single group with observations, which gave NaN on 0 degrees
of freedom, with "Box's M test compares the covariance matrices of two groups
or more". Data where every group has more observations than variables give
what they gave, identical(); n_i = p + 1 is computed.
performBoxMTest() refuses a group whose covariance matrix is singular
with more observations than variables. A variable that is constant in a
group, or a linear combination of the others in it, makes the group's
covariance matrix singular, and the statistic is not defined: with
d <- iris[, 1:4]; d[1:50, 4] <- 0.2, performBoxMTest(d, iris$Species)
returned Inf on 20 degrees of freedom (p = 0), and iris[, 1:4] with a
fifth column, the sum of the first two, returned NaN with "NaNs produced"
warnings.
Both stop, naming the group and the cause:
Box's M test needs a nonsingular covariance matrix in each group, and that of
group 'setosa' is singular: the variable 'Petal.Width' is constant in it. Its
log-determinant is -Inf and the statistic is not defined, whatever the data.
Leave out the variable, or the combination of variables, that does not vary
in the group.
or "a variable is a linear combination of the others in it (its 5 variables
have rank 4)". A group is checked by the rank of its centred data at the
tolerance of qr(), and a log-determinant that under- or overflows on the
scale of the data, which gave NaN, is refused with its own message. On 198
data sets with nonsingular groups the values are identical() to those
before; a missing value in data or in group still gives NA (the
roadmap records it).
Szroeter's test¶
performSzroeterTest() takes the null mean and variance of its statistic
from the design of the fit, and every Q and p-value changes. The test
standardised h, the rank-weighted average of the squared residuals, with
(n + 1) / 2 and (n^2 - 1) / (6 n) whatever the design. Under normal
errors h is a ratio of quadratic forms in the residuals, and its exact
mean and variance given the design are now used: with M the residual-maker
of the design in the order of the test (sqrt(w) x on a weighted fit),
A = diag(1, ..., n) and nu = n - k, E h = tr(M A) / nu and
Var h = 2 (nu tr(M A M A) - tr(M A)^2) / (nu^2 (nu + 2)), computed from the
leverages and Q1' A Q1 at a cost of O(n k^2). The reference is still the
standard normal distribution, and h, the estimate, is unchanged,
identical(). On a weighted fit whose leverage falls along the ordering the
test rejected too often: in inst/validation/szroeter-design-null-size.R,
on the samples of harrison-mccabe-size-power.R, the correctly weighted fit
of 40 observations rejected a true null hypothesis at 5% 0.0666 of the time
and rejects 0.0494, and 0.0560 and 0.0550 at 100 and 400 observations become
0.0470 and 0.0498; unweighted, 0.0444, 0.0484 and 0.0490 become 0.0482,
0.0498 and 0.0494. Unweighted p-values move by a median 0.006 at 40
observations, 0.002 at 100 and 0.0006 at 400 on those samples; weighted, by
a median 0.051 at 40. The help example
performSzroeterTest(lm(mpg ~ wt + qsec, mtcars), mtcars, order_by = "wt")
gave Q = -0.6494 (p = 0.7420) and gives Q = -0.6626 (p = 0.7462). The other
alternatives, the registered "szroeter" that runHeteroTests() runs, its
test_factory entry and every fit the test accepts change in the same way;
method, parameter, alternative and data.name do not.
performSzroeterTest() refuses a fit with one residual degree of freedom,
for which it returned a value. With n - k = 1 the residuals lie on one
line, so h takes the same value whatever the errors and its null variance
is 0. On 15 observations of y ~ g, with g a factor of 14 levels, one of
them observed twice, h is 8.5 for any response, and the test returned
Q = 0.3169 (p = 0.3756) ordered by another variable; it stops with "Szroeter
test needs at least two residual degrees of freedom: the fit has 15
observations and 14 estimable coefficients, so h takes the same value
whatever the errors and there is nothing to test."
The Breusch-Pagan entry of runAdvancedDiagnostics()¶
With auto_enhance = FALSE, the Breusch-Pagan entry of
runAdvancedDiagnostics() is Koenker's studentized statistic, where it was
the classical test. auto_enhance is the switch of the automatic bootstrap
for samples of fewer than 50 observations, but it was also passed to
performBPTestRobust() as studentize, so that turning the bootstrap off
turned the studentization off with it, and the entry, named breusch_pagan
either way, then assumed normal errors, which the default battery of the
package stopped doing in 0.15.0. auto_enhance now turns the bootstrap on
and off and nothing else, and the entry is the studentized statistic, the
default of performBPTestRobust(), whatever its value. On
lm(mpg ~ wt + qsec, data = mtcars),
runAdvancedDiagnostics(m, mtcars, auto_enhance = FALSE)$results$breusch_pagan
was the classical test, 3.135 on 2 degrees of freedom (p = 0.2086), and is
Koenker's, 3.086 on 2 (p = 0.2138), the statistic auto_enhance = TRUE
reports beside its bootstrap p-value; its method says "Studentized", its
robust_details$studentized is TRUE, and its effect size and power are
those of the studentized statistic. From 50 observations, where neither
setting bootstraps, the two settings now give the same entry. The deprecated
rrunAdvancedDiagnostics() changes with it. The White entry, the metadata,
and every call with auto_enhance = TRUE give what they gave,
identical(). The classical test is still
performBPTestRobust(m, data, studentize = FALSE).
runAdvancedDiagnostics(tests = "bp") runs the Breusch-Pagan entry, which
it refused. Its help page offers c("white", "bp") as a subset, and its
table of names maps "bp" to "breusch_pagan", but the names asked for were
compared with the values of that table rather than with its names, so "bp"
stopped with "Unsupported test requested: bp". It now gives what
"breusch_pagan" gives, under the name breusch_pagan: on
lm(mpg ~ wt + qsec, data = mtcars), tests = c("white", "bp") stopped with
that error and returns White's test and the Breusch-Pagan entry, as
tests = c("white", "breusch_pagan") does. Every other name is refused with
the message it had.
The quantile regression test on a model with one regressor¶
performQuantileRegressionTest() gives a value on a model with a single
regressor, where it stopped with "subscript out of bounds". It fits the
quantile regressions with the design of the fit as one matrix column, and the
names of the coefficients come from that column: a matrix of one column names
its coefficient after the matrix alone, so the slope had an empty name, and
reading it by name for estimate stopped. The statistic it would have
reported is anova() of quantreg::rq(y ~ x), which the test now gives and
test-quantile-joint-reference.R checks. Called directly, through
test_factory, the registry or cachedTest(), the test stopped; through
runHeteroTests(), and so runDiagnostics() and the other functions that
call it, the failure was masked: the runner ran the fallback of
"quantile_regression", the rank permutation test, and returned it under
the name asked for. On lm(stations ~ mag, data = quakes):
| Call | Before | Now |
|---|---|---|
performQuantileRegressionTest(m, quakes) |
Error: "subscript out of bounds" | F = 57.57 on 1 and 1999 degrees of freedom (p = 5.0e-14), slopes 35.0 and 47.3 at the quartiles |
runHeteroTests(m, quakes, tests = "quantile_regression"), after set.seed(1) |
the rank permutation test, rho = 0.261 (p = 0.001), under the name quantile_regression, tagged as a fallback |
the quantile regression test, as in the row above |
test_factory$run_test("quantile_regression", m, quakes) |
Error: "subscript out of bounds" | the quantile regression test, as in the first row |
So does any design with one column beside the intercept once aliased
columns are dropped, which stopped the same way: a factor of two levels,
lm(len ~ supp, data = ToothGrowth), gives F = 0.1529 on 1 and 119 degrees
of freedom, and poly(x, 1), x1:x2 alone, an offset beside one regressor
or x1 + I(2 * x1) give their value too. A design with two columns or more
beside the intercept gives the value it gave, identical(). A
model without an intercept is still refused by quantreg, which needs one,
with its own message. An exploratory size check, not to the grade of
inst/validation/, found the test on one regressor below its level, as the
two-regressor design of the sweep is under t5 errors: 2,000 replications
per cell, rejection rates of 0.029 and 0.040 at 60 and 150 Gaussian
observations.
Szroeter's test in test_factory¶
test_factory$run_test("szroeter", model, data) stops with a message that
asks for order_by, the column that orders the observations, and shows how
to pass it, test_factory$run_test("szroeter", model, data, order_by = "x");
it stopped with "argument "order_by" is missing, with no default". It still
refuses the call, and orders by nothing of its own: the registered
"szroeter" of runHeteroTests() orders by the fitted values, but the
factory's entry is performSzroeterTest(), which takes the ordering from
order_by. With order_by it returns what it returned, identical(). No
value changes.
The streaming tests¶
The streaming tests run only when asked for: runHeteroTests() runs the
exact tests on data of any size by default. Up to 0.15.0 runHeteroTests()
and runDiagnostics() replaced the White, Breusch-Pagan and Koenker tests
with performWhiteTestStreaming(), performBPTestStreaming() and
performKoenkerTestStreaming() when the data exceeded chunk_threshold_mb,
100 MB by default, and performWhiteTest(), performBPTest(),
performKoenkerTest() and the White bootstrap switched to them on their own
above 100 MB. Streaming saves memory mainly in White's test, whose auxiliary
design is several times wider than the model matrix; the auxiliary design of
the Breusch-Pagan and Koenker tests is the model matrix itself. And the
streaming tests do not always give what the exact ones give: the White
streaming test stops on any model with a factor or a poly() term, whose
squares are collinear with the other terms of its auxiliary regression, so
that above 100 MB such a model got a fallback under the name white where
smaller data got White's test. The default of chunk_threshold_mb is now
Inf, which never streams. To opt in, give a finite threshold:
chunk_threshold_mb = 100 streams above 100 MB, as the default did, and
chunk_threshold_mb = 0 on data of any size; chunk_size keeps its
meaning, and attr(, "extras")$streaming is TRUE on a test that
streamed. A fallback runs the exact test, which holds its auxiliary design
in memory even when streaming was asked for, and does not set it. The exact
tests no longer switch at all, and the three streaming functions stay
exported, unchanged and not deprecated, for a caller who wants them on very
large data. No argument changes. On data of at most 100 MB, or a model frame
of at most 100 MB when data is not given, every result is the one before
this change, identical(), and the warning about the size of the data above
50 MB is kept; above 100 MB the results change, to those of the exact tests, which
hold the full auxiliary design in memory. On lm(y ~ x1 + group) in the
design of contract_data() (tests/testthat/helper-test-recipes.R), with
group a factor of four levels, Koenker's test gives 10.52 on 4 degrees of
freedom (p = 0.032) on every route, and White's test:
| Call | 0.15.0 | Now |
|---|---|---|
runHeteroTests(m, d), data above 100 MB |
the classical Breusch-Pagan test, 8.653 on 4 degrees of freedom (p = 0.070), under the name white, tagged as a fallback |
White's test, 17.83 on 8 (p = 0.0225), as on smaller data |
performWhiteTest(m, d), data above 100 MB |
Error: "Auxiliary regression became rank deficient" | 17.83 on 8 (p = 0.0225) |
runHeteroTests(m, d, chunk_threshold_mb = 0), data of at most 100 MB |
White's test without cross products, 13.79 on 5 (p = 0.017), tagged as a fallback | unchanged |
runHeteroTests(m, d, chunk_threshold_mb = 0) or chunk_threshold_mb = 100, data above 100 MB |
the classical Breusch-Pagan test, as in the first row | White's test without cross products, as in the row above |
Above 100 MB the fallback of 0.15.0 was not White's test without cross
products, which runHeteroTests() tries first, but the next one: that test,
called through performWhiteTest(), switched to the streaming test too and
stopped on the same squares. The values of the rows above 100 MB were
computed on the 120 observations of the design, with the size the package
measures for the data set to 150 MB (tests/testthat/test-streaming.R).
The seventh step of the model context (The model context, below) builds the
three streaming tests on it. performWhiteTestStreaming(),
performBPTestStreaming() and performKoenkerTestStreaming() read the
response, the residuals, the weights and the design from the fit, and the
design chunk by chunk, its rows in the order of the fit; the auxiliary design,
several times wider than the design for White's test, is still built chunk by
chunk. data that are given are matched to the observations of the fit as by
the tests built on the context: named data in any order, numbered data in fit
order. This reaches the three functions and runHeteroTests() and
runDiagnostics() with a finite chunk_threshold_mb. Every value is
identical() to before but for the changes in bold below, and, on a poly()
term, in its last bits (below).
A term computed from the rows it is given keeps the values of the fit in
every chunk. Each chunk built its own design from the formula, on its rows
of data, so I(x > median(x)) took the median of each chunk. On
m <- lm(mpg ~ I(wt > median(wt)) + qsec, data = mtcars):
| Call | 0.15.0 | Now, the value of the fit |
|---|---|---|
performBPTestStreaming(m, mtcars, chunk_size = 8) |
2.268 (p = 0.322) | 4.892 (p = 0.087) |
performBPTestStreaming(m, mtcars, chunk_size = 16) |
2.577 (p = 0.276) | 4.892 (p = 0.087) |
performKoenkerTestStreaming(m, mtcars, chunk_size = 8) |
2.993 (p = 0.224) | 6.455 (p = 0.040) |
performKoenkerTestStreaming(m, mtcars, chunk_size = 16) |
3.400 (p = 0.183) | 6.455 (p = 0.040) |
runHeteroTests(m, mtcars, tests = c("breusch_pagan", "koenker"), chunk_threshold_mb = 0, chunk_size = 8) |
2.268 and 2.993 | 4.892 and 6.455 |
The new values are those of performBPTest() and performKoenkerTest(), and
of the same fit with the column computed beforehand, in chunks of any size.
In one chunk the median was that of the rows of the fit: on a fit that dropped
rows the test computed it on the rows it kept, and gave the value of another
design, in chunks of any size. On the 120 observations of contract_data()
(tests/testthat/helper-test-recipes.R) with x2 missing in rows 3, 7 and
11, lm(y ~ I(x1 > median(x1)) + x2, data = d) gave a Breusch-Pagan
statistic of 13.35 in one chunk and 5.75 in chunks of 7, where the fit gives
12.46, and Koenker's 9.28 and 3.99 where it gives 8.66.
cut(x, quantile(x)), which took the breaks of each
chunk, stopped the three with "factor cut(wt, quantile(wt)) has new levels"
in chunks of any size, since the fit drops the row at the minimum of wt; on
lm(mpg ~ cut(wt, quantile(wt)) + qsec, data = mtcars) the Breusch-Pagan
test gives 12.94 on 4 degrees of freedom (p = 0.012) and Koenker's 11.13
(p = 0.025), and runHeteroTests() with chunk_threshold_mb = 0, which put
the NCV test under both names (5.34), gives them. White's streaming test
stops there on the squares of the dummies, collinear with the dummies, as on
any factor, where it stopped on the new levels.
A gam given its data in another order is matched to them by row name, by
the streaming tests too. Each chunk read the rows of data by position,
which the residuals of an mgcv::gam() fit, which carry no names, did not
check. On g <- mgcv::gam(mpg ~ wt + qsec, data = mtcars) with the data
shuffled, mtcars[sample(32), ] after set.seed(1), White's streaming test
returned 16.86 where the data in the order of the fit give 11.82, the
Breusch-Pagan test 5.12 for 3.13 and Koenker's 5.04 for 3.09; each now gives
the value of the call on mtcars.
A fit whose coefficients, residuals or weights were changed after it was
fitted is refused by the streaming tests, as by the tests built on the
context, and so is runHeteroTests() that streams them, without a fallback
("The fitted model does not reproduce its residuals from its design and
coefficients", of class heteroTests_model_context_error). On
lm(mpg ~ wt + qsec, data = mtcars) with 0.5 added to the coefficient of
wt they returned the values of the fit before the edit: White's streaming
test 11.82 (p = 0.037), the Breusch-Pagan test 3.13 and Koenker's 3.09.
A fit with model = FALSE whose data are gone, given numbered data in
another order than the fit, is refused by the streaming tests, with "...
cannot use the model frame of model: it cannot be rebuilt (...), and data
does not give the response and the fitted values of the fit", as by the tests
built on the context. They read its rows by position and returned the values
of other rows: on lm(y ~ x1 + x2, data = gone, model = FALSE) for the 120
observations of contract_data(), given dplyr::as_tibble(d)[sample(120), ]
after set.seed(3), White's streaming test returned 8.887 where the data in
the order of the fit give 11.145. Given data that hold its rows, by row name,
or by row number for a fit to data that number their rows, and give its
response and fitted values, such a fit is tested on them, as before. The
numbered data of a fit to data with row names of their own, such as
dplyr::as_tibble(mtcars) for a fit to mtcars, lack those names, and are
refused, as before, now with "... cannot use the model frame of model: it
cannot be rebuilt (...), and the rows of data cannot be matched to the
observations of the fit: data has no rows named Mazda RX4, Mazda RX4 Wag,
...", by every test built on the context, where the streaming tests said
"Missing rows" and the others that data does not give the response and the
fitted values of the fit.
Also, for the three:
- Errors that become values. Without
data, a model that transforms its variables or has anoffset()term stopped with "The following variables are missing from input data: mpg, wt" (oro): onlm(log(mpg) ~ log(wt) + qsec, data = mtcars)White's streaming test gives 12.32 (p = 0.031), the value ofperformWhiteTest(). The complete rows of a fit withna.omitnumbered again, and the tibble in the order of the fit of a fit to named data, such asdplyr::as_tibble(mtcars), stopped with "Missing rows: 32" or "Missing rows: Mazda RX4, Mazda RX4 Wag, ..."; they give the value on the rows of the fit. - A multivariate
lmis refused with "... needs a model with one response;modelhas 2.", where they stopped with "'length = 4' in coercion to 'logical(1)'"; a fit withmodel = FALSEwhose data are gone, withoutdata, with the message of the context, where they stopped with "Expected a data.frame for input data". - An
mgcv::gam()fit with smooth terms is tested as before, on the design of the variables of its formula, which its model frame holds: onmgcv::gam(mpg ~ s(wt) + qsec, data = mtcars)the Breusch-Pagan statistic is 4.75 on 2 degrees of freedom, whereperformBPTest(), which tests such a fit on the basis of its smooths, gives 14.64 on 10. Which tests take such a fit, and on which design, is left to a later step of the model context. - White's streaming test still stops on collinear terms of its auxiliary
regression, the squares of dummies or of an orthogonal
poly()basis, whichperformWhiteTest()drops; that is the subject of a pull request of its own. Where it stopped before reaching that regression, on a variable missing from the model frame or on the new levels ofcut(), it now stops there. runHeteroTests()with a finitechunk_threshold_mb, which put a fallback under the names of the streaming tests where they stopped as above (White's test without cross products underwhite, and the NCV test underbreusch_paganandkoenker), runs them: onlm(y ~ x1 + x2 + offset(o))withoutdata, on the data of the test matrix (context_data(),tests/testthat/helper-model-context.R), it gave White's test without cross products, 10.80, and the NCV test's 2.22 under both other names, and gives White's streaming test, 11.28, the Breusch-Pagan test's 11.66 and Koenker's 9.29.- Unchanged: every other value of a valid call, the refusals of a zero weight
and of the guard, and every message the test suite pins. A corpus of 33,339
calls, the three in chunks of 7, 40 and 10,000 and through the runner with
chunk_threshold_mb = 0, and every other test, on 361 combinations of fits, offsets, terms, forms ofdata, dropped rows, missing groups, row names, panels and classes, on the five fits of the table of the Harrison-McCabe rank and the eight designs of White's degrees of freedom, and the examples of thirty-nine help pages, returns what it returned before,identical(), but for the 762 calls of the changes above (scripts/model-context-identity.R), every one of them a call of the three or of the runner that streams them, 55 of them the new refusals in bold; each of the 562 that now return a value, or another one, gives the value of its reference: the same call on the data frame of the fit or on the data of agamin its order, or the fit with the column of the term computed beforehand. The warnings are those of before on every call. - On a
poly()term a value can change in its last bits: the chunks computed the basis again by the recurrence of thepredvarsof the terms, and the design of the fit holds the basis thatpoly()computed by its QR decomposition. Withset.seed(101); n <- 500; d <- data.frame(x1 = runif(n, 0.5, 3), x2 = rnorm(n), x3 = rexp(n)); d$y <- 2 + d$x1 - 0.7 * d$x2 + 0.3 * d$x3 + rnorm(n, sd = d$x1),performBPTestStreaming(lm(y ~ poly(x1, 2) + x2, d), d)gave 156.20252285478642 and gives 156.20252285478654; of 621 such calls of the review of this step, 38 changed, by at most 9.9e-14 relative. The 48 calls of the corpus with a value on apoly()term areidentical()to before.
Which models the tests accept¶
Every exported test was run on an lm, a weighted lm, an lm with a zero
weight, a Gaussian glm (unweighted, weighted, with a zero prior weight), a
binomial and a Poisson glm, a survey::svyglm() fit, a plm fit,
rank-deficient fits, MASS::rlm() and fitWLS() fits, through every route
that calls it (runHeteroTests(), runDiagnostics(), the registry,
test_factory, cachedTest(), runHeteroTestsParallel(), runPanelTests(),
runTimeSeriesTests()). The result is the table of the new Supported
models page of the site, and ?test_results sets out the rules. The changes
in bold change what calls return.
A rank-deficient fit is tested on the estimable part of its design. An
aliased term, such as z2 = 2 * z, or the dummy of a cell of an interaction
that no observation falls in, leaves an NA coefficient. The residuals and
fitted values of such a fit are the projection on the column space of the
design, which the aliased columns do not change, so they are those of the
same model fitted without them. Every test refused such a fit with "Model
coefficients must be finite before running ..."; every test that accepts the
model without the aliased columns now accepts it, drops the aliased columns
from the design it builds, takes its degrees of freedom from the rank, and
returns the statistic and p-value of the model fitted without them. A test
that refuses that model, as the quantile regression test refuses a weighted
fit, refuses the rank-deficient one with the same message.
tests/testthat/test-model-eligibility.R checks all 39 tests on four such
fits (an aliased regressor in an lm, a weighted lm and a Gaussian glm,
and an empty cell of an interaction) to 1e-10; on them the two agree to the
last digit. On lm(y ~ x + z + z2) in the design of
tests/testthat/test-koenker-studentized.R:
| Call | 0.15.0 | Now |
|---|---|---|
performKoenkerTest(), performStudentizedBPTest() |
Error: "Model coefficients must be finite" | 20.87 on 2 degrees of freedom (p = 2.9e-05), as lmtest::bptest() |
performWhiteTest() |
Error | 22.43 on 5 degrees of freedom (p = 0.00043) |
performGQTest(order_by = "x") |
Error | 4.208 on 57 and 57 degrees of freedom (p = 1.0e-07) |
performRESETTest() |
Error | F = 0.359 on 2 and 145 degrees of freedom (p = 0.699) |
performQuantileRegressionTest() |
Error | F = 18.53 on 2 and 298 degrees of freedom (p = 2.6e-08) |
runHeteroTests() |
Error | White's and Koenker's tests, as above |
These are the values for lm(y ~ x + z). lmtest::bptest() and
car::ncvTest() give the same value on the rank-deficient fit;
lmtest::gqtest() and lmtest::resettest() count the aliased column and
give 56 and 144 degrees of freedom, against 57 and 145 on lm(y ~ x + z).
The quantile regression test fits quantreg::rq(), which refuses a singular
design, to the columns that were not aliased. A resampling replicate whose
refit aliases a coefficient the fit estimated, as a pairs resample of an
interaction can, fails, as it did when every test refused it. The White
streaming test stops on the empty cell, because the squares of its dummies
are collinear, as it stops on the model without the aliased column. A
coefficient that is infinite, NaN, or NA without the fit being
rank-deficient is still refused. validateTestInputs(), a function defined
outside the package that a user registers with test_factory, and
generateHeteroRecommendations() still refuse a rank-deficient fit, now with
"cannot use a rank-deficient model: the coefficient of z2 is aliased (NA)";
test_factory leaves it to any function of the package.
The model without the aliased columns is the one lm() keeps: of an aliased
set, the column that comes last in the order of the terms is left out. Most
statistics depend only on the space the columns span; White's test without
its cross products (cross_products = FALSE, method = "reduced" of
performWhiteTestRobust(), or more regressors than max_interactions) and
performHighDimensionalTest() when it keeps fewer components than columns
depend on which column is left out, and so on the order of the terms. One
test does not give the value of the model without the aliased columns:
performVarianceFormTest(form = "power") takes the logarithms of every
column of the design, the aliased ones included, and the logarithm of an
aliased column is not a combination of the logarithms of the others. With
x14 = x1 + x4, lm(y ~ x1 + x4 + x14) gives 1.352 on 6 and 150 degrees of
freedom, where lm(y ~ x1 + x4) gives 2.015 on 3 and 154 and
lm(y ~ x14 + x1) 1.343, in the design of test-model-eligibility.R; the
value does not depend on the order of the terms. With the exponential form,
the default, all three give 0.922 on 3 and 154.
A Gaussian glm with a zero prior weight is refused as the same lm
is. A zero weight removes an observation from the fit, and its residual is
an artificial zero that the tests would count. The tests refused an lm with
a zero weight but read the weights of a glm through a helper that returns
none for a glm: with cars$w <- 1 / cars$speed and a zero first weight,
performKoenkerTest(glm(dist ~ speed, data = cars, weights = w), cars)
returned 0.750 (p = 0.386) and performWhiteTest() 0.801 (p = 0.670). The
32 tests that returned a value for such a glm, performScatterDiagnostic(),
and so runHeteroTests(), runDiagnostics(), the registry, test_factory,
cachedTest(), runHeteroTestsParallel(), runPanelTests() and
runTimeSeriesTests(), now stop with the message they give for the lm,
"The model was fitted with zero or non-finite weights";
performWhiteTestBootstrap() gives it instead of "Model residuals must be
finite", and performSpatialHeteroTest() instead of "objects of different
length". The three tests that refused every glm refuse it with the message
they give the lm (below). performRESETTest() accepts a zero weight, on
an lm and on a glm, as before: its two fits are weighted least squares,
in which a zero weight drops the observation from both, and its F statistic
and degrees of freedom are those of the fit to the observations with positive
weight (1.489 on 2 and 45 degrees of freedom for cars above, either way).
The residuals of a weighted Gaussian glm are still its deviance residuals,
so no value of a glm with positive weights changes, and plotBeforeAfter()
now labels them
"(weighted residuals)", as it labels those of a weighted lm.
performWhiteTestStreaming(), performHarrisonMcCabeTest() and
performVarianceFormTest() accept a Gaussian glm with the identity link,
and return the value of the lm. They refused every glm, where the other
tests accept this one. A Gaussian glm with the identity link and prior
weights w is fitted by maximum likelihood, which for normal errors is
weighted least squares: iteratively reweighted least squares has the
working response y and the working weights w whatever the current
estimate, so its first step is the fit of lm(..., weights = w) (Nelder and
Wedderburn, 1972; McCullagh and Nelder, 1989; Green, 1984). Its design,
residuals and weights are those of the lm, and the three statistics and
their null distributions are functions of those alone; the Supported models
page sets out the argument, test by test. The limit was in the code: the
White streaming test runs checks that read the R-squared of summary(),
which a glm does not report, and the other two read the weights through a
helper that returns none for a glm. They read the prior weights now, and
the checks the R-squared of the least-squares fit.
performWhiteTestStreaming(glm(mpg ~ wt + qsec, data = mtcars), mtcars)
stopped with "needs a model fitted by stats::lm()" and returns 11.82 on 5
degrees of freedom (p = 0.037), the value of the lm;
performHarrisonMcCabeTest(..., order_by = "wt") returns 0.5595
(p = 0.693) and performVarianceFormTest() F = 1.765 on 3 and 26 degrees of
freedom (p = 0.179), as for the lm. With prior weights 1 / wt the first
two give 14.52 and 0.6362, the values of the weighted lm, and the
variance-form test refuses the weights, as it refuses those of the lm: they
did not come from fitWLS(). runHeteroTests() on such a glm, when it
takes its streaming path (data above chunk_threshold_mb, which since this
release must be given: The streaming tests, above), fell back on the
simplified White test under the name white, tagged as a fallback: 10.52 on
4 degrees of freedom (p = 0.033) for glm(y ~ x1 + x2) in the design of
contract_data() (tests/testthat/helper-test-recipes.R). It returns the
streaming White test now, 11.14 on 5 (p = 0.049), the value of the lm.
The value is that of the lm when the two decide the rank of the design
alike: glm.fit() aliases a column with the tolerance 1e-11 by default,
lm() with 1e-7, so on a nearly collinear column the glm gives the
value of lm(..., tol = 1e-11) (the Supported models page). A glm of
another family or link is refused,
now with the message the other tests give, which names the family and the
link. tests/testthat/test-model-eligibility.R compares the glm and the
lm in the three tests on 34 calls (unweighted and weighted, with na.omit,
a factor and a rank-deficient design) to 1e-10; they differ by rounding,
below 1e-12. On 936 calls of every test on lm, weighted lm, fitWLS()
and MASS::rlm() fits, every result is identical() to before.
A near-perfect Gaussian glm is refused as the same lm is. The checks
every test runs refuse a fit with an R-squared above 0.999 ("Model appears
perfectly explained"), read from summary(), which reports none for a
glm, so 30 tests gave such a glm a value: with
cars$y <- 2 + 3 * cars$speed + rnorm(50, sd = 0.05) (set.seed(1)),
performKoenkerTest(glm(y ~ speed, data = cars), cars) returned 0.957
(p = 0.328). They read the R-squared of the least-squares fit now, which for
the glm is the R-squared summary.lm() computes from its fitted values,
residuals, weights and offset, those of the lm, and refuse it.
checkModel(enhanced = TRUE), which stopped on any glm with "missing
value where TRUE/FALSE needed", warns about such a fit as about the lm,
and skips the check for a glm of another family. A Gaussian glm with a
zero prior weight no longer warns "observations with zero weight not used
for calculating dispersion" before its refusal, which summary() of a glm
gave.
fitWLS() refits a Gaussian glm as the lm it is, and refuses any other
glm. It refitted any glm and kept the class and the call of the glm
without its family, which summary() could not read and every test refused.
Of a Gaussian glm it now returns the lm it returns for the equivalent
lm, with the prior weights of a weighted glm recorded and the call of
that lm, stats::lm() without the arguments only glm() takes, so that
summary() prints it and update() fits an lm again;
performVarianceFormTest(fitWLS(glm(mpg ~ wt + qsec, data = mtcars))) gives
F = 1.765, as for the lm. Of a binomial or Poisson glm it stops with
"fitWLS() needs a model fitted by stats::lm()", and of a survey::svyglm()
fit, which it returned with the class of the svyglm, with the message the
tests give for one.
performScatterDiagnostic() refuses a glm that is not a least-squares
fit, and data that lack a variable of the model. Its absolute residuals vary
with the mean by construction:
performScatterDiagnostic(glm(am ~ wt, data = mtcars, family = binomial), mtcars, "wt")
returned -0.180, and with qsec dropped from mtcars the diagnostic of
lm(mpg ~ wt + qsec, data = mtcars) returned -0.141; both now stop with the
messages the tests give.
The registry name "ncv" checks the data it is given, as
performNCVTest() does. It ignored data, and because it is the last
fallback of "white", "koenker" and "breusch_pagan",
runHeteroTests(lm(mpg ~ wt + qsec, data = mtcars), mtcars[1:20, ]), or with
data that lack qsec, returned the Cook-Weisberg statistic, 0.591 on 1
degree of freedom, under the names white and koenker, tagged as a
fallback, as did runDiagnostics(). They now stop with the refusal of White's test, as
cachedTest("ncv", ...) and runHeteroTests(tests = "ncv") stop with that of
performNCVTest(). On data that hold the rows and variables of the fit, and
with data = NULL, every value is identical() to before.
runHeteroTests() on a tidymodels workflow without data runs the tests
on the model frame of the engine's fit. It gave them the predictors of the
workflow's mold, which lack the response; every test refused them, and the
result was the same Cook-Weisberg fallback under the names asked for. For a
workflow of linear_reg() with recipe(mpg ~ wt + qsec, data = mtcars) it
returned 0.591 under white and koenker, and returns White's statistic,
11.82 on 5 degrees of freedom, and Koenker's, 3.086 on 2, those of the lm.
Given the data the workflow was fitted to it now stops, as the engine's fit
names its response ..y, which those data do not hold, with a message that
says so and to pass data = NULL; it returned the same fallback.
Four tests account for the offsets they ignored. performRESETTest()
refitted its augmented model without the offset; performGQTest() refitted
each segment without an offset given as an argument, and, on a weighted fit,
without any offset; performQuantileRegressionTest() fitted
quantreg::rq(), which leaves the offset out; and the studentized
Breusch-Pagan statistic of performStudentizedBPTest(), of
performBPTestRobust() with studentize = TRUE, its default, and of the
registered "student_bp" refitted the design without it. On a model with an
offset they returned a value that is not that of the model. Decided on
2026-10-17, the development version refused such a model, with a message
that said the test did not account for the offset yet and which tests did, of
class heteroTests_offset_error, on which the runners did not fall back on
another test, until the offset fix planned for 0.16.0. They refused exactly
the offsets they left out: an offset() term on an unweighted fit, which the
refits of GQ evaluated again, was accepted, as were the classical statistic of
performBPTestRobust(studentize = FALSE) and an offset that is zero
everywhere. The second step of the model context is that fix, and lifted the
refusal before the release: the four now give the value of the model (The
model context, below, with the values).
Calls that go through the tests change with them. On data that lack rows
or variables of the fit, compareTestResults() and
generateHeteroRecommendations() stop, where they returned a data frame and a
report built on the Cook-Weisberg fallback. runAdvancedDiagnostics() and
compareTestResults() accept a rank-deficient fit and refuse a Gaussian glm
with a zero prior weight, and rcalculateEffectSize() and
restimate_test_power() accept a rank-deficient fit. runDiagnostics()
returns for a rank-deficient fit, where it stopped: its variance inflation
factors are all Inf, and the cutoff of its influence measures counts the
aliased coefficient (recorded in the roadmap). plotBeforeAfter(),
plotResidualsFitted() and the other plots of the residuals stop on a
Gaussian glm with a zero prior weight, as on the same lm.
Also:
- The tests read the prior weights of a Gaussian
glm,prior.weights, and never its working weights,weights, which are the weights of the last iteration of iteratively reweighted least squares. For the Gaussian family with the identity link the two are equal; for any other family or link they depend on the fitted means, and such aglmis refused.?test_resultssays so. checkModel()checks the class of a model only, and says so: the tests decide which models they accept.- A model that is neither an
lmnor aglm, such as aplm::plm()fit, is refused by every test,runPanelTests()andrunTimeSeriesTests()with one message, which names the class received, as inProvide an object fitted with stats::lm() or stats::glm() before running performKoenkerTest() (received class: plm/panelmodel), and goes on to say that a panel is tested as its pooledlm(), with the units and the periods as columns ofdata. It was refused with four different messages. - The Supported models page lists, for each of the four tests above, the
offsets it refuses and those it accepts, and records that
performRESETTest()keeps accepting a zero weight (decided 2026-10-17). - The valid calls of a corpus of 1,198 on the supported classes (an
lm, weighted or not, a Gaussianglm, weighted or not,MASS::rlm()andfitWLS()fits, and seven other specifications, through every test and route) return what they returned in 0.15.0,identical(), but the label ofplotBeforeAfter()for a weighted Gaussianglm.
The model context¶
performWhiteTest(), performBPTest() and its deprecated name
performBreuschPaganTest(), performKoenkerTest(),
performRankPermutationTest() and performHighDimensionalTest() read the
response, the design, the residuals, the weights and the offset from the
fit, through an internal model context, and no longer rebuild the design from
data. Data that are given are matched to the observations of the fit by
the guard of 0.15.0 before anything is read from them, and nothing is
dropped before the match: named data in any order, numbered data in fit
order. This is the first of eleven steps; the design, the plan and the
decisions are on the
design page
of the site. Through these tests the change reaches the base statistic of
performWhiteTestRobust() and performBPTestRobust(studentize = FALSE), the
registry names white, breusch_pagan, koenker, rank_permutation and
high_dimensional, and the default battery of runHeteroTests() and
runDiagnostics(). performWhiteTestBootstrap() does not change: its
observed statistic and its replicates keep the computation of 0.15.0, so that
both come from the same kind of design, until a later step moves both; and
the streaming tests, which runHeteroTests() runs in place of three of the
five only when asked for since this release (The streaming tests, above),
still rebuild the design chunk by chunk until the step that migrates them.
A term computed from all the rows of the data is tested as the fit
computed it. A fit that drops rows computes a term such as
I(wt > median(wt)) on all the rows of the data before it drops the
incomplete ones; the tests rebuilt the design on the rows the fit kept, where
the median is another, so they tested a design the fit never used. With
the three heaviest cars are dropped, the median of wt on the 29 that are
left is lower, and the indicator changed for the cars in between. The tests
now give the value of the same fit with the indicator computed beforehand,
d$heavy <- d$wt > median(d$wt) and lm(mpg ~ heavy + qsec, data = d):
| Call | 0.15.0 | Now |
|---|---|---|
performWhiteTest(m, d) |
9.634 (p = 0.047) | 10.00 (p = 0.040) |
performBPTest(m, d) |
8.463 (p = 0.0145) | 8.647 (p = 0.0133) |
performKoenkerTest(m, d) |
9.348 (p = 0.0093) | 9.551 (p = 0.0084) |
performRankPermutationTest(m, d), set.seed(1) |
-0.676 (p = 0.001) | -0.738 (p = 0.001) |
performHighDimensionalTest(m, d) |
2.841 (p = 0.092) | 2.877 (p = 0.090) |
A term that does not depend on the rows it is computed from (log(),
poly(), ns(), scale() through their predvars) gave the value of the
fit already.
Without data, runHeteroTests() runs White's and Koenker's tests on a
fit whose weights or offset are a column of the data. On
d$w <- 1 / d$wt and m <- lm(mpg ~ wt + qsec, data = d, weights = w) with
d <- mtcars, performWhiteTest(m) and performKoenkerTest(m) stopped with
"object 'w' not found", and runHeteroTests(m) returned the Cook-Weisberg
statistic, 1.545 (p = 0.214), under the names white and koenker, tagged as
a fallback, as did runDiagnostics(). It returns White's statistic, 14.52 on
5 degrees of freedom (p = 0.013), and Koenker's, 4.413 on 2 (p = 0.110), the
values with data. With an offset given as a column, offset = o for
d$o <- 0.1 * d$hp, it returned the Cook-Weisberg statistic 2.552 under both
names, and returns White's 17.91 (p = 0.0031) and Koenker's 0.294 (p = 0.863).
Also, for the five tests:
- Errors that become values. Weights or an offset taken from a column of the
data, with
dataomitted or the model frame given asdata: Koenker's statistic of the weighted fit above, 4.413, withperformKoenkerTest(m)and withperformKoenkerTest(m, model.frame(m)). Transformed variables andoffset()terms withoutdata:performKoenkerTest(lm(log(mpg) ~ log(wt) + qsec, data = mtcars))stopped with "The following variables are missing from input data: mpg, wt", and gives 5.518 (p = 0.063), the value withmtcars; the same holds forpoly(),ns(),I(wt > median(wt))andcut(), which the model frame holds as terms. Numbered data that hold the rows a fit withna.omitdropped, or only the rows it kept, numbered again, a tibble with the rows it dropped, the tibble the model was fitted on, and a tibble in the order of a fit to data with character row names: they stopped with "Missing rows", or with "numbers the rows that are left afresh", and give the value on the rows of the fit. Numbered data with one row per observation are matched in the order of the fit once their values reproduce it (decided on 2026-10-18). A fit withsubset =given by index:performKoenkerTest(lm(mpg ~ wt + qsec, data = mtcars, subset = 3:32), mtcars)stopped with "could not align the design matrix with model residuals" and gives 2.442, the value of the fit tomtcars[3:32, ]. Acut(x, quantile(x))term:performKoenkerTest(lm(mpg ~ cut(wt, quantile(wt)) + qsec, data = mtcars), mtcars)stopped with "factor ... has new levels" and gives 11.13 (p = 0.025), the value of the same fit withx = TRUE. - A multivariate
lm,lm(cbind(mpg, qsec) ~ wt, data = mtcars), is refused withperformKoenkerTest() needs a model with one response; `model` has 2.instead of the coercion error "'length = 4' in coercion to 'logical(1)'", also throughrunHeteroTests(). - On a fit with a
poly()term the value equals, withidentical(), what 0.15.0 computes on the fit's stored design (the model refitted withx = TRUE); differences against the design 0.15.0 rebuilt are in the last bits (measured up to 1.3e-15 relative on the statistic). - A fit whose coefficients, residuals or weights were changed after it was
fitted is refused ("The fitted model does not reproduce its residuals from
its design and coefficients", "The weights of the model are not those its
coefficients were fitted with"). A fit with
model = FALSEwhose data are gone, so that its model frame can no longer be rebuilt, is tested ondatawhendatagives the response and the fitted values of the fit, as it was tested before without that check, and refused otherwise, as withdatawhose response was reversed, for which a value was returned. - These refusals have class
heteroTests_model_context_error, andrunHeteroTests()andrunDiagnostics()keep it and run no other test under the name of the one refused, as for a refusal ofdata; on an edited fit the runner put the Cook-Weisberg test under the nameswhiteandkoenker. - The rank permutation test, when its
order_bynames a column ofdatathat the model frame does not hold, refuses numbered data taken in the order of the fit if two rows of the fit tie on every compared column, which could then have been exchanged; such data stopped with "Missing rows" before. bootstrapTestStatistic(),performWhiteTestRobust(bootstrap = TRUE)andperformBPTestRobust(bootstrap = TRUE)refuse a fit whose model frame cannot be rebuilt fromdataunder the levels of the fit, as withcut(x, quantile(x)). They refit their replicates from the formula until a later step, and those replicates would compute other breaks than the fit. Where the base statistic already failed, the error changes from "factor ... has new levels" to "datadoes not reproduce the fit". Some of these calls returned a value before, for examplebootstrapTestStatistic(performGlejserTest, ...)on such a fit, and the robust bootstraps on the same fit refitted withx = TRUE(White 11.32, p = 0.184; Breusch-Pagan without studentizing 15.79, p = 0.0033). That value compared the statistic of the fit with replicates of another design, and these calls now stop. Calls that stopped in their base statistic and now reach the checks of the bootstrap stop there instead, with its own message.- The warning "Removed N observations due to missing values in ..." is still
given for the incomplete rows of
data, which the fit left out, and nothing is dropped. Koenker's test gives the warnings of the checks of the model once; it gave them twice. - Unchanged: every other value of a valid call, every refusal of the guard of
0.15.0, numbered data in another order than the fit, which stay refused,
and every message the test suite pins; on a call with more than one defect,
the first one reported may differ, since the rows of
dataare matched before the checks of the model. Data with fewer rows than the fit now stop with "Missing rows: ..." instead of "Input Data contains only N observations ...", for the same reason.performWhiteTestRobust()andperformBPTestRobust()keep their own preparation of the data until a later step, so they still needdatafor a model that transforms its variables, and a fit withna.action = na.excludeis still refused. A corpus of 11,271 calls, every test and the bootstraps above on 262 combinations of fits, offsets, terms, forms ofdata, dropped rows and row names, the default battery and the examples of the five help pages, returns what it returned in 0.15.0,identical(), but for the 591 calls of the changes above (scripts/model-context-identity.R); the rows of the five tests in the studies ofinst/validation/that use them are reproduced byte for byte.
The second step moves the studentized Breusch-Pagan statistic
(performStudentizedBPTest(), performBPTestRobust() with
studentize = TRUE, its default, the registered "student_bp", its
test_factory entry and rvalidate_against_reference("studentized_bp")),
performRESETTest(), and so runDiagnostics(), which runs it,
performGQTest() and performQuantileRegressionTest(), and so the registered
"quantile_regression", onto the model context. They read the response, the
design, the weights and the total offset from the fit, and every refit
carries the offset. Data that are given are matched as for the first five:
named data in any order, numbered data in fit order. Goldfeld-Quandt reads its
order_by at the confirmed rows of data, or, without data, from the model
frame when it holds the variable under its own name. Without an offset,
without a term that depends on the rows it is computed from, and on data
that 0.15.0 accepted, every value is identical() to before, but on the
calls the four now refuse, in bold below: an mgcv::gam() fit with smooth
terms, a model frame rebuilt from changed data, a multivariate lm without
data in RESET, and, through the runner, a model = FALSE fit whose data
are gone.
On a model with an offset the four give the value of that model. The
refusals of the offsets they left out (Four tests account for the offsets
they ignored, above) become values. Goldfeld-Quandt, the quantile regression
test and the studentized statistic give the value of the model fitted to the
response minus the offset; RESET gives the nested F test of the model with
its offset against the model augmented with the squares and cubes of its
fitted values, X b + o, both fitted with the offset and the weights of the
fit (decided 2026-10-18). RESET of the model of y - o is another test,
which augments that model with the powers of X b alone. With
set.seed(33); n <- 160
d <- data.frame(x1 = runif(n, 1, 3), x2 = rnorm(n), u = rnorm(n))
d$o <- 0.8 * d$u
d$y <- 1 + d$x1 + d$o + rnorm(n, sd = d$x1)
m <- lm(y ~ x1 + x2, data = d, offset = o)
and the same model fitted to the response minus the offset,
lm(I(y - o) ~ x1 + x2, data = d):
| Call | 0.15.0, on m |
On the model of y - o |
Development version, on m |
Now, on m |
|---|---|---|---|---|
performRESETTest() |
F = -0.289 (p = 1) | F = 4.487 (p = 0.013) | Error: performRESETTest() does not account for an offset yet: ... |
F = 0.112 (p = 0.894), the nested F test of m |
performGQTest(order_by = "x1") |
1.634 | 2.383 | Error | 2.383 (p = 0.00044) |
performQuantileRegressionTest() |
0.897 | 3.256 | Error | 3.256 (p = 0.040) |
performStudentizedBPTest(), performBPTestRobust() |
4.877 | 9.762, Koenker's statistic on m |
Error | 9.762 (p = 0.0076), Koenker's |
In the design of test-fitwls-reestimation.R the studentized statistic of
lm(y ~ x + z, offset = o) was 19.91, and is Koenker's, 18.79. The values
reach performBPTestRobust() with and without bootstrap,
runHeteroTests(m, d, tests = "student_bp") and tests =
"quantile_regression", cachedTest(), the registry, test_factory,
runHeteroTestsParallel() and grouped data, and runDiagnostics(), which
runs RESET and returns for a model with an offset. rrefuse_offset() and the
error class heteroTests_offset_error are gone. An offset that is zero
everywhere changes nothing.
Goldfeld-Quandt fits each segment to the rows of the design of the fit.
It refitted the segments of an unweighted fit from the formula, which
computed again, on the rows of each segment, a term that depends on the rows
it is computed from: ns() and bs() placed their knots at quantiles of the
segment, and I(x > median(x)) took the median of the segment (decided
2026-10-18). It now gives the value of lmtest::gqtest(), which fits the
segments to the rows of the design, wherever the design of each segment has
full rank:
Call, on mtcars (fraction = 0.2) |
0.15.0 | Now, as lmtest::gqtest() |
|---|---|---|
performGQTest(lm(mpg ~ splines::ns(wt, 3) + qsec), order_by = "wt") |
0.574 (p = 0.773) | 0.432 (p = 0.869) |
the same, order_by = "qsec" |
4.894 (p = 0.025) | 4.570 (p = 0.030) |
lm(mpg ~ splines::bs(wt, df = 4) + qsec), order_by = "qsec" |
3.536 (p = 0.073) | 3.952 (p = 0.057) |
lm(mpg ~ I(wt > median(wt)) + qsec), order_by = "qsec" |
2.034 (p = 0.150) | 2.241 (p = 0.120) |
poly() computed another basis of the same columns on each segment, which
changes the last bits only (by at most 8.1e-15 relative in the corpus below),
and log() or I(x^2) the same columns. A weighted fit was already fitted on
the design of the fit. A segment whose design is rank-deficient, as when a
column of bs() or an indicator is zero on all its rows, takes its degrees of
freedom from its rank, as before; lmtest::gqtest() takes them from the
number of columns.
The quantile regression test fits its quantile regressions to the design
of the fit. It refitted them from the formula on the rows of the fit, which
computed again a term that depends on the rows it is computed from. A fit with
cut(x, quantile(x)) drops the rows at the minimum, which fall outside the
breaks, and the refit computed other breaks on the rows that were left and
dropped others: on lm(stations ~ cut(mag, quantile(mag)) + depth, data =
quakes), a fit to 954 of the 1000 rows, it gave F = 16.44 on 4 and 1794
degrees of freedom (p = 3.1e-13), and gives F = 17.61 on 4 and 1904
(p = 3.4e-14), the value of the same fit with the factor computed beforehand.
A term computed from all the rows, on a fit that dropped rows, is tested as
the fit computed it, by Goldfeld-Quandt and the quantile regression test
too, as by the first five (above): on the cell I(x1 > median(x1)) with
na.omit of the test matrix, Goldfeld-Quandt changes by 10% and the quantile
regression test by 0.16%, each to the value of the same fit with the
indicator computed beforehand.
RESET, Goldfeld-Quandt, the quantile regression test and the studentized
Breusch-Pagan statistic refuse a fit with smooth terms, and so do the calls
that run them. A smooth term of a Gaussian mgcv::gam() fit is usually
penalised, and a penalised fit is not the least-squares fit of its design,
which each of these tests fits again: it would test a regression the model
is not. The four refuse every fit with smooth terms, fixed ones
(s(x, fx = TRUE)) included. On
m <- mgcv::gam(stations ~ s(mag) + depth, data = quakes) the studentized
statistic returned 117.0 (p = 2.1e-20), as did
runHeteroTests(m, quakes, tests = "student_bp"), performBPTestRobust(),
test_factory and rvalidate_against_reference("studentized_bp"), and RESET
F = 14.03 (p = 9.8e-07); runDiagnostics(m, quakes), which runs RESET,
returned its list with that value; Goldfeld-Quandt and the quantile test
stopped with
could not find function "s". They now stop with cannot test an mgcv::gam()
fit with smooth terms (s(mag)), of class heteroTests_model_context_error,
which the runners keep, so runDiagnostics() stops on such a fit too, until
the last step of the model context settles which tests take it. The first
five test it as before, and a gam without smooth terms is tested as the
Gaussian glm it is.
RESET refuses a model frame that model = FALSE rebuilt from data changed
after the fit. It read that frame: with
m0 <- lm(mpg ~ wt + qsec, data = d0, model = FALSE) for d0 <- mtcars, it
gives F = 6.731 (p = 0.0043), and after d0 <- mtcars[order(mtcars$wt), ] it
returned F = 0.136 (p = 0.874), the test of a response in another order. It
now stops, as the other tests built on the context do ("cannot use the model
frame of model").
RESET refuses a multivariate lm. Without data it tested
lm(cbind(mpg, qsec) ~ wt + hp, data = mtcars) and returned F = 7.197 on 2
and 25 degrees of freedom (p = 0.0034), a statistic of neither response. It
now stops with performRESETTest() needs a model with one response; `model`
has 2., as the other tests built on the context do; with data it stopped
already, with "RESET requires data with 64 observations to match the fitted
model, got 32."
The registered "student_bp" and "quantile_regression" refuse, without
data, a fit with model = FALSE whose data are gone. Its model frame
cannot be rebuilt. On lm(stations ~ mag + depth, data = gone, model = FALSE)
for gone <- quakes, removed after the fit,
runHeteroTests(m, tests = c("student_bp", "quantile_regression")) returned
the Cook-Weisberg statistic, 188 (p = 8.8e-43), under both names, tagged as a
fallback. It now stops with "... cannot use the model frame of model: it
cannot be rebuilt ...", of class heteroTests_model_context_error, as its
default battery already did. Given data that give the response and the
fitted values of the fit, the tests run on them (below).
Also, for these four:
- Errors that become values, as for the first five: weights or an offset
taken from a column of the data with
dataomitted or the model frame given, transformed variables andoffset()terms withoutdata, numbered data that hold the rows a fit withna.omitdropped, or only those it kept, numbered again, the tibble a model was fitted on, a fit withsubset =given by index, andcut(x, quantile(x)). Goldfeld-Quandt withoutdataon a fit that transforms the variable it is ordered by stops with "The following variables are missing from the model frame ofmodel: x1. It holds the terms of the model (...)", which says to passdata; it stopped with "The following variables are missing from input data". - A fit with
model = FALSEwhose data are gone, givendatathat give its response and fitted values: Goldfeld-Quandt and the studentized statistic stopped with "object 'gone' not found", and are tested ondata, as the first five are. RESET, which builds its context withoutdata, still refuses it, now with the message of the context. - The function registered as
"student_bp", called withdata = NULL, reads the model frame, as the other routes and every other registered name do; it stopped with "Expected a data.frame for input data". - A multivariate
lmgiven withdatais refused with "... needs a model with one response;modelhas 2.", as by the first five, where the tests stopped with a coercion error, and RESET with the count of the rows ofdata. - A call with an offset on data the guard refuses now stops with the refusal
of
data, where it stopped with the refusal of the offset; the robust Breusch-Pagan test, which keeps its own preparation of the data until a later step, stops with its check of the variables ofdatawhere that comes first. - RESET reads nothing from
data: it builds its context withoutdata, and keeps the check ofdataof the tests computed from the model alone, for the variables of the model and the row names of the fit, until the step that moves those tests, so that its values on every form ofdatastay those of 0.15.0. - A fit with a zero weight is refused by Goldfeld-Quandt and the studentized statistic, and accepted by RESET, as before.
- Unchanged: every other value of a valid call, and every message the test
suite pins. A corpus of 14,037 calls, the four, their routes and every other
test on 275 combinations of fits, offsets, terms, forms of
data, dropped rows, row names and classes, the default battery and the registered names"student_bp"and"quantile_regression"throughrunHeteroTests(), and the examples of the ten help pages, returns what it returned before,identical(), but for the 966 calls of the changes above (scripts/model-context-identity.R), 24 of them the new refusals in bold; each of the 708 that now return a value, or another one, gives the value of its reference, compared on every number of the result: the same call on the data frame of the fit, on the fit to the complete rows or to the subset, on the fit with its terms computed beforehand, on the model ofy - o, or, for RESET, the nested F test written out withanova(). The rows of the four tests in the studies ofinst/validation/that use them are reproduced byte for byte, and RESET on the fit with an offset ofoffset-refits-size.Rrejects 0.0492 of the time under its null at 5000 replications, inside the release gate, [0.042, 0.058]. The warnings of the calls of the corpus are those of before, but for 39 calls that stopped and now return a value with the warning ofquantreg::anova.rqs()on the same design ("non-positive fis"), and four calls ofperformStudentizedBPTest()on a multivariatelmand on amodel = FALSEfit whose data are gone, withoutdata, which it now refuses before its deprecation warning.
The third step moves performGlejserTest(), performParkTest(),
performSzroeterTest() and the registered "szroeter",
performHarrisonMcCabeTest(), performHartleyFmaxTest() and
performSpatialHeteroTest() with the registered "spatial_hetero" onto the
model context. They read the residuals, the weights, the design and the rank
from the fit, and the variable they name, the variable Glejser's and Park's
tests regress on, the order_by of the Szroeter and Harrison-McCabe tests and
the group of Hartley's, at the rows of data the comparison confirmed, or,
without data, from the model frame when it holds the variable under its own
name. Data that are given are matched as for the first nine: named data in any
order, numbered data in fit order. The spatial test reads no column of data,
and its listw lists the observations of the fit in its order, as before.
Every value is identical() to before but for the changes in bold below.
Harrison-McCabe takes the rank of the fit, decided at the fit's
tolerance. Its null distribution is computed from an orthonormal basis of
the design, whose rank it decided again with qr() at the default tolerance,
1e-7. A Gaussian glm decides its rank at 1e-11 (glm.fit() takes
min(1e-7, epsilon / 1000)), and an lm at the tol it is given, so on a
design with a column that is a combination of the others to within 1e-7 but
not to within the fit's tolerance the residuals were those of a fit of one
rank and the null distribution that of a smaller one (decided 2026-10-18). It
now takes the rank of the fit, at its tolerance. With
x3 = x1 + 1e-9 * rnorm(120) added to the 120 observations of
contract_data() (tests/testthat/helper-test-recipes.R, seed 5) and the
model y ~ x1 + x2 + x3:
| Fit | Rank, tolerance | Statistic | p-value, 0.15.0 | Now |
|---|---|---|---|---|
glm(), order_by = "x1" |
4, 1e-11 |
0.31988 | 0.0022879 | 0.0022734 |
the same, order_by = "x2", point = 1 / 3, alternative = "two.sided" |
4, 1e-11 |
0.91344 | 0.91217 | |
glm(weights = 1 / x1^2), order_by = "x1" |
4, 1e-11 |
0.54773 | 0.79089 | 0.80233 |
the same, order_by = "x2", point = 1 / 3, two-sided |
4, 1e-11 |
0.58886 | 0.59106 | |
lm(tol = 1e-11), order_by = "x1" |
4, 1e-11 |
0.31988 | 0.0022879 | 0.0022734 |
lm(), order_by = "x1" |
3, 1e-7 (aliases x3) |
0.31840 | 0.0021134 | 0.0021134 |
glm(y ~ x1 + x2), order_by = "x1" |
3, 1e-11 |
0.31840 | 0.0021134 | 0.0021134 |
Each new value equals the test written out from the fit's own QR
decomposition, with its rank: the residual-maker of that basis, the
eigenvalues of the quadratic form on the full matrices, and Imhof's formula
integrated independently (context_hmc_by_hand() in
tests/testthat/helper-model-context.R), to 1e-8 relative on these designs,
the precision to which either decomposition fixes the direction that a column
at 1e-9 of the others adds, and to rounding on a well-conditioned fit. With the
rank of the fit, each part of the ordered sample needs at least as many
observations as the fit has coefficients: on the glm above, point = 3 and
point = 117 returned p = 0.0891 and 0.1227, and now stop with "Each part
needs at least 4, the number of coefficients."; point = 116 gives 0.11379,
where it gave 0.11151. On every other fit the two ranks agree and the value is
unchanged: a gam keeps the rank of its design at 1e-7, as before.
A gam given its data in another order is matched to them by row name.
The residuals of an mgcv::gam() fit carry no names, and the Glejser, Park,
Szroeter, Harrison-McCabe and Hartley tests matched data to them by
position, without an error: given the rows of the fit in another order under
their own row names, they paired each residual with the variable of another
row and returned a wrong value. They now match the rows by the row names of
the model frame of the fit, and give the value on the data in the order of the
fit (approved on 2026-10-19). On contract_data(), with the data of the fit
shuffled, d[sample(120), ] after set.seed(3):
Call, on d[sample(120), ] |
gam(y ~ x1 + x2), 0.15.0 |
Now | gam(y ~ s(x1) + x2), 0.15.0 |
Now |
|---|---|---|---|---|
performGlejserTest(m, data, "x1") |
-0.322 | 3.626 | -0.429 | 3.583 |
performParkTest(m, data, "x1") |
0.118 | 3.263 | -0.728 | 3.155 |
performSzroeterTest(m, data, order_by = "x1") |
-0.537 | 3.179 | -0.369 | 3.029 |
performHarrisonMcCabeTest(m, data, order_by = "x1") |
0.4818 | 0.3184 | 0.4661 | 0.3148 |
performHartleyFmaxTest(m, data, "group") |
2.185 | 3.080 | 1.867 | 2.773 |
Each new value is that of the same call on d, which is unchanged. The other
tests that read a column of data beyond the model matched a gam by
position too: the five group tests until the fourth step, below, and the
panel tests, performPesaranTest() and performBPRandomEffectsTest(), the
three streaming tests, and the replicates of performWhiteTestBootstrap()
until the steps that move them; the roadmap records them. The
spatial test reads no column of data and gave the value already.
The registered "szroeter" orders a gam with an offset argument by its
fitted values, offset included. It ordered the observations by
predict(type = "response"), which for an mgcv::gam() fit leaves out an
offset given as the offset argument, as its help page says, so it ordered such a
fit by X b alone. It now orders by the fitted values of the fit, X b + o,
which fitted() holds, as it orders the same gam with an offset() term
and the lm with that offset (decided 2026-10-20). With set.seed(1),
mtcars$o <- rnorm(32) and
m <- mgcv::gam(mpg ~ wt + qsec, data = mtcars, offset = o),
runHeteroTests(m, mtcars, tests = "szroeter") returned Q = 0.800386
(p = 0.2117), and returns 0.992131 (p = 0.1606), the value of
gam(mpg ~ wt + qsec + offset(o)) and of lm(mpg ~ wt + qsec, offset = o).
A fit with model = FALSE whose data are gone, given numbered data in
another order than the fit, is refused by the six. Such data, a tibble or a
data frame whose row names were reset, in another order, cannot be matched to
the observations of the fit (named data in any order, numbered data in fit
order), and the model frame built from them does not give the response of the
fit: the six stop with "... cannot use the model frame of model: it cannot
be rebuilt (...), and data does not give the response and the fitted values
of the fit", of class heteroTests_model_context_error, as the first nine do,
directly, through the registered names, test_factory and the runner. They
returned a value. On lm(y ~ x1 + x2, data = gone, model = FALSE) for the 120
observations of contract_data(), with gone removed after the fit and the
data given as dplyr::as_tibble(d)[sample(120), ] after set.seed(3),
Glejser's test returned -0.322 where the data in the order of the fit give
3.626, Park's 0.118 for 3.263, Szroeter's -0.537 for 3.179 and Hartley's 2.185
for 3.080, values of other rows; the spatial test, which reads no column of
data, returned its value, 0.0938, and is refused too. The Harrison-McCabe test
stopped already, with "object 'gone' not found".
runHeteroTests(tests = "spatial_hetero") runs the spatial test on data it
stopped on. On a fit whose model frame holds terms rather than variables,
given that model frame as data with its weights in
attr(data, "listw"), the spatial test stopped ("The following variables are
missing from input data"), and runHeteroTests() put the Breusch-Pagan
statistic it falls back on under its name, tagged as a fallback. It now runs
the spatial test. With m <- lm(log(mpg) ~ log(wt) + qsec, data = mtcars),
the weights of set.seed(1) and
lw <- spdep::nb2listw(spdep::knn2nb(spdep::knearneigh(cbind(runif(32), runif(32)), k = 4))),
and mf <- model.frame(m) with attr(mf, "listw") <- lw,
runHeteroTests(m, mf, tests = "spatial_hetero") after set.seed(7)
returned the Breusch-Pagan statistic 3.640 (p = 0.162) under the name
spatial_hetero, and returns Moran's I of the squared residuals, -0.1028
(p = 0.707), the value of performSpatialHeteroTest(m, mtcars, listw = lw).
The same holds for numbered data that hold the rows of the fit in its order,
on which the spatial test stopped with "Missing rows".
Also, for these six:
- Errors that become values, as for the first nine. Numbered data that hold
the rows a fit with
na.omitdropped, or only those it kept, numbered again, the tibble a model was fitted on, the tibble of a fit to repeated rows, and the tibble in fit order of a fit to data with character row names stopped with "Missing rows" and give the value on the rows of the fit. Withoutdata, a model with anoffset()term is tested when the variable the test reads is a column of the model frame, where it stopped with "The following variables are missing from input data: o", and so is, by the spatial test, which reads no variable, a model that transforms its variables; the model frame given asdatais accepted as it is. A fit withmodel = FALSEwhose data are gone is tested ondatathat give its response and fitted values by the Harrison-McCabe test, and by the registered"szroeter"andrunHeteroTests(tests = "szroeter"), which stopped with "object 'gone' not found" (the other five tests tested it already). A fit withsubset =given by index gave its value already. - The registered
"szroeter"reads its fitted values from the fit, and givesdatato the test as it is: called withdata = NULLit reads the model frame, where it stopped with "Missing rows", and numbered data in the order of the fit whose numbers are not those of the fit, such as the complete rows of a fit withna.omitnumbered again, are taken in that order, where they were refused with "Missing rows". The fitted values are the linear predictor of the fit with its offset, whichfitted()holds: the numberspredict(type = "response")computed, so ties are broken as before, but on agamwith anoffsetargument (above). Data it matched by row name before the comparison of the guard are refused by the guard asperformSzroeterTest()refuses them, with that message. - Without
data, a variable the model frame does not hold under its own name is refused with "The following variables are missing from the model frame ofmodel: group. It holds the terms of the model (...)", which says to passdata; they stopped with "The following variables are missing from input data". Such a message no longer names the variables of anoffset()term, which the tests need no more. - Numbered data with one row per observation are taken in the order of the fit, and refused when the variable the test reads is not a column of the model frame and two rows of the fit tie on every compared column, as by the rank permutation test and Goldfeld-Quandt: Hartley's test on the tibble of a fit to repeated rows stops with that refusal where it stopped with "Missing rows".
- A multivariate
lmis refused with "... needs a model with one response;modelhas 2.", where they stopped with a coercion error. A missing value of the variable at a row of the fit is refused, as before, and the message names the variable. - A fit whose coefficients, residuals or weights were changed after it was
fitted is refused by the six, as by the first nine ("The fitted model
does not reproduce its residuals from its design and coefficients", "The
weights of the model are not those its coefficients were fitted with", of
class
heteroTests_model_context_error). They returned a value computed from the edited fit: onlm(mpg ~ wt + qsec, data = mtcars)with 0.5 added to the coefficient ofwt,performGlejserTest(m, mtcars, "wt")returned t = -0.395 (p = 0.695) andperformParkTest()t = -0.653 (p = 0.519); with a residual set to 0 by hand, astests/testthat/test-pass-a-reference.Rdid to make Park's test floor it,performParkTest()returned t = 0.911 (p = 0.368). That test now makes the residual zero with an observation that lies on the regression of the others. - A fit with a zero weight is refused by the six as before, with
data, without it and on shuffled rows, and through the registered names and the runner. Anmgcv::gam()fit with smooth terms is tested by the six as before, until the last step settles which tests take it. - Unchanged: every other value of a valid call, every refusal of the guard,
and every message the test suite pins. A corpus of 16,212 calls, the six,
their routes and every other test on 279 combinations of fits, offsets,
terms, forms of
data, dropped rows, row names and classes and on the five fits of the table of the Harrison-McCabe rank, the default battery and the registered names throughrunHeteroTests(), and the examples of the sixteen help pages, returns what it returned before,identical(), but for the 594 calls of the changes above (scripts/model-context-identity.R), eight of them the new refusals in bold; each of the 264 that now return a value, or another one, gives the value of its reference, compared on every number of the result: the same call on the data frame of the fit, on the fit to the complete rows, on the data of the fit in its order, on the samegamwith anoffset()term, through the registered spatial test, or, for the rank of the Harrison-McCabe test, the test written out from the fit's own decomposition. The rows of the six tests in Pass A, the Glejser study and the Harrison-McCabe study ofinst/validation/are reproduced byte for byte at the settings of the committed tables (5000 replications, and 100,000 samples for the null distribution of the Harrison-McCabe test). The warnings of the calls of the corpus are those of before, but for three calls of Hartley's test that stopped and now return a value with its warning on unequal groups.
The fourth step moves the five group tests, performLeveneTest(),
performBrownForsytheTest(), performFlignerKilleenTest(),
performBartlettTest() and performOBrienTest(), onto the model context.
Each matched data to the residuals with its own copy of the alignment, by
row name, or by position when the residuals carry no names; the copies are
gone. They read the Pearson residuals and the weights from the fit, and the
group at the rows of data the comparison confirmed, or, without data,
from the model frame when it holds the group under its own name. The
observations of the fit whose group is missing are left out with the warning
"Removed N observations due to missing values in group", as before, on the
same rows, so the statistic is the one of before. Data that are given are
matched as for the first fifteen: named data in any order, numbered data in
fit order. None of the five is registered with the runner, so the change
reaches their direct calls, and a diagnostic registered with
registerDiagnostic() that calls one, whose refusals by the context are final
in runHeteroTests(), as for the registered names. Every value is
identical() to before but for the changes in bold below.
A gam given its data in another order is matched to them by row name, by
the group tests too. The residuals of an mgcv::gam() fit carry no names,
and the five matched data to them by position, without an error: given the
rows of the fit in another order under their own row names, they put each
residual in the group of another row and returned a wrong value. They now
match the rows by the row names of the model frame of the fit, and give the
value on the data in the order of the fit. On contract_data(), with the data
of the fit shuffled, d[sample(120), ] after set.seed(3):
Call, on d[sample(120), ] |
gam(y ~ x1 + x2), 0.15.0 |
Now | gam(y ~ s(x1) + x2), 0.15.0 |
Now |
|---|---|---|---|---|
performLeveneTest(m, data, "group") |
1.373 | 3.459 | 1.167 | 2.352 |
performBrownForsytheTest(m, data, "group") |
1.132 | 3.180 | 1.029 | 2.107 |
performFlignerKilleenTest(m, data, "group") |
3.572 | 9.367 | 2.951 | 5.402 |
performBartlettTest(m, data, "group") |
4.548 | 12.380 | 3.401 | 9.916 |
performOBrienTest(m, data, "group") |
1.187 | 3.580 | 1.042 | 3.137 |
Each new value is identical() to that of the same call on d, which is
unchanged, and equals, to 1e-10, the test computed without the package on
the residuals of the fit and the groups of d (car::leveneTest() centred at
the means and at the medians, stats::fligner.test(),
stats::bartlett.test(), and O'Brien's transformation written out); so does
the value on a gam with an offset argument given its data shuffled. The
panel tests, the three streaming tests and the replicates of
performWhiteTestBootstrap() still match a gam by position until the steps
that move them; the roadmap records them.
A fit with model = FALSE whose data are gone, given numbered data in
another order than the fit, is refused by the five. As by the fifteen tests
of the first three steps: such data, a tibble or a data frame whose row names
were reset, in another order, cannot be matched to the observations of the fit
(named data in any order, numbered data in fit order), and the model frame
built from them does not give the response of the fit, so the five stop with
"... cannot use the model frame of model: it cannot be rebuilt (...), and
data does not give the response and the fitted values of the fit", of class
heteroTests_model_context_error. They returned a value, that of other rows:
on lm(y ~ x1 + x2, data = gone, model = FALSE) for the 120 observations of
contract_data(), with gone removed after the fit and the data given as
dplyr::as_tibble(d)[sample(120), ] after set.seed(3), Levene's test
returned 1.373 where the data in the order of the fit give 3.459, the
Brown-Forsythe test 1.132 for 3.180, the Fligner-Killeen test 3.572 for 9.367,
Bartlett's 4.548 for 12.380 and O'Brien's 1.187 for 3.580.
A fit whose coefficients, residuals or weights were changed after it was
fitted is refused by the five, as by the first fifteen ("The fitted model
does not reproduce its residuals from its design and coefficients", "The
weights of the model are not those its coefficients were fitted with", of
class heteroTests_model_context_error). They returned a value computed from
the edited fit: on lm(mpg ~ wt + qsec, data = mtcars) with 0.5 added to the
coefficient of wt, and cyl as a factor, performLeveneTest(m, mtcars,
"cyl") returned F = 3.276 (p = 0.0522), performBrownForsytheTest() F = 2.639
(p = 0.0886), performFlignerKilleenTest() 4.811 (p = 0.0902),
performBartlettTest() 5.838 (p = 0.0540) and performOBrienTest() F = 1.990
(p = 0.155), the values of the fit before the edit, whose residuals the edit
leaves unchanged.
Also, for these five:
- Errors that become values. Numbered data that hold only the rows a fit with
na.omitkept, numbered again, and the tibble in fit order of a fit to data with character row names, stopped with "Missing rows" although they hold every row of the fit, and give the value on the rows of the fit; a group missing at a row of such data is left out, with its warning, as on the data of the fit. A model of the group with anoffset()term or a transformed variable, such aslog(ypos) ~ log(x1) + group, stopped withoutdatawith "The following variables are missing from input data: o" (or the variables of its terms), and is tested on its model frame; the model frame given asdatais accepted as it is. Weights taken from a column of the data, and a fit withsubset =given by index, gave their values already. - Without
data, a group the model frame does not hold is refused with "The following variables are missing from the model frame ofmodel: group. It holds the terms of the model (...)", which says to passdata; they stopped with "The following variables are missing from input data", which also named the variables of the model the tests do not read:ypos,x1andg3forlog(ypos) ~ log(x1) + x2grouped by a variableg3the model does not use, where the message now namesg3alone. With the model frame of such a fit asdata, the message names the group alone. - A
groupthat isNAor empty is refused by the Levene, Brown-Forsythe, Fligner-Killeen and Bartlett tests with "groupmust be a single column name.", as O'Brien's test refused it, beforedatais checked; they stopped with "The following variables are missing from input data: NA". - Numbered data in another order than the fit, given for a fit to data with
character row names, such as its tibble shuffled, stopped with "Missing
rows: obs1, obs2, obs3", rows the data hold; they are refused by the guard,
as by the other tests built on the context, with "... cannot match the rows
of
datato the observations of the model: ... y, x1, x2 differ from the model frame", of classheteroTests_data_rows_error. - Numbered data with one row per observation are taken in the order of the fit, and refused when the group is not a column of the model frame and two rows of the fit tie on every compared column, as by Hartley's test: on the tibble of a fit to repeated rows the five stop with that refusal, where they stopped with "Missing rows".
- A multivariate
lmis refused with "... needs a model with one response;modelhas 2.", where they stopped with a coercion error, and a fit withmodel = FALSEwhose data are gone, withoutdata, with the message of the context, where they stopped with "Expected a data.frame for input data". - A fit with a zero weight is refused by the five as before, with
dataand on shuffled rows; withoutdata, or given its model frame, they name the group the model frame lacks first, as before. Anmgcv::gam()fit with smooth terms is tested by the five as before, until the last step settles which tests take it. - Unchanged: every other value of a valid call, the observations left out
for a missing group and their warning, every refusal of the guard, and
every message the test suite pins. A corpus of 19,523 calls, the five and
every other test on 337 combinations of fits, offsets, terms, forms of
data, dropped rows, missing groups, row names and classes and on the five fits of the table of the Harrison-McCabe rank, the default battery and the registered names throughrunHeteroTests(), and the examples of the twenty-one help pages, returns what it returned before,identical(), but for the 375 calls of the changes above (scripts/model-context-identity.R), 15 of them the new refusals in bold; each of the 70 that now return a value, or another one, gives the value of its reference, compared on every number of the result: the same call on the data frame of the fit, on the fit to the complete rows, on the data of the fit in its order, or of a model of the group on its data. The rows of the five in Pass B (125,000 p-values at 5000 replications), the full sweep and block 1 of the weighted fits (8,000 at 400) ofinst/validation/areidentical()to before, replication by replication, and reproduce the committed tables row for row, the table of Pass B byte for byte. The warnings of the calls of the corpus are those of before, but for five calls that stopped and now return a value with the warning on the missing group.
The fifth step moves the two panel tests, performPesaranTest() and
performBPRandomEffectsTest(), and so runPanelTests(), which runs them, and
performScatterDiagnostic() onto the model context. They read the Pearson
residuals and the weights from the fit, and the id and time of the panel
tests and the vars of the scatter diagnostic at the rows of data the
comparison confirmed, or, for the panel tests without data, from the model
frame when it holds them under their own names. Data that are given are
matched as for the first twenty tests: named data in any order, numbered data
in fit order. None of the four is registered with the runner, so the change
reaches their direct calls and runPanelTests(). Every value is
identical() to before but for the changes in bold below.
The panel tests and the scatter diagnostic match data by name. They
read the i-th row of data with the i-th residual, and since 0.15.0 refused
the data they could not show to be in that order (Data in another order than
the fit, under 0.15.0). They now match the rows as the other tests do, and
give the value on the rows of the fit. On the design of that entry, with
set.seed(2); dp <- d[sample(n), ]:
Call, on dp |
0.15.0 | Now, the value on d |
|---|---|---|
performPesaranTest(m, dp, "id", "time") |
Error: "... it reads data by position, the i-th row with the i-th observation, and the rows of data, matched by name, are not in the order of the fit." |
-0.987 (p = 0.324) |
performBPRandomEffectsTest(m, dp, "id") |
Error, the same | 1.187 (p = 0.276) |
performScatterDiagnostic(m, dp, "x") |
Error, the same | 0.468 |
runPanelTests(m, dp, "id", "time") |
Error, the same | 1.187 and -0.987 |
Before 0.15.0 the three returned -0.392, 0.506 and -0.080, values of other
rows. So do rows after those of the fit, rbind(d, d[1:5, ]), which they
refused for their number ("requires data with 120 observations to match the
fitted model, got 125"); a fit to a subset given by index, given its data; and
the tibble in the order of the fit of a fit to data with character row names,
on which they stopped with "Missing rows: obs1, obs2, obs3" or "the row names
of data are not those of the fit".
On a fit that dropped rows, the random-effects test and the scatter
diagnostic give the value of its observations. On the 120 observations of
contract_data() (tests/testthat/helper-test-recipes.R) with x2 missing in
rows 3, 7 and 11, m <- lm(y ~ x1 + x2, data = d) drops them, and
performBPRandomEffectsTest(m, d, "id") refused the full data with
"Breusch-Pagan random effects test requires data with 117 observations to
match the fitted model, got 120.", as did the scatter diagnostic and
runPanelTests(). They give the value of the fit to the complete rows: the
random-effects test 2.016 (p = 0.156), with its warning on unequal panel
lengths, and the scatter diagnostic 0.343 for x1 and -0.163 for x2, on the
full data, shuffled or as a tibble. Pesaran's test now reaches the panel of
such a fit, which is unbalanced: units 1 and 2 keep 8 and 9 of their 10
periods, and the test refuses it (below), where it stopped on the complete rows
with "number of items to replace is not a multiple of replacement length". On a
fit that drops every row of a unit, here the ten rows of unit 12, the panel
stays balanced: Pesaran's
test refused the full data with "requires data with 110 observations to match
the fitted model, got 120.", and gives -0.0607 (p = 0.952), the value of the fit
to the 110 complete rows, and the random-effects test 4.083 (p = 0.043).
Pesaran's test refuses an unbalanced panel (decided by the author on
2026-10-24). It arranges the residuals in a matrix of periods by units, and a
unit observed at fewer periods than the panel holds stopped it with "number of
items to replace is not a multiple of replacement length", or, when its number
of periods divides that of the panel, had its residuals recycled over the
periods it lacks, and the test returned a wrong value without an error. On the
120 observations of contract_data() without the first five rows, unit 1
keeps 5 of its 10 periods, and performPesaranTest(m, d, "id", "time")
returned 0.234 (p = 0.815), where plm::pcdtest(model = "pooling"), which
correlates each pair of units over the periods both hold (Pesaran 2004),
gives -0.373. The test now stops, on every form of data and through
runPanelTests(), with performPesaranTest() needs a balanced panel, every
unit observed at the same periods, and the panel of the fit is unbalanced: 1 of
its 12 units is observed at fewer of its 10 periods, id = 1 at 5 of them. ...,
which points to plm::pcdtest() for such a panel, of class
heteroTests_model_context_error. The full data of a fit that dropped such
rows, which it refused for their number of rows, are refused so too. A fit that
drops every row of a unit keeps a balanced panel, and is tested (above). The
refusal went when a later change computed the statistic of an unbalanced
panel from the periods each pair of units shares (Pesaran's test on an
unbalanced panel, above).
Pesaran's test refuses a pair of id and time that holds more than one
observation of the fit (decided 2026-10-18). It arranges the residuals in a
matrix of periods by units, and a repeated pair put one of its observations in
the cell of another period, without an error: on contract_data() with the
second observation of unit 1 moved to period 5, d$time[2] <- 5L, it returned
0.516 (p = 0.606), where the panel gives 0.264 (p = 0.792). It now stops with
performPesaranTest() needs one observation of the fit for each pair of `id`
and `time`, and 1 pair holds more than one: id = 1 and time = 5 (2
observations). ..., of class heteroTests_model_context_error, and so does
runPanelTests(), also on a fit with na.exclude (below), where it returned
0.253 with d$x2[50] <- NA. With the period of the first observation,
d$time[2] <- 1L, it returned the value of the panel by the chance of the
order of the rows, and now stops too. The random-effects test reads no period
and is unchanged.
A gam given its data in another order is matched to them by row name, by
the panel tests and the scatter diagnostic too. Their check of the order of
0.15.0 compared the row names of data with the names of the residuals,
which those of an mgcv::gam() fit lack, so they read such data by position
and returned a wrong value without an error. On contract_data(), with the
data of the fit shuffled, d[sample(120), ] after set.seed(3):
Call, on d[sample(120), ] |
gam(y ~ x1 + x2), 0.15.0 |
Now | gam(y ~ s(x1) + x2), 0.15.0 |
Now |
|---|---|---|---|---|
performPesaranTest(m, data, "id", "time") |
-0.570 | 0.264 | -0.456 | -0.036 |
performBPRandomEffectsTest(m, data, "id") |
0.163 | 3.100 | 0.070 | 4.311 |
performScatterDiagnostic(m, data, c("x1", "x2")) |
0.011, -0.119 | 0.332, -0.167 | -0.012, -0.067 | 0.308, -0.120 |
Each new value is identical() to that of the same call on d, which is
unchanged, and equals, to 1e-10, the test written out without the package
on the residuals of the fit and the rows of d that carry their names.
A fit with model = FALSE whose data are gone, given numbered data in
another order than the fit, is refused by the four, with "... cannot use
the model frame of model: it cannot be rebuilt (...), and data does not
give the response and the fitted values of the fit", of class
heteroTests_model_context_error, as by the first twenty. They returned the
values of other rows: on lm(y ~ x1 + x2, data = gone, model = FALSE) for the
120 observations of contract_data(), with gone removed after the fit and
the data given as dplyr::as_tibble(d)[sample(120), ] after set.seed(3),
Pesaran's test returned -0.570 where the data in the order of the fit give
0.264, the random-effects test 0.163 for 3.100 and the scatter diagnostic
0.011 for 0.332.
A fit whose coefficients, residuals or weights were changed after it was
fitted is refused by the four, as by the first twenty ("The fitted model
does not reproduce its residuals from its design and coefficients", "The
weights of the model are not those its coefficients were fitted with", of
class heteroTests_model_context_error). They returned the values of the fit
before the edit, whose residuals the edit leaves unchanged: on
lm(mpg ~ wt + qsec, data = mtcars) with 0.5 added to the coefficient of
wt, and units and periods mtcars$id <- rep(1:8, each = 4) and
mtcars$time <- rep(1:4, 8), Pesaran's test returned 0.830 (p = 0.407), the
random-effects test 16.70 (p = 4.4e-05) and the scatter diagnostic -0.141 for
wt and 0.345 for qsec.
Also, for these four:
- Errors that become values, besides those above: the complete rows of a fit
with
na.omitnumbered again, the tibble of such a fit and the tibble it was fitted on, which they refused with "Missing rows" or for their number of rows, give the value on the rows of the fit (Pesaran's test where that panel is balanced, and on an unbalanced one since Pesaran's test on an unbalanced panel, above), and so does a fit withmodel = FALSEwhose data are gone, given its rows shuffled under their own row names. - Without
data, the panel tests readidandtimefrom the model frame, when the model uses them under their own names, as before, and refuse them otherwise with "The following variables are missing from the model frame ofmodel: id, time. It holds the terms of the model (...)", which says to passdata, of classheteroTests_model_context_error; they stopped with "idandtimemust be columns indata." (or "idmust be a column indata."), which they still give for data that lack them. The scatter diagnostic andrunPanelTests()needdata, as before. - Numbered data with one row per observation are taken in the order of the
fit, and refused when the columns they read are not columns of the model
frame and two rows of the fit tie on every compared column, as by the tests
of the first four steps: on the tibble of a fit to repeated rows the panel
tests stop with that refusal, where they stopped with "Missing rows: 1.1,
2.1, 3.1"; the scatter diagnostic, which reads
x1andx2, columns of the frame, gives the value of the fit. - A multivariate
lmis refused with "... needs a model with one response;modelhas 2.", where they stopped on its number of rows, and a fit withmodel = FALSEwhose data are gone, withoutdata, with the message of the context, where the panel tests stopped with "datamust be a data frame.". An edited fit given its rows shuffled, or withoutdata, is refused with the message of the edit, where they refused the order or lackedid. - Pesaran's test refuses a unit or a period missing at an observation of the
fit with "Pesaran CD test requires
datato contain the rows used to fit the model. Missing rows: 5, where time, thetimevariable, is missing.", where it stopped with "number of items to replace is not a multiple of replacement length"; and a fit to repeated rows with the refusal of the repeated pairs, where it stopped on the unbalanced panel. The random-effects test leaves an observation whose unit is missing out of its sums by unit, as before, and takes an unbalanced panel, with its warning on unequal panel lengths, as before. The message "number of items to replace is not a multiple of replacement length" can no longer be reached. - A fit with
na.action = na.excludeis read as before, until the step of the model context that movesna.exclude(decided by the author on 2026-10-24): its residuals, padded withNAat the rows it dropped, against the rows ofdataby position, so the random-effects statistic and the correlations of the scatter diagnostic areNA, and data in another order are refused. The refusals of Pesaran's test above, of a missing unit or period, of a repeated pair and of an unbalanced panel, apply to the rows it reads so, the rows ofdata, those the fit dropped included: they do not see that the panel of the fit itself is unbalanced, and the statistic is computed as before, withNAresiduals at the rows the fit dropped, until that step. Withx2missing in rows 3, 7 and 11 ofcontract_data(), whose panel is balanced indatabut not in the fit, Pesaran's test returns 0.713 on such a fit, as before, whereplm::pcdtest()gives 0.189 on the rows of the fit and the same fit withna.omitwas refused (it gives 0.189 since Pesaran's test on an unbalanced panel, above). - The scatter diagnostic refuses a fit with a zero weight before it matches
data, as before; the panel tests refuse it after, as before. A model written with the columns of a data frame,lm(d$y ~ d$x + d$z), givend, is refused by the four with "The following variables are missing from input data: d.", as by the tests built on the context, where the scatter diagnostic and the panel tests said that no column of the model frame can be compared. - Unchanged: every other value of a valid call, every refusal of the guard,
and every message the test suite pins. A corpus of 22,542 calls, the four,
the options of
runPanelTests()and every other test on 356 combinations of fits, offsets, terms, forms ofdata, dropped rows, missing groups, row names, panels and classes, on the five fits of the table of the Harrison-McCabe rank and the eight designs of White's degrees of freedom, the default battery and the registered names throughrunHeteroTests(), and the examples of the twenty-five help pages, returns what it returned before,identical(), but for the 566 calls of the changes above (scripts/model-context-identity.R), 22 of them the new refusals in bold; each of the 352 that now return a value, or another one, gives the value of its reference, compared on every number of the result: the same call on the data frame of the fit, on the fit to the complete rows, on the data of agamin its order, or on the fit with the factor ofcut()computed beforehand. The rows of the two panel tests in the full sweep and block 1 of the weighted fits ofinst/validation/(2,400 p-values at 400 replications) areidentical()to before, replication by replication, and reproduce the committed tables. The warnings of the calls of the corpus are those of before, but for 54 calls on fits that dropped some rows of some units, on which the random-effects test now returns a value with its warning on unequal panel lengths, directly or throughrunPanelTests().
The sixth step moves the tests computed from the model alone onto the model
context: performSpearmanTest(), performHarveyTest(),
performArchLMTest(), performMcLeodLiTest(), performCookWeisbergTest()
and performNCVTest(), performSpreadLevelTest(),
performDavidianCarrollTest() and performVarianceFormTest(), with
fitWLS(), and so their registered names ("cook_weisberg", "ncv",
"spread_level", "variance_form"), runTimeSeriesTests(), and the
runners with them. They read the residuals, fitted values, design and
weights from the fit, as before, and nothing from data, which they now
match to the observations of the fit as every other test does, and so does
RESET (below). A second formula, the var_formula of the Cook-Weisberg test, of the variance
form test and of fitWLS(), and the against of the variance form test, is
evaluated on the rows of data the comparison confirmed, or, without
data, on the model frame. Every value is identical() to before but for
the changes in bold below.
The tests computed from the model alone match data as every other test
does (decision 31 of the design of the model context, decided by the
author on 2026-10-26: one rule for every test). The nine, RESET and
fitWLS() without var_formula read nothing from data, and only checked
that it held the variables of the model and the row names of the fit; RESET
kept that check through the second step. They now match data through the
guard, named data in any order and numbered data in the order of the fit, so
data the guard cannot match are refused, with its message and class
(heteroTests_data_rows_error), where they returned the value of the fit,
correct since they read nothing from data: numbered data in another order,
rows in front of those of the fit and repeated row names, 1,491 calls of the
corpus below. Numbered data in the order of the fit whose row numbers
are not the fit's, which the check of 0.15.0 refused, are taken. On mtcars
with its row names reset, d <- mtcars; rownames(d) <- NULL and
m <- lm(mpg ~ wt + qsec, data = d):
| Call | 0.15.0 | Now |
|---|---|---|
performSpearmanTest(m, dplyr::as_tibble(d)[sample(32), ]), after set.seed(1) |
1.445 (p = 0.159) | Error, the refusal of the guard: "... cannot match the rows of data to the observations of the model: data has no row names of its own ..." |
performSpearmanTest(m, rbind(d[1:5, ], d)) |
1.445 (p = 0.159) | Error, the same |
performSpearmanTest(m, dup), with dup <- rbind(d, d[1, ]) and attr(dup, "row.names") <- c(1:32, 1L) |
1.445 (p = 0.159) | Error: "... the row names of data are not unique ..." |
performSpearmanTest(mn, renumbered), the 30 complete rows of a fit that dropped rows 3 and 7, mn <- lm(mpg ~ wt + qsec, data = dn), numbered again |
Error: "Spearman rank requires data to contain the rows used to fit the model. Missing rows: 31, 32" |
0.951 (p = 0.35), the value on the full data dn |
The same holds for the other eight, RESET (6.731 on d) and fitWLS(),
which ignored data; named data in another order, d[sample(32), ], give the
value on d, as before. A data that is neither a data frame nor NULL is
refused by the nine and RESET as before, with the message "data in ... must
be a data frame or NULL." of each test, and now by fitWLS() too, with
"data in fitWLS() must be a data frame or NULL.": without var_formula it
ignored such a data, a list or a matrix, and returned the fit (with
var_formula it refused it, and keeps its message).
A second formula is evaluated on the rows of the fit. fitWLS() and the
variance form test matched the rows of data to the fit by row name, and,
when no name matched and the numbers of rows agreed, took them by position,
without a check: on a tibble in another order than the fit they evaluated the
variance formula on other rows and returned another value without an error.
They now match data as the other tests do, named data in any order and
numbered data in the order of the fit, and refuse other data with the
message and class of the guard (heteroTests_data_rows_error).
The Cook-Weisberg test with var_formula refused such data already. On
mtcars with its row names reset, d <- mtcars; rownames(d) <- NULL,
m <- lm(mpg ~ wt + qsec, data = d) and
tb <- dplyr::as_tibble(d)[sample(32), ] after set.seed(1):
Call, on tb |
0.15.0 | Now | The value on d |
|---|---|---|---|
coef(fitWLS(m, tb, var_formula = ~ hp)) |
22.01, -4.953, 0.8037 | Error: "fitWLS() cannot match the rows of data ..." |
16.38, -4.587, 1.028 |
performVarianceFormTest(m, tb, var_formula = ~ hp) |
0.0203 (p = 0.888) | Error, the same | 0.0631 (p = 0.803) |
performVarianceFormTest(m, tb, against = ~ hp) |
0.0890 (p = 0.768) | Error, the same | 0.00297 (p = 0.957) |
So are rows in front of those of the fit, rbind(d[1:5, ], d) (0.0105 with
var_formula = ~ hp), repeated row names, and the tibble of a fit to
repeated rows, whose rows tie on every column of the model frame, so that a
variable of the formula beyond it could belong to other rows. Named data in
another order, d[sample(32), ], give the value on d, as before.
A variable of a second formula is read from data or the model frame,
never from the workspace (decided on 2026-10-16). Without data, a
variable that the model frame did not hold under its own name was found in
the environment of the formula, whatever the order of its values; and
fitWLS() and the variance form test did so with data that lacked it too.
With hp2 <- rev(mtcars$hp) in the workspace and
m <- lm(mpg ~ wt + qsec, data = mtcars),
performCookWeisbergTest(m, var_formula = ~ hp2) returned 1.453
(p = 0.228), performVarianceFormTest(m, var_formula = ~ hp2) 4.374
(p = 0.045), performVarianceFormTest(m, against = ~ hp2) 0.266 (p = 0.61),
and fitWLS(m, var_formula = ~ hp2) a fit with the weights of hp2. They now
stop with "The following variables are missing from the model frame of
model: hp2. ... Variables that are not part of the model must be supplied
through data: those of var_formula are not read from the workspace.", of
class heteroTests_model_context_error, and with data that lack it with
"The following variables are missing from input data: hp2.", as the
Cook-Weisberg test did. The functions of the formula, such as log() or
I(), are found as before, a variable of the model is read from the model
frame, and a constant, a value of length one such as k <- 2 in
~ I(hp^k) or pi, is not a variable and is evaluated as R evaluates it, as
before: on lm(mpg ~ wt + hp, data = mtcars),
performCookWeisbergTest(m, var_formula = ~ I(hp^k)) gives 0.5496
(p = 0.458), the value of car::ncvTest(m, ~ I(hp^2)), and given mtcars
too, where it refused k with "The following variables are missing from
input data: k.".
A fit with model = FALSE whose data changed after the fit is refused by
the tests computed from the model alone and fitWLS(), with or without
data, with "... cannot use the model frame of model: the model was fitted
with model = FALSE, so its model frame is rebuilt from the data it was
fitted on, and the rebuilt frame no longer holds the response or the fitted
values of the fit ...", of class heteroTests_data_rows_error, as by every
other test built on the context and by RESET since the second step. They read
that frame, or ignored it, and returned a value: with
d0 <- mtcars; m0 <- lm(mpg ~ wt + qsec, data = d0, model = FALSE) and then
d0 <- d0[order(d0$wt), ], the variance form test returned 5.817
(p = 0.0035) where the fit gives 1.765 (p = 0.179), Harvey's test, on the
design of the reordered data, 3.041 (p = 0.219) for 2.446 (p = 0.294), and
fitWLS(m0) refitted the reordered response, with coefficients 28.42, -5.863
and 0.594 for 14.08, -4.728 and 1.194; the others gave the value of the fit,
which their statistic does not take from the frame (Spearman's test 1.445,
the Cook-Weisberg test 0.591).
A fit whose coefficients, residuals or weights were changed after it was
fitted is refused by the nine tests, as by the first twenty-four ("The
fitted model does not reproduce its residuals from its design and
coefficients", "The weights of the model are not those its coefficients were
fitted with", of class heteroTests_model_context_error). They returned the
values of the fit before the edit: on lm(mpg ~ wt + qsec, data = mtcars)
with 0.5 added to the coefficient of wt, Spearman's test 1.445
(p = 0.159), Harvey's 2.446, the ARCH LM test 3.457 with lags = 2,
McLeod-Li 10.74 with lags = 5, Cook-Weisberg 0.591, the spread-level test 0.438, Davidian-Carroll 1.746 and
the variance form test 1.765. fitWLS(), which is not a test, refits such a
fit as before.
A fit with model = FALSE whose data are gone is refused without data,
with the message of the context, "... cannot use the model frame of
model: it cannot be rebuilt (...)", of class
heteroTests_model_context_error: the Cook-Weisberg, NCV, spread-level
and Davidian-Carroll tests returned the value of the fit (the Cook-Weisberg
test 0.591 on mtcars), which nothing checked. Given numbered data in another
order than the fit, such as a shuffled tibble, they are refused with the same
message, as the first twenty-four are: those data do not give the response
of the fit.
performDavidianCarrollTest() refuses a multivariate lm with "...
needs a model with one response; model has 2.": without data it returned
2.064 (p = 0.136) on lm(cbind(mpg, qsec) ~ wt + hp, data = mtcars), a
statistic of neither response. The other eight and fitWLS() stopped on
such a fit with a coercion error ("'length = 4' in coercion to
'logical(1)'"), "incompatible dimensions" or the count of its rows, and now
give that message.
Also, for these:
- Errors that become values: a fit with
model = FALSEwhose data are gone, givendatathat give its response and fitted values, in the order of the fit or under their own row names in another order, is tested on them: Spearman's, Harvey's, the ARCH LM, McLeod-Li and variance form tests stopped with "Input data must be a data.frame", RESET refused it, andfitWLS()stopped with "object 'gone' not found"; onmtcars, Harvey's test gives 2.446 and RESET 6.731 (p = 0.0043), the values of the fit with its model frame. The nine and RESET, with or without a second formula, give the value of the fit on numbered data with one row per observation in the order of the fit, such as the complete rows of a fit that dropped rows numbered again, the tibble of a fit to data with character row names or of a fit to repeated rows, which they refused with "Missing rows" (above), and on the data of anmgcv::gam()fit that dropped rows, which they refused for their number of rows. - Values that change: the Cook-Weisberg test with
var_formulaon anmgcv::gam()fit given its data in another order under their own row names: the residuals of agamcarry no names, and it read such data by position (N17); it matches them by the row names of the model frame of thegam, and gives the value of the data in the order of the fit. - Messages: without
data, a variable of a second formula that the model frame does not hold under its own name,zforlm(y ~ x1 + x2)orx1forlm(log(y) ~ log(x1) + x2), is refused with the message above, where the formula stopped with "object 'z' not found" or "var_formulacould not be evaluated (...)"; with the model frame asdata, with "The following variables are missing from input data"; with a second formula, data the guard refuses are refused before the model is checked, so that on such data a weighted fit, which the variance form test refuses, or a fit with a zero weight, which the Cook-Weisberg test refuses, meets the refusal of the guard first; repeated row names and the tibble of a fit to repeated rows are refused by the guard, where the variance form test said "Variance form test requiresdatato contain the rows used to fit the model. Missing rows: ..."; a multivariatelmand a fit withmodel = FALSEwhose data are gone are refused with the messages of the context; and the Cook-Weisberg test withvar_formularefuses a variable of the formula that is missing at an observation of the fit with "Cook-Weisberg test requiresdatato contain the rows used to fit the model. Missing rows: 5, where z, thevar_formulavariable, is missing.", with the warning "Removed 1 observations due to missing values in z" of before, where it said "Missing rows: 5". - Unchanged: every other value of a valid call, every refusal of the guard, the
zero-weight refusals and the acceptance of a zero weight by RESET, a smooth
gamtested as before by the nine, and every message the test suite pins. A corpus of 30,344 calls, the nine,fitWLS(), their second formulas, with variables and with constants, registered names andrunTimeSeriesTests(), and every other test, on 361 combinations of fits, offsets, terms, forms ofdata, dropped rows, missing groups and variables, row names, panels and classes, on the five fits of the table of the Harrison-McCabe rank and the eight designs of White's degrees of freedom, the default battery and the registered names throughrunHeteroTests(), and the examples of the thirty-six help pages, returns what it returned before,identical(), but for the 4,090 calls of the changes above (scripts/model-context-identity.R), 2,674 of them the new refusals in bold, 1,491 of those of data the guard refuses (decision 31) and 576 with a variable of the workspace; each of the 350 that now return a value, or another one, gives the value of its reference, compared on every number of the result: the same call on the data frame of the fit, on the fit to the complete rows, or on the data of agamin its order, the same formula with the number in place of a constant, or the test a route computes. The warnings of the calls of the corpus are those of before.
The caches¶
The caches of cachedTest() and of runHeteroTests(use_cache = TRUE), its
default, and the cache of the checks the tests make first, follow a contract
decided on 2026-10-21, set out on ?cachedTest and in the Architecture
page. The changes in bold change what calls return.
A cached result is the result of its own call. The two caches kept a
result under a key of the test, the coefficients and the residuals of the
model and data, and fits that shared those shared one entry: the second
call returned the result of the first. lm(mpg ~ wt + offset(qsec), data =
mtcars) and lm(I(mpg - qsec) ~ wt, data = mtcars) have the same
coefficients and residuals to the last bit and other fitted values; after
the first, cachedTest("cook_weisberg", <second>, mtcars) returned 0.747
(p = 0.388) in place of its own 0.0728 (p = 0.787), "spread_level" 0.650
in place of 1.413 and "szroeter" 1.111 in place of 0.673, and
runHeteroTests() returned the battery of the first fit with the first fit
as attr(, "model"). A fit whose weights were changed after the fit, which
the tests refuse ("The weights of the model are not those its coefficients
were fitted with"), returned the result of the fit before the change
(Koenker's 0.481 for lm(mpg ~ wt, data = mtcars, weights = 1 / wt) with
its weights reversed), and so did a fit with model = FALSE whose data
changed after the fit, which the tests refuse too. The key now holds the
test and its arguments and what the test reads of the model and the data:
the class, family and link of the model; from the model context of the fit,
its model frame, response, design, residuals and fitted values, prior
weights, total offset, rank, aliased coefficients, the names of its
observations and its na.action; its coefficients, the weights it stores,
its terms with their predvars, call, contrasts, levels, residual degrees
of freedom and attributes, and whether it keeps its model frame; and
data, whole. Each such call computes its own result. The key leaves out
the environment of the formula and the options of the session, which
?cachedTest sets out.
The checks of the model are those of the model. The checks the tests
make first are kept in a cache whose key held the class, the coefficients,
the residuals and the residual degrees of freedom of the model. A fit that
shares those with another and has other fitted values passed, on a hit, the
refusal of a fit with an R-squared above 0.999 that it meets alone. With
set.seed(5); x <- runif(80, 1, 5); e <- residuals(lm(rnorm(80) ~ x)), the
fits ma <- lm(y ~ x, data.frame(y = 1 + 2 * x + e, x = x)) and
mc <- lm(y ~ x, data.frame(y = 1 + 200 * x + e, x = 100 * x)) have the
same coefficients and residuals to rounding, and the same to the last bit
once mc[c("coefficients", "residuals")] <- ma[c("coefficients",
"residuals")]; after performKoenkerTest(ma), 2.047 (p = 0.153),
performKoenkerTest(mc) returned the same value, where alone it stops with
"Model appears perfectly explained (R² = 1.000)", as it now does after
ma too. The key holds the fitted values, the weights, the prior weights,
the offset, the family, the link, the rank and the intercept too. The checks
of data sorted in their key the names they were given, which drops NA:
after a check of an NA name, refused, every check of the same columns with
no names was answered with its refusal. The internal check
rvalidateDataInputs() shows it: after
rvalidateDataInputs(mtcars, required_vars = NA, min_obs = 0),
performKoenkerTest(lm(mpg ~ wt + qsec, data = mtcars), mtcars) stopped
with "The following variables are missing from input data: NA"; it returns
3.086 (p = 0.214). Since the review of increment 4 (#143) the group tests
refuse an NA group before that check. The key keeps the names as they are
given, NA, empty names and their order included, which the message on the
missing ones keeps; and the checks of the sample size and of the
requirements of a test are keyed on the name of the test as it is given,
which they report, rather than in lower case.
A test that draws random numbers is never cached. The package flags each
test it registers: "wild_bootstrap", "white_bootstrap",
"rank_permutation" and "spatial_hetero" draw them, and run on every
call, through cachedTest(), runHeteroTests(), the registry and
runHeteroTestsParallel() alike, and a battery that includes one is not
stored as a whole. Before, a call with another seed returned the replicates
of the first, and a call answered from the cache drew no random numbers, so
that those drawn after it were not those of a call without the cache: with
m <- lm(mpg ~ wt + qsec, data = mtcars), set.seed(2);
cachedTest("wild_bootstrap", m, mtcars) after the same call from
set.seed(1), p = 0.233, returned that p-value; it returns its own, 0.238.
A seeded call now returns what it returns without the cache and leaves the
generator where it leaves it, which tests/testthat/test-cache-contract.R
checks for each of the four, with a draw after each call. A diagnostic added
with registerDiagnostic() or test_factory$register(), or a name of the
package registered again, is not cached either, since what it draws or reads
cannot be known; "box_m", which tests data, neither. A result whose
computation moved .Random.seed is not stored.
The check of normality draws its sample on every call. The check that
performBartlettTest() makes of its residuals tests a random sample of
5,000 values of a longer vector, and its result was kept in the validation
cache: a second identical call drew no random numbers, and the stream after
it was not that of a call without the cache. It is no longer stored. With
set.seed(1); d <- data.frame(x = runif(6000), g = factor(rep(c("a", "b",
"c"), 2000))); d$y <- 1 + d$x + rnorm(6000) and m <- lm(y ~ x, data =
d), two calls performBartlettTest(m, d, group = "g") from set.seed(2)
left runif(1) at 0.588, and leave it at 0.963, as two calls without the
cache did.
runHeteroTests() stores no call that fell back on a diagnostic the
caches may not hold. When a test fails, the runner falls back on another,
which can be a diagnostic a user registered under the name of a fallback, or
a resampling test, and the call was stored with its result: after
registerDiagnostic("breusch_pagan", <a function that counts its calls>),
runHeteroTests(m, d, tests = "white") on a fit of 15 observations, which
White's test refuses, returned the first count on every call. It returns a
new one. A fallback of the package, such as White's test without its cross
products, may be kept (found by the review of #144).
runHeteroTests(use_cache = TRUE), the default, returns the suite of
use_cache = FALSE. A result computed through the cache lacked the mark
attr(, "extras")$streaming = FALSE that the route without it adds; the two
now return identical() suites, on a first call and on one answered from
the cache.
The names of the estimates of performQuantileRegressionTest() do not
depend on the options. They were "tau_" and format(taus), so that
options(digits = 1) named c(0.25, 0.75) "tau_0.2" and "tau_0.8", and
a result read from the cache kept the names of the call that stored it. They
are formatted with the digits, the penalty for scientific notation and the
decimal mark of R's defaults: "tau_0.25" and "tau_0.75" whatever the
options, the names of the default options before.
- Each cache is bounded and drops the entry used least recently: those of
cachedTest()and of whole calls ofrunHeteroTests()hold at mostgetOption("heteroTests.cache_size", 100)results each, the validation cachegetOption("heteroTests.validation_cache_size", 1000), which it held since #97 but by overwriting the entry stored first.0turns a cache off. The two caches of results grew without bound, by about 33 KB per new model forrunHeteroTests()and 25 KB per call forcachedTest(); an entry now keeps the result without the model it carries (3.4 KB for White's test onmtcars, 31 KB with it), and the model of the call is put back on a hit. The keys were the names of variables of an environment, which R never frees, and each left 0.2 to 0.3 KB afterclearTestCache()andclearAnalysisCache(): a call ofrunHeteroTests()left three, 0.6 KB on a model of two regressors, the 0.8 KB per call that long simulations kept. The keys are a character vector now, and a cleared cache leaves nothing. - The key holds values, not environments: a model refitted to the same data
in another environment is answered from the cache, with the model of the
call as
attr(, "model"),identical()to the result the test computes. It is built from the model context withoutdata, and hashed with xxhash64 once per call ofrunHeteroTests(), where 0.15.0 hashed the residuals and the data once for the call and once for each test: about 0.7 ms on a fit of 1,000 observations, 2.3 ms on 10,000 and 20 ms on 100,000, against 0.2, 1.5 and 15 ms for the key of 0.15.0 and 2.6, 8.4 and 80 ms for the Cook-Weisberg test. runHeteroTests(use_cache = TRUE)on a Gaussianglmwith a zero prior weight, which the tests refuse, no longer warns "observations with zero weight not used for calculating dispersion": the warning came from the key of 0.15.0, which calledsigma()on the model. A fit withmodel = FALSEwhose data are gone, whose model frame the tests rebuild fromdata, runs without the cache, since its key cannot be built withoutdata.- Unchanged: every value computed without the cache, and every value read
from it on the call that stored it. On the 342 cells of the matrix of the
model context and of the rank of the Harrison-McCabe test, 16,524 records
(the default battery of
runHeteroTests()andcachedTest()with each of the eleven deterministic tests, without the cache, on empty caches and once more, and the three resampling tests on twelve cells) areidentical()to those of 0.15.0, value, warnings and the random numbers drawn after the call, but for four groups. The batteries run with the cache carry the markextras$streaming(above) in the 214 cells where no test fell back or stopped, and all 1,026 of them, on empty caches, once more and in a pass over all the cells that never clears the caches, are the battery without the cache, value and numbers drawn after it. In that pass, 558 calls (63 batteries, 495 single tests) that 0.15.0 answered with the entry of another cell, such as a fit with anoffset()term with that of the same fit with anoffsetargument, or a fit weighted by a vector of the workspace with that of the same fit weighted by a column of the data, now compute their own value, which equals the call without the cache. 26 resampling calls repeated from the same seed, which 0.15.0 answered from the cache, draw their replicates again, and the numbers drawn after them are those of a call without the cache. And the warnings of the five cells of the zero prior weight and of two calls of the quantile regression test on a fit withmodel = FALSEwhose data are gone (above). None of the 12,384 calls answered from the cache, or run with it, returns a value other than the same call without it; a call answered from the cache gives none of the warnings of the call that stored its result, as before.
White's degrees of freedom¶
White's test takes its degrees of freedom from the rank of its auxiliary
regression with the intercept, and its p-value changes where the auxiliary
columns span the constant. The auxiliary regression of the squared
residuals on the regressors, their squares and their cross-products, Z, has
an intercept, so the degrees of freedom of n R² are the rank of [1, Z]
minus one, as lmtest::bptest() counts them given the same terms as
varformula. performWhiteTest() took the rank of Z alone, one more when
the columns of Z span the constant, as the squares of the dummies of
contr.sum, contr.helmert and contr.poly contrasts, the last the default
for an ordered factor, the dummies of every level of a factor in a model
without an intercept, an interaction with no observation in the cell of its
baseline levels, and the squares of an orthogonal poly() basis do. The
statistic is the same, and the p-value was too large. It now drops the
columns of Z that are combinations of the intercept and the columns before
them, at the default tolerance of qr(), so that the regression and its
degrees of freedom agree. On the data of contract_data()
(tests/testthat/helper-test-recipes.R), with half a factor of two levels,
h one of three, sub one of two within each level of group, and o4 and
o3 the levels of group and of h as ordered factors:
| Model | Statistic | Degrees of freedom (p), 0.15.0 | Now, as lmtest::bptest() |
|---|---|---|---|
lm(y ~ x1 + group), contr.sum |
17.83 | 9 (p = 0.0372) | 8 (p = 0.0225) |
lm(y ~ x1 + group), contr.helmert |
17.83 | 9 (p = 0.0372) | 8 (p = 0.0225) |
lm(y ~ 0 + group + x1) |
17.83 | 9 (p = 0.0372) | 8 (p = 0.0225) |
lm(y ~ x1 + o4), contr.poly |
17.83 | 9 (p = 0.0372) | 8 (p = 0.0225) |
lm(y ~ x1 + group), treatment contrasts |
17.83 | 8 (p = 0.0225) | unchanged |
lm(y ~ x1 + group), contr.sum, cross_products = FALSE |
13.79 | 6 (p = 0.0321) | 5 (p = 0.0170) |
lm(y ~ x1 + group, weights = 1 / x1), contr.sum |
16.28 | 9 (p = 0.0612) | 8 (p = 0.0385) |
lm(y ~ x1 + half), contr.sum |
6.191 | 5 (p = 0.288) | 4 (p = 0.185) |
lm(y ~ x1 + group / sub), contr.helmert for both |
19.41 | 17 (p = 0.306) | 16 (p = 0.248) |
lm(y ~ x1 + half * h), no observation at (even, p) |
17.68 | 11 (p = 0.0893) | 10 (p = 0.0606) |
lm(y ~ poly(x1, 2) + x2) |
11.95 | 9 (p = 0.216) | 8 (p = 0.153) |
lm(y ~ o3), contr.poly |
1.696 | 3 (p = 0.638) | 2 (p = 0.428) |
lm(y ~ x1 + x2) |
11.14 | 5 (p = 0.0486) | unchanged |
The first five rows are one model: the four levels of group, coded in four
ways or without the intercept, span the same columns, and under the default
treatment contrasts the test gave the 8 degrees of freedom it now gives in
all five. The correction reaches cross_products = FALSE and more regressors
than max_interactions, performWhiteTestRobust(), whose asymptotic
interval, effect size and power follow the degrees of freedom, the registered
"white" and its fallback without cross products, runHeteroTests() and the
functions that share its default battery, and test_factory.
performWhiteTestBootstrap() does not change on such a design: its p-value
counts the replicates whose statistic reaches the observed one, and there no
statistic changes. Nor does performWhiteTestStreaming(): it stops on such
a design, whose auxiliary columns are collinear with its intercept, as on any
design with collinear auxiliary terms, and where it gives a value its
degrees of freedom were and are those of the exact test.
On a nearly collinear auxiliary design the statistic changes too. The
columns kept are now those lm(e^2 ~ Z) keeps. On the
mgcv::gam(y ~ s(x1) + x2) fit of the matrix of the model context
(tests/testthat/helper-model-context.R), whose 65 auxiliary columns have
rank 60 at the tolerance of qr(), the auxiliary regression of 0.15.0
silently lost a column: of the 60 columns its decomposition without the
intercept kept, lm() aliased one given the intercept, so the regression
had 59 regressors and was scored on 60 degrees of freedom. The test gave
45.92 on 60 degrees of freedom (p = 0.910); it now gives 46.05 on 60
(p = 0.908), 0.28% more, the value of lm(e^2 ~ Z) and of
lmtest::bptest() with the same terms. No computation keeps the statistic
of 0.15.0 and gives the value of those references: its columns without the
lost one give 45.92 on 59, which is neither.
White's test no longer depends on where its regressors are located. The
rank of [1, Z] was decided on the columns as they are: the square of a
regressor far from zero, such as x = 1e4 + u or a date written as
YYYYMMDD, differs from a combination of the intercept and the regressor by
less than the tolerance of qr() relative to its size, and was dropped. On
lm(y ~ x1 + x2) with 1e4 added to x1, on the data of contract_data(),
the test gave 10.34 on 4 degrees of freedom (p = 0.035), where the same model
at zero gives 11.14 on 5 (p = 0.0486); on a date, lm(y ~ ymd + z) with
ymd from 20240101 to 20240128, 42.049245 on 4 (p = 1.6e-08) for 42.049252
on 5 (p = 5.8e-08), the value of the days counted from zero. The rank is now
decided on the auxiliary design of the centred regressors, which spans the
same columns with the intercept. The regression is fitted on Z as before,
which keeps every other value bit for bit, and on the centred design only
when the fit of Z would lose one of the columns kept. The test gives the
value of lm(e^2 ~ Z_c) and of lmtest::bptest() on the centred terms, and
moving x1 by 1e2, 1e4 or 1e6 changes its statistic by at most 1.3e-10
relative and its degrees of freedom not at all. The robust form, the
registered "white", the runner and the White bootstrap change with it.
This holds for regressors that enter the model linearly, which are centred
one by one. A product or a power of a variable located far from zero inside
the model stays ill-conditioned once its columns are centred, and can still
lose a column; how far is far depends on the spread of the variable. On 200
observations of two standard normal regressors,
lm(y ~ I(x1 + 2e3) * I(x2 + 2e2)), with x1 about 2,000 standard
deviations from zero, keeps every coefficient, and White's test gives 46.83
on 7 degrees of freedom where lm(y ~ x1 * x2), the same model, gives 51.90
on 8; lm(y ~ I(x1 + 2e3) + I((x1 + 2e3)^2)) gives 8.05 on 3 for 11.29 on
4. The degrees of freedom held, and the statistic moved by at most 3e-5
relative, while the shift was at most 700 standard deviations of the
regressor, and the column was lost from 800 to 2,000 on (recorded in the
roadmap). Further out, lm() itself aliases the interaction or the square,
before White's test sees it: with I(x1 + 1e4) * I(x2 + 1e3) the fit is
additive, and the test gives 35.02 on 5, the value of lm(y ~ x1 + x2). A
year from 1990 to 2020, about 220 standard deviations from zero, and its
square, year * z and income * age give the value of the centred model to
seven digits or more.
White's test refuses an auxiliary regression that fits the squared
residuals exactly, where it returned the number of observations whatever the
data. Such a regression has an R² of 1, and n R² is n. It fits them
exactly with as many independent terms as observations, and with fewer when
the squared residuals are a combination of its terms, as on a paired design:
on lm(y ~ treat + subject) with two observations of each of 30 subjects,
the two residuals of a subject sum to zero, so their squares are equal and a
combination of the subject dummies, and every route to White's test returned
60 on 30 degrees of freedom (p = 0.0009); the runner rejected at the 5% level
in 200 of 200 homoscedastic draws. On 22 observations of six regressors,
whose squares and products make 28 terms with the intercept,
performWhiteTest() returned 22 on 21 degrees of freedom (on 22 in 0.15.0;
p = 0.40). The test now stops when the residual sum of squares of its
auxiliary regression is at most sqrt(.Machine$double.eps), 1.5e-8, of the
total: an exact fit leaves rounding error, about 1e-30 of the total on the
paired design, and squared residuals that depart from the fit by 2e-4 of
their size leave 2e-7, and are tested. It stops with "White's auxiliary
regression fits the squared residuals exactly: ...", and the advice depends
on the call: cross_products = FALSE when the products were in and the
regressors alone do not fit the squared residuals exactly, Koenker's test
(performKoenkerTest()) when they do not, and more data when the regression
has as many terms as observations ("There are too few observations for
White's test with these regressors; use ..."). With fewer terms the design
forces the exact fit, and more data of the same design do not help: 500
subjects of two observations are refused as 30 are, with "The design forces
the exact fit, as a factor with two observations per level does, however
many levels it has; use more observations per level." So
do its robust form, the registered "white", the White bootstrap and
performWhiteTestStreaming(), which stopped on the saturated design with
"Auxiliary regression became rank deficient". runHeteroTests() falls back
on White's test without cross products, as on any refusal of White's test,
and returns it under the name white, tagged as a fallback: 15.98 on 12
degrees of freedom (p = 0.19) for the six regressors. Where the regressors
alone fit the squared residuals exactly, as on the paired design, the
Breusch-Pagan and Koenker tests it would fall back on next fit them exactly
too, and would report the number of observations under the name white: it
takes no fallback and stops with the refusal, and so does runDiagnostics().
A factor of 10 levels with two observations each beside x1, 20
observations, shows why the refusal of a saturated regression alone was not
enough: with the products it has as many terms as observations, and the
runner fell back on White's test without them, which fits the squared
residuals exactly too, and returned 20 on 11 (p = 0.045), where it returned
20 on 19 (p = 0.39; on 20 in 0.15.0) before; it now stops. Koenker's and the Breusch-Pagan
tests return 60 on 30 and 63.25 on 30 on the paired design, a meaningless
value, as before; refusing it there is left to a change of their own
(recorded in the roadmap).
The streaming test counts the terms of its auxiliary regression by their
rank and fits them centred, from its cross-products, as performWhiteTest()
counts and fits them, so it refuses the same designs with the same message
and advice, and runHeteroTests() falls back, or stops, as it does without
streaming. On 30 pairs the squares of the subject dummies equal the dummies:
it refuses 31 independent terms for 60 observations as an exact fit the
design forces, where 0.15.0 stopped on the collinear terms. With one subject
observed a third time there is no exact fit, and it still stops on the
collinear terms, which performWhiteTest() drops and it does not; the
runner, when it streams, falls back there on White's test without cross
products, as in 0.15.0.
The refusal covers the exact fit only: one step away from the paired
design, White's test still rejects almost always. White's, Koenker's and the
Breusch-Pagan tests are not valid with fixed effects for small groups, a
factor with few observations per level, and over-reject, the more the
smaller the groups and the more groups there are, as they did before this
change. In 2,000 homoscedastic draws of lm(y ~ x + subject), with normal
x and errors, White's test rejected at the 5% level in 99% of them with 30
subjects of two observations and one of them observed a third time, in all
but one with 30 pairs whose weights vary within pairs, in 48%, 22%, 16% and
12% with 30 subjects of three, four, five and eight observations, and in 22%
with 100 subjects of eight; Koenker's test in 99%, 100%, 54%, 22%, 16%, 12%
and 22%, and the Breusch-Pagan test in 64%, 67%, 38%, 26%, 21%, 15% and 24%.
The help pages of the three tests say so, and ?performWhiteTest cites Juhl
and Sosa-Escudero (2014), who derive tests for fixed effects models. The
three tests also warn on such a fit (Fixed effects for small groups,
above).
A regressor named like a column White's test generates, such as a beside
a_sq, stopped the test with "Auxiliary regression failed: duplicated name
'a_sq' in data frame using '.'". The auxiliary regression is now fitted with
lm.fit() on its matrix, and the test gives the value of the same data with
other names: 27.64 on 5 degrees of freedom for the design of
tests/testthat/test-performWhiteTest.R. On every other design the fit
gives the value of before, bit for bit.
tests/testthat/test-performWhiteTest.R compares White's test on seventeen
designs, with and without cross products, with lm(e^2 ~ Z) written out on
the centred columns and with lmtest::bptest(), and pins the values above. Unchanged: every other
value, as on continuous regressors and factors with treatment contrasts,
where the auxiliary regression of 0.15.0 had the degrees of freedom it
counted. A corpus of 19,523 calls, that of the model context
(scripts/model-context-identity.R), returns what it returned before,
identical(), but for 44 calls of White's test, its robust form and the
runner on the fits with a poly() term and the gam with a smooth term,
which the harness allows as N2 and verifies, each, against White's test
written out on the cell to 1e-10, with the statistic identical() to
before on the poly() fits; a corpus of 504 calls of every route to White's
test on 28 designs, identical() but for 204 calls on designs that span the
constant, each with the statistic identical() and the degrees of freedom
and p-value of lm(e^2 ~ Z) and of lmtest::bptest(). The studies of
inst/validation/ fit continuous regressors, whose auxiliary columns do not
span the constant, and their rows of White's tests do not move: every value
of White's statistic, degrees of freedom and p-value they compute is
identical() to before, replication by replication, 280,593 in all. Pass A
at its 5000 replications, and White's row of the full sweep and its nine
rows of blocks 1 to 3 of the weighted fits, run alone from their seeds at
the replications of the committed tables, reproduce those tables row for
row; the White bootstrap's scenarios of block 4 of the weighted fits, of the
offset refits and on a transformed response, and the survey designs, give
the same tables as before at fewer replications from the same seeds.
The changes after the review of N2 (the location, the exact fits, the names)
were checked the same way. The corpus of the model context runs eight more
cells: its model with x1 moved by 1e4 and by 1e6, the six regressors on 22
observations above, the paired design, the factor of 10 levels beside x1,
the regressors a and a_sq with a copy that names the second b, and an
ordered factor. Of its 19,979 calls, 19,946 return what they returned
before, identical(), and the other 33 are calls of White's test on those
cells and of runDiagnostics() on the two whose regressors fit the squared
residuals exactly, each verified: against White's test written out on the
centred design, every other number of the result that of before or computed
from the new degrees of freedom; by the refusal, which the harness requires
of every route to White's test on the three exact fits, and of the runner
and runDiagnostics() where the regressors fit them exactly; or as the same
call on the model at zero or with the other names. The harness now also compares the
robust test's interval, effect size and power, and every attribute of the
result, with what they compute to from the degrees of freedom.
The studies of inst/validation/ fit regressors near zero, and their rows
of White's tests do not move: White's row of the full sweep and its nine
rows of blocks 1 to 3 of the weighted fits, run alone from their seeds at
the replications of the committed tables, reproduce those tables row for
row; Pass A at 500 replications, the survey designs at 100, the White
bootstrap on a transformed response at 60 and 20, the White bootstrap of the
offset refits at 60 and the White bootstrap's scenarios of block 4 of the
weighted fits at 40 (8 for the weights that are too weak), from the same
seeds, give the same tables as before; and each of the 131,433 values of
White's statistic, degrees of freedom and p-value these runs compute is
identical() to before.
Printing and summarising the results of runHeteroTests()¶
runHeteroTests() returns a hetero_test_suite, which had no print or
summary method: print() showed the list, each test as an htest and then
the attributes of the suite, the whole fitted model with its call and
coefficients among them, and summary() gave the Length Class Mode table
of a list. print() now shows the call of the model, or its formula, and the
number of observations on one line, wrapped when it is wider than the
console, then a table with one row per test, its statistic, degrees of freedom
and p-value, with the significance codes of summary.lm(), and a note, cut to
one line, for each test that did not run as asked. summary() returns a
summary.hetero_test_suite, laid out as the summary of an lm: the call, the
number of observations, the weights, offset and family where there are any,
the table, printed by printCoefmat() so that digits and
options(show.signif.stars) apply, the legend of the codes, the notes in full
(a fallback, naming the test that ran in its place and why; a refusal or a
failure, with its message; a streaming implementation; a parameter that is
not a degree of freedom, such as the B of a resampling test), and a line
saying that several tests of one model are not independent evidence.
coef(summary(s)) returns the table as a numeric matrix, as
coef(summary(fit)) does for an lm, and ?summary.hetero_test_suite
documents every field. A grouped run, a hetero_grouped_suite, has the same
methods, with one table for all the groups, in which a failed test shows NA
and the notes say why. On lm(stations ~ mag + depth, data = quakes), before:
runHeteroTests(model, quakes)
#> $white
#>
#> White's test for heteroscedasticity
#>
#> data: model
#> X-squared = 125.56, df = 5, p-value < 2.2e-16
#> alternative hypothesis: heteroscedasticity present
#>
#>
#> $koenker
#> ...
#> attr(,"model")
#>
#> Call:
#> lm(formula = stations ~ mag + depth, data = quakes)
#>
#> Coefficients:
#> (Intercept) mag depth
#> -192.04290 47.90872 0.01318
#>
#> attr(,"data_source")
#> [1] "data.frame"
and now:
runHeteroTests(model, quakes)
#> Heteroscedasticity tests of lm(formula = stations ~ mag + depth, data =
#> quakes), n = 1000
#>
#> Statistic df p-value
#> white 125.6 5 <2e-16 ***
#> koenker 111.3 2 <2e-16 ***
#> ---
#> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
These eight methods (print() and summary() of each class, and print()
and coef() of the two summaries) are exported S3 methods, an addition to
0.16.0 outside its "no API change", approved by the author on 2026-10-21. No
value changes: tidy(), glance(), augment() and autoplot() of a suite
return what they returned, and a single test still returns and prints its
htest. A suite whose model runHeteroTests() fitted itself carries a new
attribute, attr(, "model_origin"): "formula" when it was given a
formula, "group" for each group of a grouped run, and "survey" for the
data of a survey design (runSurveyHeteroTests()). The call of such a model
names the runner's own variables (stats::lm(formula = model, data =
prep$data)), so summary() shows its formula instead; a model you fitted is
shown by its call, and its suite has no such attribute.
Through runHeteroTests() a result whose data.name is "model" now reports
the formula of the model instead, as Koenker's test and the others already
did. Five tests of the package reported "model" there, the name of the
argument of the function the registry calls them with: White's test
("white"), its bootstrap ("white_bootstrap"), its form without cross
products (the first fallback of "white"), the studentized Breusch-Pagan
statistic ("student_bp") and Szroeter's test ("szroeter"); so
runHeteroTests(m, d)$white printed data: model where $koenker printed
data: mpg ~ wt + qsec. Those results change, and so does that of a
diagnostic added with registerDiagnostic() whose data.name is "model",
such as one that calls performWhiteTest(model, data) on its argument. No
other data.name changes: a diagnostic of yours keeps what it says it
tested, such as the two halves of the residuals a var.test() compares, and
the NCV and variance-form tests keep the formula followed by what else they
were run against. This changes a field of what runHeteroTests(), and so
runDiagnostics() and runSurveyHeteroTests(), return; no statistic, degree
of freedom or p-value changes. Called directly, a test still names the object
it is given (performWhiteTest(m, d)$data.name is "m"), and
cachedTest(), the registry and runHeteroTestsParallel() return what the
registered function returns, "model" among them, as before. The corpus of
the model context (scripts/model-context-identity.R), whose runner records
now hold the data.name (they hold the tests of a suite, not its
attributes), returns what it returned before: of its 19,523
calls 18,885 are identical(), and the other 638, all runner calls, differ
in the data.name of a test alone, each from "model" to the formula of its
model, which the harness allows and checks against deparse(formula(model));
so do 81 of the 132 runner records on the stored design and of its
verification, and none in any other way.
Packaging¶
- The logo shows in the README, and the
R CMD checkNOTE "Non-standard file/directory found at top level: 'assets'" is gone. The logo is inman/figures/logo.png, where R tooling looks for it, andassets/is left out of the built package. - CI runs the suite once per job, and the full model-context matrix and the
two longest cache tests in one job only;
HETEROTESTS_FULL_MATRIX=falseskips them locally too (see the development guide). No function changes.
Documentation¶
- The documentation site, https://diogoribeiro7.github.io/heteroTests/,
moves from pkgdown to Material for MkDocs and shows the package logo. Its API
reference is generated from the help pages by
tools/build-docs.R, in the sections the pkgdown index had, andtests/testthat/test-reference-index.Rchecks that every export is in one; the vignettes become its user guide. No function changes. - The pkgdown site's addresses, such as
reference/performWhiteTest.html,articles/tutorial.htmlandnews/index.html, redirect to the pages that replaced them; the guides no longer print the[INFO]line of every test they run; and the titles of?performBPTestStreamingand?performKoenkerTestStreaminglose a stray backslash before "Pagan". ?performWildBootstrapTesthas a section Size: the rates at the 5% level of the Rademacher and Mammen multipliers with Gaussian,t5and centred exponential errors at 30, 50, 100 and 200 observations (inst/validation/wild-multiplier-size.Randinst/validation/wild-no-multiplier-size.R). With skewed errors either multiplier needs about 200 observations to hold the level, where a plain residual bootstrap, without a multiplier, held it from 50 at a cost of about 1.4 points of power at 30; it is to be offered asdistribution = "none"in 1.0.0. No value changes.
0.15.0¶
0.15.0 makes the interface match its description before 1.0.0. Every test
takes (model, data, ...), with data = NULL in second position and
performBoxMTest(data, group) the documented exception;
performBPTestRobust() chooses the studentized statistic with studentize,
as performHarveyTest() and lmtest::bptest() do;
rrunAdvancedDiagnostics() is renamed runAdvancedDiagnostics(); and
?test_results documents what every test returns and the inputs every test
refuses. Most of these changes break calls, not values: an old call form that
0.15.0 replaces still works in 0.15.x, warns with the call to use instead
and, apart from the changes in bold, returns what it returned in 0.14.0, and
it stops working in 1.0.0 (the table under Calls that changed gives each).
The changes in bold change what calls return: every resampling function
draws 999 replicates by default; the default battery of runHeteroTests() is
White and Koenker; performGlejserTest() corrects for asymmetric errors by
default; the tests refuse a glm that is not a least-squares fit, which most
of them returned a value for, and so do the registry names "ncv" and
"student_bp", deprecated in 0.14.0; they refuse a model with an NA
coefficient and data that lack rows or variables of the fit, which some of
them returned a value for, and performRESETTest() refuses a
survey::svyglm() fit, as the other tests did already; the Goldfeld-Quandt,
RESET, White bootstrap and wild bootstrap tests read the prior weights of a
Gaussian glm, and the quantile regression test refuses a weighted one; the
tests that read data refuse data whose rows they cannot match to the
observations of the fit, and runHeteroTests() and runDiagnostics() stop
on such data rather than fall back on another test; the robust White and
Breusch-Pagan tests and the White bootstrap give the value on the data of the
fit for rows they had renumbered; seeded resampling with parallel = TRUE is
reproducible, and runHeteroTestsParallel() gives the same value with one
worker as with several; the bootstrap tests estimate the weights of a
fitWLS() fit again in each replicate; and the studentized Breusch-Pagan
statistic reads the weights of a fitWLS() fit.
Resampling tests draw 999 replicates by default¶
Every resampling function draws 999 replicates by default, so its default
p-value changes. A resampling p-value is (1 + #) / (B + 1), the number of
replicates at least as large as the observed statistic, plus one, over the
number of replicates plus one. A test at level alpha that rejects when it is
at most alpha is exact only when alpha * (B + 1) is an integer: 999 is at
the 5% and 1% levels, and 1000 is at neither. The defaults were a mix:
| Function | Default before | Default now |
|---|---|---|
performWildBootstrapTest() |
B = 499 |
B = 999 |
performWhiteTestBootstrap() |
B = 1000 |
B = 999 |
bootstrapTestStatistic() and rbootstrap_test_statistic() |
B = 1000 |
B = 999 |
performWhiteTestRobust() and performBPTestRobust(), with bootstrap = TRUE |
B = 1000 |
B = 999 |
runAdvancedDiagnostics() and rrunAdvancedDiagnostics() |
bootstrap_B = 500 |
bootstrap_B = 999 |
performSpatialHeteroTest() |
permutations = 499 |
permutations = 999 |
performRankPermutationTest() |
B = 999 |
unchanged |
With set.seed(1) before each call, the default p-value was and is:
| Call | Before | Now |
|---|---|---|
performWildBootstrapTest(lm(mpg ~ wt + qsec, data = mtcars)) |
0.2400 | 0.2330 |
performWhiteTestBootstrap(lm(dist ~ speed, data = cars)) |
0.1668 | 0.1670 |
bootstrapTestStatistic(performKoenkerTest, lm(dist ~ speed, data = cars), cars)$p_value |
0.0799 | 0.0800 |
performWhiteTestRobust(lm(dist ~ speed, data = cars), bootstrap = TRUE)$p.value_bootstrap |
0.1688 | 0.1690 |
performBPTestRobust(lm(dist ~ speed, data = cars), bootstrap = TRUE)$p.value_bootstrap |
0.0799 | 0.0800 |
runAdvancedDiagnostics(lm(mpg ~ wt + qsec, data = mtcars), mtcars), White |
0.0259 | 0.0290 |
runAdvancedDiagnostics(lm(mpg ~ wt + qsec, data = mtcars), mtcars), Breusch-Pagan |
0.2335 | 0.2420 |
performSpatialHeteroTest(lm(mpg ~ wt + hp, data = mtcars), listw = lw) |
0.5100 | 0.5080 |
where lw joins each car to its four nearest neighbours in coordinates drawn
by set.seed(7); cbind(runif(32), runif(32)). runAdvancedDiagnostics()
bootstraps both tests because mtcars has fewer than 50 rows. The registered
tests "wild_bootstrap", "white_bootstrap" and "spatial_hetero", and so
runHeteroTests() and the functions built on it, change with them. The B
in the parameter of the two bootstrap tests, robust_details$bootstrap$B of
the robust tests, which is recorded even when bootstrap = FALSE, and
metadata$bootstrap_B of runAdvancedDiagnostics() report the new default.
Pass the old value to reproduce a result of 0.14.0.
Of the recorded evidence, only the full sweep calls a resampling test with
its default, performSpatialHeteroTest(). Its row was regenerated from the
same seeds: under the null it rejected 0.045 of the time and rejects 0.050,
and against the alternative 0.362 and 0.350
(inst/validation/full-sweep-size-power.csv). The other studies give B
explicitly.
White and Koenker are the default battery¶
runHeteroTests() and the functions that share its default run White's
test and Koenker's studentized Breusch-Pagan test, c("white", "koenker"),
where they ran White's and the classical Breusch-Pagan test,
c("white", "breusch_pagan"). The classical statistic divides by
2 sigma^4, the variance of the squared errors only when they are normal. In
the full sweep, with 150 observations and 400 replications, it rejects a true
null hypothesis 28.75% of the time at the 5% level when the errors have a
t5 distribution (115 of 400 samples), against 4.75% (19 of 400) for
Koenker's statistic; under Gaussian errors they reject 5.75% and 4.50%, and
both reject every sample against sd = x^2
(inst/validation/full-sweep-size-power.csv). Pass A, at 5000 replications
and 100 observations, agrees: 21.86% against 4.24% under t5 errors. What
Koenker's statistic gives up is a little power when the errors are normal: in
Pass A it rejects 72.70% of the time against sigma^2 = exp(0.4 x1), where
the classical test rejects 77.44% (inst/validation/pass-a-size-power.csv).
The default changes in runHeteroTests(), runDiagnostics(), summary()
and the deprecated test() on a HeteroDiagnostic object,
compareModelDiagnostics(), compareTestResults(),
runSurveyHeteroTests() and the tests the dashboard selects at start, and so
in generateDiagnosticReport() and generateHeteroRecommendations(), which
call runHeteroTests() with its default. The classical test still runs by
name, tests = "breusch_pagan", or c("white", "breusch_pagan") for the old
battery. The second element of a default suite is now called koenker, so
code that reads $breusch_pagan from it gets NULL. For
lm(mpg ~ wt + qsec, data = mtcars), runHeteroTests(m, mtcars) returned
White's test (11.82 on 5 degrees of freedom, p = 0.0373) and the classical
Breusch-Pagan test (3.135 on 2, p = 0.2086), and returns White's test and
Koenker's (3.086 on 2, p = 0.2138).
Two other functions follow the new default. suggestRemediation() suggests
variance modelling when the Koenker test rejects as well as when the classical
test does; it read only "breusch_pagan", so it would never have suggested it
for a default suite. suggestDiagnosticsForProfile() recommends Koenker's
test, where it recommended the classical one, beside White's as the baseline
for every data profile. runAdvancedDiagnostics() is unchanged: with its
default auto_enhance = TRUE its Breusch-Pagan entry is already the
studentized statistic, but auto_enhance = FALSE turns the studentization
off with the bootstrap and runs the classical test, which is listed under
0.16.0. None of the recorded evidence calls these functions
with their default battery, so no recorded number moves.
The Glejser test corrects for asymmetric errors by default¶
performGlejserTest() computes the statistic of Im (2000) and of Machado
and Santos Silva (2000) by default, robust = TRUE, where it computed
Glejser's (1969), so the statistic and p-value it reports by default
change. Glejser's statistic regresses the absolute residuals on the
variable tested, and has its nominal level only when positive and negative
errors are equally likely. In inst/validation/glejser-skewed-errors.csv, at
5000 replications, it rejects a true null hypothesis 8.8% to 17.7% of the
time at the 5% level when the errors are skewed, at every sample size from 50
to 1000. The corrected statistic, added in 0.13.0, subtracts the mean sign of
the residuals times each residual from its absolute value. Under the same
skewed errors it rejects 4.6% to 5.7% of the time from 150 observations on,
and all 27 of its null cells from 150 observations, with symmetric errors
too, are inside the release-gate interval, [0.0421, 0.0579]. At 50
observations it is not: under skewed errors it rejects 5.7% to 6.6% of the
time, and four of those six cells are above the interval, at 0.0580, 0.0592,
0.0626 and 0.0658, where Koenker's statistic on the same regressor is at
0.0586 to 0.0714. Under symmetric errors the two statistics differ little:
the corrected one rejects 0.5 to 0.7 percentage points less often at 50
observations, and against sigma^2 = exp(gamma x1) with Gaussian errors it
has 1.1 and 1.9 points less power at 50 observations, and 0.8 and 0.1 at 150.
The method of the result says which statistic was computed, and by default
now reads "Glejser test for heteroscedasticity (robust to asymmetric
errors)". Pass robust = FALSE for Glejser's statistic: it returns what the
default returned in 0.14.0, checked with identical() against 0.14.0. The
default was and is:
| Call | 0.14.0 | Now |
|---|---|---|
performGlejserTest(lm(mpg ~ wt + qsec, data = mtcars), mtcars, "wt") |
t = -0.450 (p = 0.656) | t = -0.395 (p = 0.695) |
performGlejserTest(lm(dist ~ speed, data = cars), cars, "speed") |
t = 2.059 (p = 0.045) | t = 1.943 (p = 0.058) |
performGlejserTest() is not a registered test, so runHeteroTests(), the
dashboard and the recommendations do not call it, and no other function of
the package changes with it. Of the recorded evidence, the full sweep and
the known-weights block of the weighted fits call it with its default. Their
two rows were regenerated from the same seeds, and with robust = FALSE the
same runs reproduce the rows of 0.14.0 byte for byte: in the full sweep the
size moves from 0.0600 to 0.0575, the size under t5 errors from 0.0575 to
0.0550, and the power stays 1.000 (inst/validation/full-sweep-size-power.csv);
with known weights the rejection rate moves from 0.0475 to 0.0450
(inst/validation/weighted-fits-size.csv). Pass A and the Glejser study name
Glejser's statistic, and now give robust = FALSE; their numbers do not
change.
Inputs every test refuses¶
Calls that returned a value for a glm that is not a least-squares fit,
for a model with an NA coefficient, or for data that cannot be matched to
the fit now stop with an error. tests/testthat/test-input-validation.R
gives every test these inputs, and each is refused with one message
(?test_results). Before 0.15.0 these calls returned a value:
| Input | Tests that returned a value and now stop | Message |
|---|---|---|
A glm of a family other than gaussian, or with a link other than the identity, such as glm(am ~ wt, family = binomial) |
performWhiteTest(), performWhiteTestRobust(), performWhiteTestBootstrap(), performBPTest() and performBreuschPaganTest(), performBPTestStreaming(), performBPTestRobust(), performKoenkerTest(), performKoenkerTestStreaming(), performStudentizedBPTest(), performGQTest(), performParkTest(), performSzroeterTest(), performWildBootstrapTest(), performQuantileRegressionTest(), performRankPermutationTest(), performHighDimensionalTest(), performSpatialHeteroTest(), performSpearmanTest(), performLeveneTest(), performGlejserTest(), performBrownForsytheTest(), performFlignerKilleenTest(), performBartlettTest(), performHarveyTest(), performArchLMTest(), performMcLeodLiTest(), performHartleyFmaxTest(), performOBrienTest(), performCookWeisbergTest(), performNCVTest(), performSpreadLevelTest(), performPesaranTest(), performBPRandomEffectsTest(), performDavidianCarrollTest(), performRESETTest(): 36 names, all but the three tests that already refused every glm |
"<test>() needs a model fitted by stats::lm(): a glm with the binomial family and the logit link is not a least-squares fit." |
A model with an NA coefficient, such as lm(mpg ~ wt + I(2 * wt), data = mtcars) |
performSpreadLevelTest(), performDavidianCarrollTest(), performPesaranTest(), performBPRandomEffectsTest(), performRESETTest() |
"Model coefficients must be finite before running ..." |
data that lack rows the model was fitted to |
performVarianceFormTest() |
"... requires data to contain the rows used to fit the model. Missing rows: ..." |
data that lack a variable of the model |
performVarianceFormTest(), performPesaranTest(), performBPRandomEffectsTest() |
"The following variables are missing from input data: ..." |
A survey::svyglm() fit, of any family |
performRESETTest(), the only test that did not refuse one |
"The model is a survey::svyglm() fit, whose residuals are scaled by the sampling weights. ..." |
The values these calls returned tested nothing the help pages describe. A
binomial or Poisson fit has residuals whose variance depends on the mean by
construction: performKoenkerTest(glm(am ~ wt + hp, data = mtcars, family =
binomial), mtcars) returned 0.963 (p = 0.618), and performRESETTest() on
the same fit F = 86.1 (p < 0.0001). The residuals of a survey::svyglm()
fit are scaled by the sampling weights, which are not precision weights, and
every other test refused one already. A model with an
aliased term was computed by the five tests as if the term were not there,
and refused by the others. The variance-form test returned a value for data
that lack rows of the fit or a variable of the model, and the Pesaran and
random-effects tests for data that lack a variable, which they did not look
for. performPesaranTest() on data that lack rows stopped with "number of
items to replace is not a multiple of replacement length" in the designs
tried, and now names the rows, as the others do. In
test-input-validation.R, on the design it uses, all 36 names returned a
value for a binomial and for a Poisson glm before 0.15.0.
A glm with the Gaussian family and the identity link is a least-squares fit
and is still accepted. Without prior weights, and with positive ones, every
test that accepts it gives the result of the lm in that design to 1e-8,
the five tests below since they read its weights. The exception is the
Fligner-Killeen statistic, which moves by about 1e-6 of its value: with an
even number of residuals the two in the middle tie at exactly half their
difference from the median, and rounding, which differs between the two fits
by about 1e-15, decides which ranks first. A Gaussian glm with a zero
prior weight is not yet refused as the same lm is: the tests that read the
Pearson residuals, such as the Koenker, White, Glejser, Spearman, GQ and wild
bootstrap tests, return a value for it, and performWhiteTestBootstrap()
stops with "Model residuals must be finite". It is recorded under 0.16.0.
performWhiteTestStreaming(), performHarrisonMcCabeTest() and
performVarianceFormTest() refuse every glm, as before.
The registry names deprecated in 0.14.0, "ncv" and "student_bp", and
the "student_bp" entry of test_factory refuse such a glm too, with the
message of performNCVTest() and performStudentizedBPTest(), and so does
rvalidate_against_reference("studentized_bp", ...). They call the
implementations behind those two functions rather than the functions, and
returned a value for it through runHeteroTests(tests = ...),
runDiagnostics(), cachedTest(), runHeteroTestsParallel() and
test_factory$run_test(). runHeteroTests() falls back on "ncv" when
White's, Koenker's, the classical, the quantile regression or the rank
permutation test stops, so runHeteroTests(<binomial glm>, data) returned
the Cook-Weisberg statistic under the names white and koenker, tagged as
a fallback, where 0.14.0 returned White's and the classical test. Decided on
2026-10-16: it now stops with the refusal of performWhiteTest(), as
runHeteroTests(<binomial glm>, data, tests = "cook_weisberg") stops with
that of performCookWeisbergTest(), and with grouped data it returns each
test as failed, with its refusal. runDiagnostics() stops at the same point,
where it stopped later, at performRESETTest(). No name holds a fallback
computed on such a fit. On an lm, a Gaussian glm and a weighted lm, the
two names, the factory entry and the default battery return values
identical() to those they returned before.
performGQTest(), performRESETTest(), performWhiteTestBootstrap() and
performWildBootstrapTest() now read the prior weights of a Gaussian glm,
and performQuantileRegressionTest() refuses a weighted one, as it refuses a
weighted lm. These five read the weights of an lm, to refit with them or
to refuse a weighted fit, and found none on a glm, so they took a weighted
Gaussian glm for an unweighted fit. Decided on 2026-10-15: they read its
prior weights, and give what they give for the same model fitted by
lm(..., weights = w). test-input-validation.R asserts it to 1e-8, for
the statistic and the p-value, on four seeded designs, the resampling tests
from the same seed and the same B; the largest differences there are
6e-14 in a statistic and 1e-14 in a p-value. On
glm(dist ~ speed, data = cars, weights = 1 / speed), the two bootstraps
after set.seed(1):
| Call | 0.14.0 | Now, as for the lm |
|---|---|---|
performGQTest(order_by = "speed") |
5.42 (p = 0.0004) | 2.65 (p = 0.023) |
performRESETTest() |
F = -21.5 (p = 1.000) | F = 1.37 (p = 0.264) |
performWhiteTestBootstrap(B = 999) |
p = 0.695 | p = 0.701 |
performWildBootstrapTest(B = 999) |
p = 0.378 | p = 0.373 |
performQuantileRegressionTest() |
2.05 (p = 0.155) | Error: "performQuantileRegressionTest() does not support weighted fits: ...", the message for the weighted lm |
The observed statistics of the two bootstraps, 0.819 and 0.766, were already
those of the lm; their replicates were drawn as for an unweighted fit. On
glm(mpg ~ wt + hp, data = mtcars, weights = 1 / wt) RESET returned
F = -6.13 (p = 1.000), and returns 6.11 (p = 0.0065), the value of the lm.
No recorded evidence fits a glm, a model with an NA coefficient or data
that do not match the fit: the rows of the full sweep for the six tests whose
checks changed besides the glm one, the Pesaran, random-effects,
variance-form, RESET, spread-level and Davidian-Carroll tests, run again from
their seeds, are identical.
Data in another order than the fit¶
Calls that returned wrong values on data holding the rows of the fit in
another order now stop with an error. The tests match the rows of data to
the residuals by row name. A tibble, a data.table, or a data frame whose row
names were reset, as dplyr::arrange() resets them, has row names that only
number its rows, so such data were matched by position: in another order than
the fit, or with rows in front of those of the fit, each residual was read
with another observation's row, without an error. With
set.seed(1); n <- 120
d <- data.frame(x = runif(n, 1, 5), z = rnorm(n))
d$y <- 1 + d$x + d$z + rnorm(n, sd = d$x)
m <- lm(y ~ x + z, d)
performKoenkerTest(m, d) gives 17.56, and
performKoenkerTest(m, dplyr::arrange(d, x)) gave 0.114. It now stops with
"performKoenkerTest() cannot match the rows of data to the observations
of the model: data has no row names of its own (a tibble, a data.table, or
a data frame whose row names were reset, as by dplyr::arrange()), so its rows
are matched to the observations by position, and in that order its values are
not those of the fit: y, x, z differ from the model frame. Pass the rows of
the fit in the order of the fit, or a data frame with its row names."
Before a test reads data, it takes the rows that the row names pick for the
observations of the fit and compares them, row by row, with every column of
the model frame of the fit. A column that data holds under its own name,
such as y or x, is compared value by value, each number to 1e-8 of its
own size, and dates and times exactly. Any other column is computed from
data as the fit computed it: a term such as log(y), log(x),
poly(x, 2), ns(x, 3) or offset(o) through the terms of the fit, and
weights or an offset given as arguments from the expressions of the call. It
is computed on all the rows of data, as model.frame() computed it before
dropping rows, or else on the matched rows, and compared value by value too.
So rows that agree on the response but not on a transformed regressor, a
weight or an offset are told apart, as when the data are sorted on a 0/1
response. A term that depends on the rows it is computed from, such as
I((x - mean(x))^2), I(x > median(x)) or rank(x), comes out otherwise on
the rows a fit with subset = or na.omit kept, which are the right rows; it
is not compared, and neither is a column that names a variable data does not
hold or whose computation fails. A column computed from a vector of the
workspace, such as weights = wts, is read by no test from data and needs
nothing more. For the others, the columns that are compared must tell the
rows of the fit apart.
The guard refuses data it cannot confirm, even when they are the data of the
fit. It does so in two cases. One is when no column of the model frame can be
compared, as with lm(d$y ~ d$x). The other is when the compared columns tie
on rows of the fit that differ in a term that depends on the rows, or in a
term of variables of data that cannot be computed. That happens with a
discrete response and a row-dependent regressor on the rows an na.omit fit
kept, as in lm(score ~ g + I(x > median(x)) + z2), and with y01 ~ myf(x)
after rm(myf) or ns() after detach("package:splines"). Pass the full data
the model was fitted on, with the rows it dropped, or its variables as
columns of data. Data whose rows differ are refused, and so are repeated row
names. Data with row names of their own, such as
d[sample(n), ], are still matched by name, the tibble or data.table the
model was fitted on is accepted, and so are rows after those of the fit. On
eighteen fits (unweighted, weighted, with an offset given as an argument or as
a term, log(y), na.omit with three rows dropped, a Gaussian glm, a
fitWLS() fit, log(x) + poly(x, 2), model = FALSE, a discrete response
with weights or an offset from the workspace, and seven with a term computed
from the rows under na.omit or subset =), every exported test on
the data of the fit, with data = NULL, on the same rows shuffled with their
row names, on the rows the fit kept, and on the tibble and the data.table the
model was fitted on returns what it returned, identical(), except the three
tests below on the shuffled rows, which returned wrong values, and the
robust and bootstrap tests that stopped with an error on the tibble of a
subset = fit and now return a value (below); some errors changed their
message.
The guard covers the 23 tests that returned wrong values on such data:
performWhiteTest(), performWhiteTestStreaming(),
performWhiteTestRobust(), performWhiteTestBootstrap(), performBPTest()
and performBreuschPaganTest(), performBPTestStreaming(),
performKoenkerTest(), performKoenkerTestStreaming(),
performRankPermutationTest(), performHighDimensionalTest(),
performParkTest(), performSzroeterTest(), performHarrisonMcCabeTest(),
performGlejserTest(), performHartleyFmaxTest(), the five group tests
(performLeveneTest(), performBrownForsytheTest(),
performFlignerKilleenTest(), performBartlettTest(), performOBrienTest()),
performPesaranTest() and performBPRandomEffectsTest(), and also
performCookWeisbergTest() with var_formula, which read the variables of
the variance formula from the same rows. Six tests that start the same way,
performGQTest(), performStudentizedBPTest(), performBPTestRobust(),
performWildBootstrapTest(), performSpatialHeteroTest() and
performQuantileRegressionTest(), returned the right value on reordered
numbered data, the quantile test to rounding (a relative difference of
3e-15 on the data above), because they take everything but an ordering from
the fit or refit on the rows; they refuse it too, so that every test that
reads data treats it alike, and so do the registry names "szroeter",
"wild_bootstrap", "quantile_regression" and "student_bp" in
runHeteroTests(), which returned the right value too. Exempting them would
leave the cases with rows in front of those of the fit, on which they were
wrong. The tests that read nothing from data, such as
performSpearmanTest() or performRESETTest(), are unchanged.
bootstrapTestStatistic() refused such data already (0.13.0); with one of
these tests as test_function it now stops with the test's message, before
its own check.
A model fitted with model = FALSE keeps no model frame, and the tests
rebuild it from the data the fit refers to. When those data changed after the
fit, the rebuilt frame held other rows, and the tests computed on it: after
m <- lm(y ~ x + z, d, model = FALSE) and d <- dplyr::arrange(d, x),
performKoenkerTest(m), performKoenkerTest(m, d) and Koenker's test in
runHeteroTests(m) gave 0.114 instead of 17.56. The response of the rebuilt
frame is now compared with that of the fit, and its design, times the
coefficients, plus its offset, with the fitted values, and such a frame is
refused ("cannot use the model frame of model").
A tibble that has more rows than the fit and incomplete rows was renumbered
when a test dropped those rows, and its rows then matched others: on a fit to
the first 100 rows of a tibble of 120, three of them incomplete, Koenker's
statistic was 0.0165 where the rows of the fit give 7.93. It is refused, with
the advice to pass as.data.frame(data).
performWhiteTestRobust(), performWhiteTestBootstrap() and
performBPTestRobust() give the value on the data of the fit where they gave
another one. They call White's or the Breusch-Pagan test again on the rows
they matched to the fit, and a tibble they had subset numbered those rows
afresh, so the inner test matched them to other observations. With a fit to
set.seed(4); ds <- d[sample(n), ] and data = tibble::as_tibble(d), valid
data, performWhiteTestRobust() and the observed statistic of
performWhiteTestBootstrap() were 3.09 and
performBPTestRobust(studentize = FALSE) 0.140, where the rows of the fit
give 21.79 and 37.06. They now pass the rows under the row names of the fit,
and return those values.
performPesaranTest(), performBPRandomEffectsTest() and
performScatterDiagnostic() read data by position, the i-th row with the
i-th residual, so they gave wrong values on data with row names of their own
in another order too. With d$id <- rep(1:12, each = 10),
d$time <- rep(1:10, 12) and set.seed(2); dp <- d[sample(n), ], the
Pesaran statistic was -0.987 on d and -0.392 on dp, the random-effects
statistic 1.19 and 0.506, and performScatterDiagnostic(m, ., "x") 0.468 and
-0.080. They refuse such data ("the rows of data, matched by name, are not
in the order of the fit"), data whose row names are not those of the fit,
which cannot show their order, and data with another number of rows than the
fit, with the message the random-effects test gave already ("requires data
with 117 observations to match the fitted model, got 120"), where the Pesaran
test stopped with "arguments imply differing number of rows" and the scatter
diagnostic with "incompatible dimensions".
runHeteroTests() and runDiagnostics() stop with this error, and do not
fall back on another test for it. On dplyr::arrange(d, x) above,
runHeteroTests(m, ...) returned White's statistic 1.50 and Koenker's 0.114,
both computed on misaligned rows (21.79 and 17.56 on d). When one of their
tests stops, the runners run a substitute under its name, the last of them the
Cook-Weisberg test, which reads nothing from data; for this refusal there
is none, so no name can hold a test computed on, or regardless of, such data.
The error keeps the class of the refusal, heteroTests_data_rows_error.
The guard is a stopgap. 0.16.0 replaces it with the model context of the
roadmap, which takes the response, design and weights from the fit and
matches the rows of data by name or by verified position, so that named
data in any order, and numbered data in fit order, give the value on the data
of the fit.
Seeded resampling in parallel is reproducible¶
bootstrapTestStatistic(), performWhiteTestBootstrap(), the robust tests
with bootstrap = TRUE and runHeteroTestsParallel() return the same value
for the same seed with parallel = TRUE, and a different value from the one
they returned in parallel before. The replicates computed in worker
processes drew from generators that set.seed() in the session did not
reach, so a seeded call returned another p-value each time it ran. Replicate
i, or test i in runHeteroTestsParallel(), now draws from stream i of
a L'Ecuyer-CMRG generator (parallel::nextRNGStream()) seeded by one draw
from the session's generator, on the forked path (parallel::mclapply()) and
on the socket cluster alike, so the value depends on the seed and not on the
number of workers or the path. With set.seed(1),
bootstrapTestStatistic(performKoenkerTest, lm(dist ~ speed, data = cars),
cars, parallel = TRUE, n_cores = 2)$p_value gave 0.0740 and then 0.0760 on
two runs, and gives 0.0830 on every run.
A serial call draws every replicate from the session's generator, as before,
and returns what it returned: 0.0800 for that call with parallel = FALSE.
The parallel and the serial value differ because their draws differ; both
are bootstraps of the same data, and
tests/testthat/test-parallel-reproducibility.R checks that their replicates
have the same distribution. The help pages say this under Parallel
replicates.
runHeteroTestsParallel() with one worker returns a different value for a
resampling test. With n_cores = 1, and when the default comes to one
worker, for one test or on a machine with two cores or fewer, it runs the
tests in the session, where they drew from the session's generator. Test i
draws from stream i there too now, so the value is the one the workers
give, whatever n_cores. With set.seed(99),
runHeteroTestsParallel(lm(mpg ~ wt + qsec, data = mtcars), mtcars, tests =
c("koenker", "wild_bootstrap"), n_cores = 1) gave a wild bootstrap p-value
of 0.2180 in 0.14.0, that of performWildBootstrapTest() called from the
same seed with the 499 replicates of then (0.2120 with the 999 of now), and
gives 0.2110, as with n_cores = 2 or 3.
performWhiteTestBootstrap(parallel = TRUE) forked its workers with
parallel::mclapply() on every platform, so on Windows, which cannot fork,
it stopped with 'mc.cores' > 1 is not supported on Windows whenever the
machine had more than two cores. It runs its replicates on a socket cluster
there, as bootstrapTestStatistic() does, and gains n_cores, the number of
workers, which the result does not depend on: with set.seed(1),
performWhiteTestBootstrap(lm(dist ~ speed, data = cars), parallel = TRUE,
n_cores = 2) gives 0.1750 on every run, against 0.1670 serially.
No recorded evidence runs a resampling test with parallel = TRUE.
Bootstrap tests estimate the weights of a fitWLS() fit again¶
On a fit returned by fitWLS(), performWildBootstrapTest(),
performWhiteTestBootstrap() and bootstrapTestStatistic() estimate the
weights again in each replicate by default, so the p-values they return on
such a fit change. Since 0.13.0 they held the weights fitWLS() estimated
fixed in every replicate, as if they were known. The replicates then leave out
the estimation that the observed statistic went through, and the tests
rejected a true null hypothesis too often, more so with more observations. In
inst/validation/weighted-fits-size.R, the data have standard deviation
exp(0.3 x) and fitWLS() fits its default exponential variance function,
which is correctly specified. With normal errors, 2000 replications and
B = 499, the rejection rates at the 5% level are:
| Test | n | Weights held fixed, before | Estimated again, now | True weights |
|---|---|---|---|---|
performWildBootstrapTest() |
150 | 0.0865 | 0.0605 | 0.0540 |
performWildBootstrapTest() |
600 | 0.1050 | 0.0515 | 0.0480 |
performWhiteTestBootstrap() |
150 | 0.0640 | 0.0415 | 0.0540 |
bootstrapTestStatistic() with performKoenkerTest() |
150 | 0.0825 | 0.0450 | 0.0455 |
(inst/validation/weighted-fits-size.csv, the rows whose weights are
"estimated, held fixed", "estimated, re-estimated" and "true"). At 2000
replications the 99% interval around 0.05 is [0.037, 0.063], and all four
rates with the weights estimated again are inside it. The release gate is set
at 5000 replications, and against its interval, [0.042, 0.058], the wild
bootstrap with 150 observations, 0.0605, is outside, above it, and the White
bootstrap, 0.0415, just below it. Against [0.041, 0.057], the interval around
0.049, the rate of an exact test with B = 999, the wild bootstrap with 150
observations is outside and the other three are inside. No study of the new
default at 5000 replications with B = 999 has been run; ROADMAP.md lists
it under 0.16.0.
Those four rows were measured in 0.12.0 by repeating the resampling outside
the package, and were not run again. The package reproduces the repetition on
200 of the 2000 samples of each: the same p-values, the same replicates and
the same random draws (inst/validation/weighted-fits-refit-reproduction.csv).
Each replicate is refitted as fitWLS() fitted the model: by least squares,
as the model fitWLS() was given, with its weights if it had any. The
variance function fitWLS() recorded is then fitted again to the residuals,
with the same form and on the regressors fitWLS() evaluated. Last, the
replicate is refitted with the weights that gives. A replicate whose
variance function cannot be fitted, where fitWLS() would fall back to equal
weights, counts as a failed replicate, and the p-value is computed from the
replicates that remain, as for a refit that fails; a warning gives the number
that failed. fitWLS() records the weights of a weighted model it refits, as
initial_weights in its "variance_function" attribute, for this. A fit
made by an earlier version has no initial_weights, and its model frame,
which still holds those weights, is read instead.
The new argument reestimate_weights chooses. NULL, the default, estimates
the weights again on a fitWLS() fit and holds them fixed on any other model.
reestimate_weights = FALSE holds them fixed and returns what 0.14.0
returned. TRUE on a model that fitWLS() did not return is an error. An
lm, a weighted lm and a Gaussian glm carry no variance function, and they
return what they returned. Two kinds of fitWLS() fit keep the values of
0.14.0 under the default. One on which fitWLS() fell back to equal weights
estimated no weights. The other has a variance regressor computed from the
response. Each replicate regenerates the response and holds the design
fixed, so such a regressor is one that uses a variable of the response which
the right-hand side of the model does not use, or one that is the response
term itself, as ~ . gives. Examples are ~ y or ~ . for y ~ x + z, and
~ y or ~ x for I(y / x) ~ z. Its weights cannot be estimated again from
the regressors fitWLS() recorded, so the default holds them fixed and warns.
The rule is conservative for ~ x with I(y / x) ~ z: giving the response as
a column, r ~ z, says that x stays. On both kinds of fit, TRUE is an
error. In bootstrapTestStatistic() and
rbootstrap_test_statistic() the argument comes after ..., so it has to be
named in full.
With set.seed(1) before each call, for
fit <- fitWLS(lm(dist ~ speed, data = cars)):
| Call | Before, and with reestimate_weights = FALSE |
Now |
|---|---|---|
performWildBootstrapTest(fit, cars) |
0.6730 | 0.6990 |
performWhiteTestBootstrap(fit, cars) |
0.7640 | 0.7290 |
bootstrapTestStatistic(performKoenkerTest, fit, cars)$p_value |
0.7020 | 0.6750 |
The functions that call these with their default change with them on a
fitWLS() fit. These are the registered tests "wild_bootstrap" and
"white_bootstrap", which runHeteroTests(), runHeteroTestsParallel()
and the functions built on them run by name, and performWhiteTestRobust() and
performBPTestRobust() with bootstrap = TRUE, and so
runAdvancedDiagnostics() when it bootstraps. They have no argument of their
own; to hold the weights fixed, call the bootstrap function directly with
reestimate_weights = FALSE.
Of the recorded evidence, only the rows "estimated, held fixed" of
weighted-fits-size.R call these functions on a fitWLS() fit. The script
passes reestimate_weights = FALSE there, so they do not move, and it calls
the package for the rows "estimated, re-estimated".
The studentized Breusch-Pagan statistic on a fitWLS() fit¶
On a fit returned by fitWLS(), performStudentizedBPTest(),
performBPTestRobust() with its default studentize = TRUE, and the
registered "student_bp" computed the statistic of the unweighted fit. They
now compute that of the weighted fit, which is Koenker's. They read the
weights from the model frame, and the model frame of a fitWLS() fit is that
of the model fitWLS() refitted. It holds that model's weights, none for an
unweighted model, and not the estimated ones. On a correctly specified fit the
statistic therefore reported the heteroscedasticity that the weights remove.
In the design of inst/validation/weighted-fits-size.R, with
set.seed(s) for s from 1 to 5, x and z uniform on (1, 5),
y = 1 + 2 x + 0.5 z + exp(0.3 x) e with normal e, 150 observations and
fit <- fitWLS(lm(y ~ x + z)), performStudentizedBPTest(fit) returned
p = 0.0019, 0.0015, 4.2e-07, 0.00097 and 0.013. It now returns 0.50, 0.86,
0.22, 0.56 and 0.53, the p-values of performKoenkerTest(). Since 0.15.0 the
three read the weights of the fit, as the other tests do (decided on
2026-10-16).
The values change on a fitWLS() fit only:
- the statistic and p-value of
performStudentizedBPTest(); - the statistic, the asymptotic p-value and the bootstrap p-value of
performBPTestRobust(), and so the Breusch-Pagan entry ofrunAdvancedDiagnostics(); "student_bp".
The model frame of a fitWLS() fit is unchanged. Nothing else changes. A
corpus of seeded calls was run on the development version before and after
the change: the corpus of the review of #129, extended to every test on the
models that fitWLS() did not return.
- On an
lm, a weightedlm, anlmwith an offset given either way or with weights and an offset, a Gaussianglmwith and without weights, and a model oflog(y), all 472 calls return what they returned, with the same warnings and random draws. 464 areidentical(). In the other 8 the result holds theglm, whose family closures compare equal only up to their environments. - On the six
fitWLS()fits, 228 of the 234 calls of the 39 other tests and runners return what they returned: 226identical(), and 2 up to theinitial_weightsthat a fit of a weighted model now records. The six that change areperformStudentizedBPTest().
The statistic now equals Koenker's, so on a fitWLS() fit it has Koenker's
level, which was already documented. The weights are taken as known, and with
the variance function correctly specified and normal errors it rejects 0.077
of the time at the 5% level with 150 observations and 0.110 with 600
(inst/validation/weighted-fits-size.csv, performKoenkerTest(), weights
"estimated"). The bootstrap of performBPTestRobust() now compares the
statistic with replicates computed the same way. In an exploratory run by the
reviewer of #129, not recorded in inst/validation/ (20 samples of a
correctly specified design, B = 99), the 0.14.0 path rejected all 20 and
the new default 1 of 20, with a mean p-value of 0.498. The statistic still
drops an offset, as before; that is
listed for 0.16.0 in ROADMAP.md. No recorded evidence calls these on a
fitWLS() fit: weighted-fits-size.R runs performStudentizedBPTest() and
performBPTestRobust() on fits with known weights only.
Calls that changed¶
| Old form | New form | Old form still works |
|---|---|---|
performArchLMTest(model, lags) |
performArchLMTest(model, data = NULL, lags = 1) |
In 0.15.x, with a warning, when lags is a number given second by position; not in 1.0.0 |
performMcLeodLiTest(model, lags) |
performMcLeodLiTest(model, data = NULL, lags = 10) |
As above |
performDavidianCarrollTest(model, degree) |
performDavidianCarrollTest(model, data = NULL, degree = 2) |
In 0.15.x, with a warning, when degree is a number given second by position; not in 1.0.0 |
performCookWeisbergTest(model, var_formula) |
performCookWeisbergTest(model, data = NULL, var_formula = NULL) |
In 0.15.x, with a warning, when var_formula is a formula given second by position; not in 1.0.0 |
performNCVTest(model, var_formula) (deprecated) |
performNCVTest(model, data = NULL, var_formula = NULL) |
As above, with the deprecation warning of 0.14.0 as well |
performHarveyTest(model, auxiliary, studentize) |
performHarveyTest(model, data = NULL, auxiliary, studentize) |
In 0.15.x, with a warning, when auxiliary is a string given second by position, alone or followed by studentize; not in 1.0.0 |
performRESETTest(model, power) |
performRESETTest(model, data = NULL, power = 2:3) |
In 0.15.x, with a warning, when power is a number given second by position; not in 1.0.0 |
performSpearmanTest(model) |
performSpearmanTest(model, data = NULL) |
Yes: data is optional |
performSpreadLevelTest(model) |
performSpreadLevelTest(model, data = NULL) |
Yes: data is optional |
performWhiteTest(model, data, ...) and the 28 other tests that took data second |
performWhiteTest(model, data = NULL, ...) |
Yes: data is now optional |
performBPTestRobust(model, data, studentized = TRUE) |
performBPTestRobust(model, data, studentize = TRUE) |
In 0.15.x, with a warning; not in 1.0.0 |
rrunAdvancedDiagnostics(model, data, ...) |
runAdvancedDiagnostics(model, data, ...) |
In 0.15.x, with a warning; the old name goes in 1.0.0 |
Calls that name their arguments, such as performArchLMTest(model, lags = 3),
are unchanged and do not warn.
The (model, data, ...) convention¶
Every test now takes (model, data, ...), with data = NULL in second
position. The nine tests above that took no data, which read everything
they need from the model, use the model frame when it is omitted, as
performVarianceFormTest() and fitWLS() have done since 0.12.0. Giving them
the data does not change the result, except that
performCookWeisbergTest(model, data, var_formula) and performNCVTest()
evaluate var_formula in data, matched to the residuals by row name, so the
variance model can use a variable the mean model does not; with data = NULL
it is evaluated in the model frame, as before. performBoxMTest(data, group)
is the documented exception: it tests the covariance matrices of the data,
not the residuals of a model. tests/testthat/test-model-data-convention.R
checks the signatures, the warnings and that each old and new call returns
the value of the old one. The registry entries "cook_weisberg" and
"spread_level" pass the data on.
The other 29 tests, which read variables from data and required it, can now
be called without it: data = NULL stands for the model frame of model, so
performWhiteTest(model) returns what
performWhiteTest(model, stats::model.frame(model)) returns, which is what
runHeteroTests(model) passes them. A call that gives data returns what it
returned. The model frame holds the terms of the model rather than its
variables (log(x) rather than x), so a model that transforms a variable
or has an offset() term still needs data, as does a test on a column that
is not a variable of the model, such as the group of performLeveneTest().
On a weighted fit whose weights are a column of the data (weights = w), the
eight tests that rebuild the design from data, performWhiteTest(),
performWhiteTestRobust(), performWhiteTestBootstrap(), performBPTest()
and performBreuschPaganTest(), performKoenkerTest(),
performRankPermutationTest() and performHighDimensionalTest(), stop with
"object 'w' not found" without it, as they do when given the model frame, and
so do they on a fit with offset = o. runHeteroTests() and
runDiagnostics() then return the Cook-Weisberg test, which they fall back
on when a test stops, under the names white and koenker (in 0.14.0 under
white and breusch_pagan), so pass data for such a fit. One call returns
another value: performRESETTest(model, NULL), which
read NULL as power and returned NaN, now computes the test with the
default powers, and a data frame given second, which was read as one power
per column, is now read as data. In a seeded comparison of 30 tests in 39
settings on five designs, among them a weighted fit and a fit with NA rows,
565 of the 571 calls that returned a value before return an identical()
value, the other six being such performRESETTest() calls, and a call
without data is identical() to one given the model frame in all 195
cases.
studentize everywhere¶
The argument that chooses the studentized statistic is called studentize
throughout, as in lmtest::bptest() and performHarveyTest().
performBPTestRobust() called it studentized; the old name is still
accepted in 0.15.x, does what studentize does and warns that it stops
working in 1.0.0. The field robust_details$studentized of the result keeps
its name, so the returned object does not change.
runAdvancedDiagnostics()¶
rrunAdvancedDiagnostics() is renamed runAdvancedDiagnostics(), the last of
the r-prefixed exports 0.14.0 reviewed, so that it follows the camelCase
names of the other runners. The old name still works, returns what it
returned and warns with .Deprecated() that it goes in 1.0.0, with the names
deprecated in 0.14.0.
The return contract¶
A new help page, ?test_results, documents every field a perform*Test()
returns: the five every test returns (statistic, parameter, p.value,
method, data.name), the optional alternative and estimate, and the
fields a few tests add, such as robust_details or bootstrap.
tests/testthat/test-return-contract.R runs every exported test, and the
settings that add fields, and fails on a field the page does not document or
one that breaks it. Six tests needed a change to comply, and no number
changes in any of them:
- the
parameterofperformSpearmanTest(),performSpreadLevelTest(),performBartlettTest(),performFlignerKilleenTest()andperformBPRandomEffectsTest()gains its name,df; it had none, soprint()showedt = -0.41855, = 30, p-value = 0.6785; performBoxMTest()gains thedata.nameevery other test returns.
Over 156 seeded results of every exported test on four designs, one of them a
weighted fit, 132 are identical() to those of 0.14.0 and the other 24, of
these six tests, are identical() once the added name or data.name is
removed.
Input validation¶
tests/testthat/test-input-validation.R gives every exported test the same
four bad inputs: a glm, a model with an NA coefficient, data that lack
rows the model was fitted to, and data that lack a variable of the model.
Where a test already stopped, it now stops with the message the others give:
- the eight tests that read everything from the model check the
datathey are given since 0.15.0, with the messages of the tests that usedata; the model frame is accepted as it is; performBPRandomEffectsTest()names the rowsdatalacks, where it stopped with "arguments must have same length";performWhiteTestStreaming()refuses aglm, where it stopped with "missing value where TRUE/FALSE needed", and it,performHarrisonMcCabeTest()andperformVarianceFormTest()refuse it with one message.
No value changes: over the 217 calls of the (model, data, ...) comparison
and the 156 results of the return-contract comparison, every result is
identical() to 0.14.0, apart from the df names and the data.name above.
The tests that returned a value for one of the four inputs refuse it too, which
changes what those calls return (Inputs every test refuses, above).
The advice for a small sample¶
warnDiagnosticAssumptions(), and so generateHeteroRecommendations(), told
the user of a sample of fewer than 100 observations to "rely on wild
bootstrap or permutation-based tests for reliable inference". In the
package's validation study of the multiplier,
inst/validation/wild-multiplier-size.R, with B = 999, the wild bootstrap
holds its level with Gaussian and t5 errors at 30, 50 and 100
observations, but not with skewed ones: with centred exponential errors it
rejects a true null hypothesis 8.2%, 7.3% and 6.1% of the time at the 5%
level with the Rademacher multiplier, and 7.8%, 7.0% and 5.9% with the
Mammen multiplier. The warning now says so, as the help page of
performWildBootstrapTest() does since this release, and asks the user to
check the residuals for skewness and to read p-values near the level with
caution. Only the text changes: which warnings are given, and when, does not.
0.14.0¶
0.14.0 reviews the exported names before 1.0.0: it deprecates 22 exports and
the registry names "student_bp" and "ncv", and removes nothing.
performBreuschPaganTest(), performStudentizedBPTest() and
performNCVTest() give way to performBPTest(), performKoenkerTest() and
performCookWeisbergTest(), which gains var_formula; the three *Enhanced
functions to enhanced = TRUE on their base names;
rbootstrap_test_statistic(), test(), simulate_type_I_errors() and
simulate_power_analysis() to
bootstrapTestStatistic(), runDiagnostics(), simulate_size() and
simulate_power(); and twelve internal helpers are to leave the exports (the
table under Deprecated names gives each old name and what to use instead).
No exported function returns a different value: each deprecated function
returns what it returned in 0.13.0 and warns, and performStudentizedBPTest()
keeps its own implementation, which disagrees with performKoenkerTest() on a
fitWLS() fit until the model frame of such a fit is corrected in 0.16.0. The
dashboard and benchmark functions, which move to a companion package, are not
deprecated yet because that package has no name, and the deprecated names are
removed in 1.0.0.
Deprecated names¶
0.14.0 reviews the exported names before 1.0.0. Nothing is removed: each name
below still works, returns what it returned before and warns with
.Deprecated(), naming its replacement where it has one. The names go in
1.0.0.
| Deprecated | Use instead | Note |
|---|---|---|
performBreuschPaganTest() |
performBPTest() |
The same function under a second name |
performStudentizedBPTest() |
performKoenkerTest() |
Same statistic, except on a fitWLS() fit (below) |
runHeteroTests(tests = "student_bp") |
tests = "koenker" |
Warns once per call of runHeteroTests() or runDiagnostics() |
performNCVTest(model, var_formula) |
performCookWeisbergTest(model, var_formula) |
performCookWeisbergTest() gains var_formula; with one, it returns what performNCVTest() returned |
performNCVTest(model) |
performCookWeisbergTest(model) |
Same statistic, degrees of freedom and p-value; labelled as the Cook-Weisberg test |
runHeteroTests(tests = "ncv") |
tests = "cook_weisberg" |
Warns once per call of runHeteroTests() or runDiagnostics() |
plotResidualsFittedEnhanced(model) |
plotResidualsFitted(model, enhanced = TRUE) |
enhanced = FALSE, the default, keeps the plot the base name drew |
plotDiagnosticSuiteEnhanced(model) |
plotDiagnosticSuite(model, enhanced = TRUE) |
As above |
checkModelEnhanced(model, data) |
checkModel(model, data, enhanced = TRUE) |
checkModel(model) still checks the class only |
std_error(), std_warning() |
None | Internal helpers |
checkData(), validateTestInputs() |
None | Internal helpers |
ht_enable_log_capture(), ht_log_history(), ht_clear_log_history() |
None | ht_set_log_level() stays |
TestFactory, test_factory |
registerDiagnostic(), runHeteroTests() |
Objects cannot warn: deprecated on their help page and here only |
rbootstrap_test_statistic() |
bootstrapTestStatistic() |
The same function; each name records its own call in the result |
rcalculateEffectSize(), restimate_test_power(), rvalidate_against_reference() |
None | Internal helpers |
test(x, tests) |
runDiagnostics(x$model, x$data, tests) |
Its name masks devtools::test(); summary() of a HeteroDiagnostic object is unchanged |
simulate_type_I_errors() |
simulate_size() |
Joins the snake_case simulation family |
simulate_power_analysis() |
simulate_power() |
As above |
performStudentizedBPTest() and performKoenkerTest() agree to 1e-8 in
statistic, degrees of freedom and p-value in the designs of
tests/testthat/test-koenker-studentized.R: a Gaussian null, sd = x^2 and a
t5 null at 150 observations, a factor regressor and a weighted fit. Both
refuse a rank-deficient fit. On a fit returned by fitWLS() they do not
agree: the model frame of that fit still holds the weights of the model
fitWLS() refitted, not the estimated ones, and performStudentizedBPTest()
reads them from there. On the fitWLS() design of the same file it returns
42.60, where performKoenkerTest() and an lm() fitted with the estimated
weights return 1.75. performKoenkerTest() is the correct one. The deprecated
name and "student_bp" keep their old implementation in 0.14.0, so no value
changes; the model frame is to be corrected in 0.16.0.
rrunAdvancedDiagnostics() stays exported; its name is still under review.
The simulation functions, simulate_*() and sigma_*(), keep their
snake_case names on purpose, as a documented family; the rest of the package
uses camelCase.
Reference index¶
The reference index of the pkgdown site groups the functions: the tests of
the variance by family, and the model checks (performRESETTest(),
performVIFDiagnostic(), performInfluenceDiagnostics(),
performScatterDiagnostic() and checkModel()) apart from them, so that they
are not mistaken for heteroscedasticity tests. No function changes.
The planned export list¶
tests/testthat/test-public-api.R lists the exports planned for 1.0.0, the
names deprecated in 0.14.0 and the three that move to a companion package,
from the export inventory in ROADMAP.md. An export in none of these lists
fails the test, so a new export has to be added to the plan on purpose.
0.13.0¶
The bootstrap tests change value in three cases where their replicates were
wrong: on an unweighted fit with an offset given as an argument, which all
three dropped; in rbootstrap_test_statistic(), on data whose rows are in
another order than the fit's or include rows the fit dropped; and in
performWhiteTestBootstrap(), on a transformed response such as log(y),
where the p-value was 1 whatever the data. Four more results change value:
fitWLS() keeps the offset of the model it refits, and
performVarianceFormTest() changes with it; the Szroeter test registered for
runHeteroTests(tests = "szroeter"), and so for the functions built on it,
orders the observations by the fitted values; runHeteroTests() and
runDiagnostics() given a survey design no longer test a fit weighted by its
sampling weights; and tidy() returns one row per test. Apart from these, no
exported test returns a different value for a call that ran in 0.12.0, though
rbootstrap_test_statistic() now refuses data it cannot match to the fit. The
bootstrap tests also accept a weighted fit, holding the weights of a
fitWLS() fit fixed, and the package gains the Harrison-McCabe test and, as
performGlejserTest(robust = TRUE), a Glejser test that keeps its level under
skewed errors.
Bootstrap tests on a fit with an offset¶
rbootstrap_test_statistic(), performWhiteTestBootstrap() and
performWildBootstrapTest() keep an offset given as an argument, and change
value on an unweighted fit with one. lm(..., offset = o) keeps o in the
model frame and not in the formula, and each replicate of an unweighted fit
was refitted from the formula alone, so the replicates were those of a model
without the offset. A weighted fit already kept it, and an offset written in
the formula, offset(o), was always evaluated again from the data. For
lm(dist ~ speed, data = cars, offset = 0.06 * speed^2), with set.seed(1)
and B = 999, rbootstrap_test_statistic() with performKoenkerTest() gave
p = 0.095 and gives 0.106, and performWhiteTestBootstrap() gave p = 0.222
and gives 0.201, the values the same model gives with the offset in the
formula. performWildBootstrapTest() dropped such an offset too, where it ran
at all (below): for this model its replicates change and its p-value, 0.079,
does not. In inst/validation/offset-refits-size.R, where the offset is
0.5 x^2, the White bootstrap rejected 0.012 of the time at the 5% level and
now rejects 0.044. rbootstrap_test_statistic() with performKoenkerTest()
rejected 0.047, because Koenker's statistic happens to miss what the dropped
offset leaves in the residuals, and now rejects 0.040. That is inside the 99%
interval around 0.045, [0.037, 0.053], the rate at which a test whose level
is exact gives p < 0.05 with B = 199, which is what the cells with
B = 199 are judged against; read against 0.05 it would be below the release
gate, [0.042, 0.058]. With B = 999, its default, whose exact rate is
0.049, it rejects 0.046, inside the release gate and inside the interval
around 0.049, [0.041, 0.057].
performWildBootstrapTest() runs on unweighted fits it stopped on. It
evaluated the call of the fit again in the model frame, which holds log(wt)
and not wt, and (offset) in place of the name of an offset given as an
argument. So lm(mpg ~ log(wt), data = mtcars) stopped with
object 'wt' not found, and so did a transformed response, an offset written
in the formula and an offset given as an argument, unless the model frame
happened to hold the variables they are computed from. Each replicate is now
refitted on the design stored with the fit, with its offset. In
inst/validation/offset-refits-size.R it rejects 0.044 of the time at the 5%
level with an offset given as an argument, and 0.048 with log(x) as well.
Calls that ran before give identical values, except with an offset given as an
argument (above).
rbootstrap_test_statistic() on reordered data and dropped rows¶
rbootstrap_test_statistic() matches the rows of data to the fit by name,
and changes value when they are in another order or include rows the fit does
not have. It took the fitted values of an unweighted fit by position, so
data holding the fitted rows in another order regenerated each row from
another row's fitted value, without an error. For
lm(mpg ~ wt + qsec, data = mtcars) with mtcars sorted by hp,
set.seed(1) and B = 999, the p-value with performKoenkerTest() was 0.004
and is 0.230; with mtcars as given it is 0.219. In
inst/validation/offset-refits-size.R, with the rows in a random order, it
rejected 0.257 of the time at the 5% level and now rejects 0.049.
Rows of data the model was not fitted to, such as rows it dropped for
missing values, are now left out, weighted or not, for the observed statistic
and the replicates alike; the rows that stay keep the order of data, so data
holding exactly the fitted rows gives the same values as before. Pairs
resampling drew those rows too, and each replicate was refitted to a different
number of rows. For lm(Ozone ~ Solar.R + Wind, data = airquality), whose fit
drops 42 rows, pairs resampling with performKoenkerTest(), set.seed(1) and
B = 999 refitted replicates of 92 to 126 rows and gave the interval 2.884 to
18.904; every replicate now has the 111 rows of the fit, and the interval is
3.134 to 19.599. Null resampling refused such data and now runs. Data that
lacks a row the model was fitted to is refused.
Matching by name cannot see two things. A tibble or a data.table has no row
names of its own; when it has as many rows as the fit, as the complete cases
do, they are taken in the order of the fit. And a variable the formula finds
outside data, such as offset(o) with o in the environment of the
formula, keeps the order of the fit, so it cannot follow data in another order
nor the rows that pairs resampling draws; given as an argument, the same
offset does. The model is therefore refitted to the rows of data, and data
whose refit does not give the residuals of the fit, to a relative 1e-8, is
refused, weighted or not: rows in another order taken by position, values that
differ from those of the fit, or such a variable. These gave wrong replicates
without an error. A term computed from all the data, such as
I(x - mean(x)), does not count as a difference. The package's own tests
refuse a variable that is not a column of the data, so only a test function of
the caller's own could reach that case.
performWhiteTestRobust(bootstrap = TRUE) and
performBPTestRobust(bootstrap = TRUE) resample through
rbootstrap_test_statistic(), so their bootstrap p-values change with it,
here and with an offset (above).
The White bootstrap on a transformed response¶
performWhiteTestBootstrap() regenerates a transformed response, and
changes value on a model with one. For a response such as log(y) in
log(y) ~ x, it wrote each regenerated response into a column named
log(y), which the refit did not read: the refit computed log(y) from y
again, so every replicate was the data of the fit, every replicate statistic
was the observed one, and the p-value was 1 whatever the data. For
lm(log(Ozone) ~ Solar.R + Wind + Temp, data = airquality), with
set.seed(1) and B = 999, all 999 replicate statistics were the observed
one, 34.715, and the p-value was 1; it is now 0.004, the value the same model
gives with log(Ozone) held in a column of its own, where
performWhiteTest() gives 0.00007. The regenerated response now goes into a
column of its own, which the refit reads in place of log(y). In
inst/validation/white-bootstrap-transformed-size.R, with constant variance
on the log scale, the test rejects 0.047 of the time at the 5% level at 5000
replications, inside the interval [0.037, 0.053] around 0.045, the rate at
which a test whose level is exact gives p < 0.05 with B = 199, and 0.892
against a standard deviation that grows with a regressor. Models with a plain
response give identical values.
fitWLS() on a model with an offset¶
fitWLS() keeps the offset of the model it refits, and changes value on a
model with one. It refitted the model with the estimated weights without the
offset, whether that was given as an argument, lm(..., offset = o), or
written in the formula, offset(o), so the weighted fit was that of a model
without it. For lm(dist ~ speed, data = cars, offset = 0.06 * speed^2) it
returned the coefficients -13.996 and 3.685; it now returns -4.356 and 2.038,
those lm() gives for the same call with the estimated weights. The weights
themselves do not change: they come from the residuals of the model, which
carry its offset. performVarianceFormTest() refits through the same code and
changes with it: on that model F = 0.290 (p = 0.593) before and F = 0.773
(p = 0.384) now. In the design of the guard added to
tests/testthat/test-fitwls-offset.R, an exponential variance function that
is correctly specified and the offset 2 x^2, it rejected 70% of the time at
the 5% level when the offset was a variable of the calling environment, which
the old code could find, and rejects 4.8% now. autoCompareRemediations() and
runHeteroTests(tests = "variance_form") change with them. Models without an
offset give identical values.
fitWLS() and performVarianceFormTest() also run on models they stopped on.
The weighted refit evaluated the call of the fit again in its model frame,
which holds log(wt) and not wt, and weights and an offset given as
arguments as (weights) and (offset). So
fitWLS(lm(mpg ~ log(wt), data = mtcars)) stopped with
object 'wt' not found, a model fitted with weights that are a column of the
data with object 'w' not found, and a model whose offset is a column of the
data with object 'o' not found. The refit now uses the design stored with
the fit. On a model fitted with weights, fitWLS() replaces them with the
estimated ones, as it did when the old refit could find them; those calls give
identical values.
The Szroeter test run by name¶
runHeteroTests(tests = "szroeter") changes value. The registered
Szroeter test ordered the observations by the second column of the data,
whatever it was: the response, a regressor or a column the model does not use,
so the result depended on the order of the columns. For
lm(mpg ~ wt + hp, data = mtcars) with mtcars as given it ordered by cyl,
which the model does not use, and gave Q = -0.709 (p = 0.761); depending on
which column came second, Q ran from -1.419 to 1.333. It now orders by the
fitted values, so the alternative is that the variance rises with the mean,
and gives Q = 1.099 (p = 0.136) whatever the order of the columns. With a
single regressor that is the order of the regressor when its slope is positive
and the reverse order when it is negative. The fitted values are computed from
the coefficients, so observations with the same regressors tie exactly, and
ties keep the order of the rows the model was fitted to.
Everything that runs the registered test changes with it: runDiagnostics(),
test() and summary() on a HeteroDiagnostic, compareModelDiagnostics(),
compareTestResults(), runSurveyHeteroTests(), runHeteroTestsParallel(),
cachedTest() and the dashboard, when given "szroeter".
performSzroeterTest() does not change, and no number recorded in
inst/validation/ moves: the studies call it with an explicit ordering.
Survey designs and their sampling weights¶
Survey designs are no longer tested as if their sampling weights were
precision weights, and runSurveyHeteroTests() says what it does.
The help page of runSurveyHeteroTests() described a survey-weighted fit. The
weights never reached the fit: they were passed to lm() as a local variable,
where lm() does not look, and a tryCatch() refitted without them, so the
function ran the ordinary tests on an ordinary least-squares fit. It still does, and apart
from the Szroeter test (above) its values are unchanged. The help page now says
that the design is not used and when the tests hold their level regardless,
and the function warns when the design has unequal weights or clusters.
runHeteroTests(formula, design) and runDiagnostics(formula, design) took a
different route. They fitted the model with survey::svyglm() and tested that
fit, whose residuals are multiplied by the square roots of the sampling
weights. Sampling weights say how many population units an observation stands
for, not how variable its error is. In inst/validation/survey-designs-size.R,
on homoscedastic data sampled with probabilities that depend on the regressor,
that statistic rejected 97% to 100% of the time; on heteroscedastic data it
rejected 6% to 12% of the time where the unweighted test rejected 98% to 100%.
These two calls change value: they now do what runSurveyHeteroTests()
does, with the same warning. A svyglm fit passed to a test directly is
refused.
The package has no design-based test of constant variance. The same study tried two, a Wald and a score test on the regression of the squared residuals on the regressors, estimated with the design. They are the only procedures near their level when selection depends on the response or the sample is clustered, where the function itself rejects 37% to 100% of the time. In the designs the ordinary tests handle correctly they rejected 6% to 25% of the time, and 3 of their 40 size cells fell inside the Monte Carlo band. They were not adopted.
tidy() of a test with two parameters¶
tidy() returns one row per test. A result with two parameters, such as
the F statistic of performVarianceFormTest() or the wild bootstrap with its
degrees of freedom and number of replications, came back as two rows, and
every table and plot built from the tidy output counted the test twice. The
second parameter is now in a new column, parameter2, and estimate is given
when a test reports exactly one. glance() follows.
Bootstrap tests on a weighted fit¶
performWildBootstrapTest(), performWhiteTestBootstrap() and
rbootstrap_test_statistic() accept a weighted fit. In 0.12.0 they refused
one, because each replicate was refitted without the weights. A weighted fit
is ordinary least squares on sqrt(w) y and sqrt(w) X, where the null
hypothesis is constant variance, and the three now resample there: they draw
from the Pearson residuals, divide each draw by sqrt(w) to return it to the
scale of its observation, and refit with the weights.
performWhiteTestRobust(bootstrap = TRUE) and
performBPTestRobust(bootstrap = TRUE), which resample through
rbootstrap_test_statistic(), accept a weighted fit too. Accepting a weighted
fit changes nothing for an unweighted one; there the same seed gives the same
values as before, except in the three cases corrected above: an offset given
as an argument, data in another order or with rows the fit dropped, and a
transformed response in the White bootstrap.
In inst/validation/weighted-fits-size.R, at 2000 replications with
B = 499, the three reject a correctly weighted fit 0.045 to 0.054 of the time
at the 5% level in two designs, and reject weights that are too weak in every
replication.
On a fitWLS() fit they over-reject. They hold the estimated weights fixed in
each replicate, as the asymptotic tests take them as known, and share their
error. In a design where the variance function is correctly specified and the
true weights give 0.054 with 150 observations and 0.048 with 600, the three
reject 0.064 to 0.086 of the time at the 5% level with 150 observations, and
the wild bootstrap 0.105 with 600; performKoenkerTest() rejects 0.077 and
0.110. Estimating the weights again in each replicate, which the package does
not do, brings the three to 0.042 to 0.060 with 150 observations and the wild
bootstrap to 0.051 with 600. That is why the question is open: ROADMAP.md
proposes an argument that does it, and leaves its default to 0.14.0. A default
that estimates the weights again will change the values these tests return on
a fitWLS() fit.
performQuantileRegressionTest() still refuses a weighted fit, and its help
page says why.
The Harrison-McCabe test¶
performHarrisonMcCabeTest() adds the test of Harrison and McCabe (1979)
beside performGQTest() and performSzroeterTest(), the other two tests
along an ordering of the observations. The statistic is the share of the
residual sum of squares that falls in the first part of the sample ordered by
a variable, and it agrees with lmtest::hmctest() to 1e-8. The p-value is
exact under normal errors given the regressors, by Imhof's (1961) formula, and
reproduces the exact critical values published by Harrison (1981). The 1979
article was not read: the definition rests on Harrison (1981), by its first
author, and on lmtest. lmtest::hmctest() simulates its p-value from a
regression on a constant alone, whatever the model, and
skedastic::harrison_mccabe() drops repeated eigenvalues before applying
Imhof's formula, which changes its p-value. Weighted fits are supported.
In inst/validation/harrison-mccabe-size-power.R, at the 5% level, the test
rejects a true null hypothesis 4.3% to 5.3% of the time under Gaussian errors
at 40 to 400 observations, with and without weights. Under t5 errors it
rejects 11.7% to 16.3%, as the Goldfeld-Quandt and Szroeter tests do, and the
help page points to performGlejserTest(robust = TRUE) there. Against the two
alternatives of Pass A it has 2.3 to 4.8 percentage points less power than the
Goldfeld-Quandt test and 5.4 to 7.9 less than Szroeter's. No value any other
function returns changes.
A Glejser test for skewed errors¶
performGlejserTest() gains robust = TRUE, the Glejser test of Im (2000)
and of Machado and Santos Silva (2000), which keeps its level when the errors
are skewed. No other R implementation of it was found.
Glejser's statistic has its nominal level when positive and negative errors
are equally likely. When they are not, and the variable tested is correlated
with the regressors, it does not in general, and a larger sample does not
help. In inst/validation/glejser-skewed-errors.R, at the 5% level and with
50 to 1000 observations, it rejected a true null hypothesis 12% to 14% of the
time under exponential errors and 15% to 18% under lognormal errors, which is
what the asymptotic theory gives for those designs. The help page said the
test was not valid there and pointed to Koenker's test.
With robust = TRUE the auxiliary regression is run on |e| - m e, where m
is the proportion of positive residuals less the proportion of negative ones.
In the same study it rejected 4.4% to 5.7% of the time from 150 observations
on, and 5.7% to 6.6% at 50 under skewed errors. Under Gaussian errors it gives
up one or two percentage points of power at 50 observations. Against Koenker's
statistic on the same regressor it has 1 to 4 points less power under Gaussian
errors and 1 to 10 points more under exponential errors.
The default is robust = FALSE, and every value the function returned before
is unchanged.
Measured and not adopted¶
Two of the candidate tests in ROADMAP.md were measured and are not in the
package, because they do not hold their level. The studies ship in
inst/validation/, and the roadmap says what either would need.
- The modified Wald test for group-wise heteroscedasticity in a fixed-effects
panel, as Stata's
xttest3computes it. Ininst/validation/modified-wald-size-power.R, at 5000 replications, it rejects a true null hypothesis at the 5% level 10% to 100% of the time in balanced panels of 10 to 100 units and 5 to 100 periods, under Gaussian andt5errors. - The consistent test of a variance function of Dette, Neumeyer and Van
Keilegom (2007), as a prototype that follows the authors' manuscript. In
inst/validation/variance-form-dnvk-size-power.R, at 5000 replications, its Cramér-von Mises statistic rejects a true exponential or power variance function that is not constant 1.9% to 3.6% of the time at the 5% level, at 50 to 400 observations under Gaussian andt5errors.
Also¶
-
listDiagnostics()returns the namesrunHeteroTests(tests = )accepts, sorted: the diagnostics the package registers and any added withregisterDiagnostic(). No exported function listed them before. Given a name that is not registered,runHeteroTests()now stops withUnknown tests: <name>. listDiagnostics() returns the registered names.It used to say onlyUnknown tests: <name>, in an error raised from the internal.run_hetero_core(). -
runHeteroTests()with a fitted model and grouped data now runs. The per-group refit passed the weights tolm()as a local variable, wherelm()does not look, and stopped withinvalid type (closure) for variable '(weights)'for everylm,glmand parsnip fit, weighted or not. Only a formula with grouped data worked. Each group is now fitted by evaluating the model's own call on it, so a weighted model uses that group's rows of the weights, aglmkeeps its family and asubsetis applied within the group. A model whose weights, offset or subset are vectors in the workspace cannot be split by group and is refused with a message that says so. -
autoplot()of a test suite showed nothing when the tests rejected. The bars were the p-values on a linear axis from 0 to 1, so a test with p = 0.002 had no visible bar, and the colouring of significant tests was decided byisTRUE(p < 0.05), which isFALSEfor a suite of more than one test. The plot now draws one horizontal bar per test whose length is-log10(p), on an axis labelled with p-values, with a dashed line at a newalphaargument (default 0.05) and the p-value printed beside each bar. The grouped method draws one panel per group; it used to add a panel variable for thesuggestionscolumn as well. -
plotBeforeAfter()draws the two models in two panels, each with its own axes, where it used to overlay them in one panel in two similar blues. Weighted residuals and the residuals of a transformed response are not in the units of the original ones, so a common axis could flatten one of them. -
print()on the result ofsuggestRemediation()now uses its print method. The method existed but was not registered inNAMESPACE, so results printed as a bare list with anattr(,"class")line. This also affects the dashboard's suggestions panel and the remediation section of generated reports, which print the same object. -
suggestDiagnosticsForProfile()had two definitions, and only the one loaded last ever ran. The other still recommended"hc_covariance", a diagnostic removed in 0.8.0, for data of 10,000 rows or more; had it been the one loaded, passing its recommendations torunHeteroTests(tests = )would have stopped withUnknown tests: hc_covariance. It is gone, and the function returns exactly what it returned before. New tests check that every name it can return is a registered diagnostic, and that no function in the package has two different definitions. -
The cache that keeps the result of each input validation, so that the same model and data are not checked twice, now keeps the 1000 most recent results. It kept every one, and a test on a new model, as in a simulation, never reuses one, so it grew by three entries with each test: after 10^5 calls of
performWhiteTest()on new models with 150 observations it held 300,001 entries and R's memory had grown by 655 MB, where it now holds 1000 and has grown by 3 MB. Simulation workers had reached 7.6 to 8.9 GB each. A test repeated on the same model still reuses the result, and no value changes. -
The help page of
performStudentizedBPTest()calledperformKoenkerTest()"the absolute-residual variant", which it has not been since 0.6.5. Both compute Koenker's studentizedn R^2statistic from the regression of the squared residuals on the regressors, and the page now says so. It also cited a section of Davidson and MacKinnon (2004) that does not exist; the test is in Section 7.5. The vignette on statistical theory gave the classical Breusch-Pagan statistic asn R^2, which is Koenker's, and said thatperformKoenkerTest()uses the absolute residuals; it now gives each statistic as the package computes it. No value changes. -
The help page of
performWhiteTestBootstrap()gave its p-value as the share of replicates whose statistic is at least the observed one. Since 0.7.0 the function has returned(1 + #{T_b >= T_obs}) / (B_eff + 1), whereB_effcounts the replicates with a finite statistic, and the page now says so. Its example ofparallel = TRUEnow runs only where that starts at most two processes, the limit ofR CMD check --as-cran. No value changes. -
The release workflow runs
R CMD check --as-cran --run-dontteston the source tarball it is about to tag, and creates no tag or GitHub release if that check warns or errors. The check used to run only weekly and on demand, and it failed on the example above on 2026-09-28 and 2026-10-05 without holding up 0.12.0.
0.12.0¶
Tests on weighted fits change value, and the package gains a test of the shape
of the variance. No test changes value on an unweighted fit: re-running the
full sweep reproduces its 32 existing rows digit for digit. fitWLS() with its
default arguments returns the weights it returned in 0.11.2 for any model with
an intercept.
Tests on a weighted fit read the wrong residuals¶
lm(..., weights = w) states that the error variance is sigma^2 / w_i. The
raw residuals of such a fit are heteroscedastic by assumption, and every test
read them. A model fitted with exactly the right weights was therefore
rejected as often as the same model fitted with none, and the workflow the
README and the vignettes describe -- diagnose, refit with fitWLS(), test
again -- could not succeed.
inst/validation/weighted-fits-size.R measures it: data with sd = x^2,
150 observations, a model fitted with weights 1 / x^4, 400 replications. Run
against 0.11.2, the 27 heteroscedasticity diagnostics that ran all rejected in
400 samples out of 400, and three could not run at all. The 27 that accept a
weighted fit now reject between 2.2% and 7.7% of the time, against a Monte
Carlo standard error of 1.1%. A fit from fitWLS() fared no better: with the
variance function correctly specified, performKoenkerTest() on 0.11.2
rejected the weighted fit 99.9% of the time.
The tutorial vignette showed the fault on real data. For quakes,
compareModelDiagnostics(list(base_model, wls_model)) reported a White
statistic of 125.6 for the least-squares fit and 249.2 for the weighted one,
and Breusch-Pagan statistics of 191.9 and 463.7, directly above the sentence
"The weighted least squares fit typically reduces the heteroscedasticity
metrics." It now reports 42.2 and 40.8 for the weighted fit.
On a weighted fit the tests are now computed from the Pearson residuals
sqrt(w_i) * e_i. Those are the residuals of the equivalent unweighted
regression of sqrt(w) y on sqrt(w) X, and they have constant variance when
the weights are right. car::ncvTest() and plot.lm() make the same choice.
The null hypothesis becomes that the weights are adequate.
- Every residual-based test reads the Pearson residuals. Variance regressors, grouping variables and ordering variables stay on their original scale.
performNCVTest()andperformCookWeisbergTest()now reproducecar::ncvTest()on a weighted fit to1e-8. They did not before.performGQTest()refits each segment with its weights and compares weighted residual sums of squares. It reproduceslmtest::gqtest()on the transformed regression to1e-8atfraction0.1, 0.2 and 0.3.performRESETTest()fits the augmented model by weighted least squares. Before, it compared a weighted restricted fit with an unweighted augmented one, which are not nested, and rejected 23.0% of the time under the null in the design above. It now rejects 5.5%.performArchLMTest()andperformMcLeodLiTest()read a variance that rises along the sample as volatility clustering, whatever the weights: 72.3% and 79.7% on a correctly weighted fit. Now 4.0% and 4.8%.performStudentizedBPTest()used the raw residuals and weighted the auxiliary regression instead, the treatment inlmtest::bptest(), which tests the raw residuals and is therefore not a reference for weighted fits. It now agrees withperformKoenkerTest()on weighted fits as it does on unweighted ones.performStudentizedBPTest(),performBPTestRobust()andperformWildBootstrapTest()failed withobject 'w' not foundwhenever the weights were a column of the data. They rebuilt the design matrix from the model frame, where that column is stored as(weights).- The residual plots (
plotResidualsFitted(),plotSpreadLevel(),plotBeforeAfter()and the others) and the variance-pattern summary of the recommendation engine use the Pearson residuals of a weighted fit.plotBeforeAfter()drew a weighted remedy as heteroscedastic as the original however good the weights were. performWildBootstrapTest(),performWhiteTestBootstrap(),rbootstrap_test_statistic()(and sobootstrap = TRUEin the two*Robusttests) andperformQuantileRegressionTest()refit the mean model without the weights. They now refuse a weighted fit, asrbootstrap_test_statistic()already refused aglm, instead of returning a statistic for a different model.- A fit with a zero weight is refused: the observation is not part of the fit and its Pearson residual is an artificial zero.
Weights that were estimated are not weights that are known¶
The reference distributions take the weights as given. That is exact for
weights known in advance and approximate for a fitWLS() fit, whose weights
come from the same residuals, and the approximation does not improve with the
sample size. With the variance function correctly specified and Gaussian
errors, at the 5% level (2000 replications, standard error 0.5%):
Test on a fitWLS() fit |
n = 150 | n = 600 |
|---|---|---|
performKoenkerTest() |
7.7% | 11.0% |
performWhiteTest() |
6.3% | 8.3% |
performBPTest() |
10.6% | 11.8% |
performHarveyTest() |
0.0% | 0.0% |
performVarianceFormTest() |
4.8% | 5.2% |
Harvey's test never rejects because it repeats the regression the weights were
estimated from. The statistics remain useful for comparing a weighted fit with
the original, which is what compareModelDiagnostics() does, but their
p-values on a fitWLS() fit are not calibrated. The help page of
performKoenkerTest() says so, and the last row is the test built for that
case.
performVarianceFormTest(): is the variance function the right shape?¶
Every other test in the package has constant variance as its null hypothesis. Rejecting it says that the variance moves with some variables, not whether an exponential or a power function of them describes it. The new test takes a variance function as its null hypothesis.
It fits the variance function, refits the model by weighted least squares, and
regresses the squared standardized residuals on the regressors of the variance
function and on terms added to them: by default their squares and pairwise
products, or the terms of a one-sided formula given as against. The
statistic is the F test of the added terms, the regression-based construction
of Wooldridge (1990, 1991). Keeping the variance regressors in that regression
is what makes the test valid with estimated weights, and it is also where the
information is. A Koenker test on the weighted fit asks about directions the
estimation has already fitted: with variance x^3 fitted as exponential, at
600 observations, it rejected 6.9% of the time, less often than the 11.0% it
rejects when the fitted form is right. The new test rejected 92.9%. The White
test, which also contains squares, rejected 87.9%, but its level on a
fitWLS() fit is not 5%.
inst/validation/variance-form-size-power.R runs both forms against five
true variance functions at 50, 150 and 400 observations with Gaussian and
t_5 errors, 5000 replications a cell:
- When the form under test is the true one, or the variance is constant, the rejection rate runs from 4.2% to 5.7% over the 24 cells, and all of them are inside the release-gate interval [0.042, 0.058].
- Against a log-variance that is quadratic in
x, power is 51.7% at 50 observations and 98.0% at 150 for the exponential form. - Against the other log-linear form, power grows more slowly: 22.4% at 150
observations and 74.1% at 400 for an exponential null and variance
x^3. - Against
sigma^2 = 0.5 + x, which is close to both forms over the range ofx, it rejects between 4.3% and 6.6% of the time. The test cannot separate shapes the data cannot separate, and the help page says so.
The test accepts either the unweighted model with var_formula and form, or
a fitWLS() result, whose variance function it then tests. It is registered
as "variance_form" for runHeteroTests().
fitWLS() fits the variance function it is asked for¶
fitWLS(model, data = NULL, var_formula = NULL, form = c("exponential",
"power")). var_formula names the variance regressors and form the shape:
exponential, Harvey's (1976) log sigma^2 = g0 + z'g, or power, Park's (1966)
sigma^2 proportional to z^d. The defaults reproduce 0.11.2. The fit
carries the variance function as its "variance_function" attribute, next to
the "variance_model" attribute it already had.
One value changes. The variance function now always has its own constant.
fitWLS() regressed log(e^2) on the model's design matrix, so for a mean
model without an intercept the log-variance had no constant either, which
pins the variance at 1 where the regressors are zero. Models with an intercept
are unaffected.
Withdrawn before release¶
An additive variance function, sigma^2 = a + z'b, was implemented as a third
form and removed. Nothing keeps its fitted variances positive. Under its own
null, at 5000 replications, the test rejected 6.1% of the time with 150 and
with 400 observations, outside the release gate, and the fitted variances were
not all positive in 17% of samples at 50 observations and 6% at 150. It needs a
constrained estimator before it can be offered.
Also¶
- The Park (1966) reference pointed at another paper. Its DOI is
10.2307/1910108and it is a one-page note, Econometrica 34(4), 888. The help page cited10.2307/1909774, which is Henshaw (1966), with pages 888-898, and the theory vignette gave 888-908. tests/testthat/test-weighted-fits.Randtest-variance-form.Rpin the corrections by simulated size and power as well as by reconstruction, since no stored value and no comparison withlmtest::bptest()would have caught the fault.- The full sweep covers
performVarianceFormTest(): size 5.8%, power 97.8% against a log-variance quadratic inx. - The tutorial vignette gains a step that tests the variance function of its weighted fit.
0.11.2¶
Changes requested by the CRAN review of the 0.11.1 submission. No statistic, default or exported function changes.
- The
Descriptionfield no longer says "in R", and now cites the methods the package implements, with DOIs: White (1980), Breusch and Pagan (1979), Koenker (1981), Goldfeld and Quandt (1965), Harvey (1976), Cook and Weisberg (1983), Engle (1982), Bartlett (1937), Brown and Forsythe (1974) and Hartley (1950). - No example reaches an unexported object through
:::. The six help pages that did -- forrhandleMissingValues(),rTEST_REQUIREMENTS,rvalidateDataInputs(),rvalidateDistributionalAssumptions(),rvalidateGroupingVariable()andrvalidateModelInputs()-- document internal objects, where::would fail. Their examples are removed rather than the objects exported. -
No example uses
\dontrun{}. The recommendations example runs in well under a second and is unwrapped. The report example takes about ten seconds and needspandoc, so it runs under\donttest{}behindrmarkdown::pandoc_available()and writes totempdir(). The dashboard example is guarded byinteractive(), since printing the Shiny application it returns starts a blocking server. -
The
\donttest{}part of theperformWhiteTestBootstrap()example uses 499 bootstrap replications rather than 999. With that part included, asR CMD check --as-cranruns it, the example took 5.0 to 5.7 seconds, over the five-second limit for examples; it now takes about 1.7 seconds.
0.11.1¶
Two panel p-values change value, though not their conclusions.
The panel p-values no longer underflow to zero¶
performBPRandomEffectsTest() and performPesaranTest() computed their upper
tails as 1 - pchisq() and 1 - pnorm(). Both collapse to exactly zero once
the statistic is large: at LM = 182.9 the subtraction returns 0 where the
upper tail returns 1.1e-41, which is the value plm::plmtest() reports. A
p-value of exactly zero is wrong, and it discards the magnitude that anything
working on a log scale or applying a multiplicity correction needs. Both now
pass lower.tail = FALSE.
Both are now checked against plm¶
Neither statistic had a reference implementation among the packages the
accuracy table compares against, which is why both were wrong until 0.11.0 and
why simulated size was the only thing that caught them. plm has had both all
along.
performBPRandomEffectsTest() reproduces plm::plmtest(type = "bp") exactly,
and performPesaranTest() reproduces plm::pcdtest(model = "pooling"), both
to 1e-8 across N = 20/30/40 and T = 5/6/8.
The model argument is the part worth recording. pcdtest() defaults to
model = NULL, which tests the series rather than any model's residuals and
gives 0.817 where this package gives 1.580. performPesaranTest() takes a
fitted lm, whose residuals are identical to the pooling residuals, so
"pooling" is the comparison that puts both on the same quantity; "within"
and "random" give 1.220 and 1.258, which are different estimands rather than
disagreements.
plm and withr join Suggests. The tests attach plm with
withr::local_package() rather than calling through ::, because pcdtest()
calls plm() internally without qualifying the namespace and fails when the
package is only installed.
Continuous integration¶
R CMD check --as-cranno longer runs on every merge. CRAN submission is not the near-term target, and that job costs about an hour: it installs a LaTeX toolchain and every check dependency, then rebuilds the vignettes. It had also stopped completing -- three consecutive runs onmainwere cancelled or left queued inside it -- so it was a gate producing a signal nobody received. It now runs weekly and onworkflow_dispatch.- A concurrency group is added, keyed by pull request number for pull requests
and by commit for everything else. A ref-keyed group would have put pushes,
the weekly schedule and manual dispatches on
maininto one queue, and GitHub keeps only one pending run per group, so a third arrival would cancel the pending one.
0.11.0¶
Two panel statistics were wrong against their published definitions. Both change, hence the minor version.
Where these came from¶
A size and power sweep over all 32 exported perform*Test() functions, each
driven by a null and an alternative appropriate to what it actually tests.
Twenty-six of the twenty-eight heteroscedasticity tests hold their nominal
level: at 400 replications and n = 150, size ranges from 3.0% to 6.2% against a
Monte Carlo standard error of 1.1%, with power between 95% and 100% against
sd = x^2. The two below did not, and neither has a reference implementation
among the packages the accuracy table compares against, so only simulation
would have found them.
performBPRandomEffectsTest() rejected a third of the time under the null¶
The statistic was
T^2 / (2(T - 1)) * sum_i (sum_t e_it)^2 / sum_it e_it^2
against Breusch and Pagan's (1980) equation 5,
nT / (2(T - 1)) * [ sum_i (sum_t e_it)^2 / sum_it e_it^2 - 1 ]^2
The bracketed ratio is close to one under the null, and the published statistic
measures its squared departure from one. Dropping the - 1 and the square
leaves a quantity that sits at T^2 / (2(T - 1)) -- 3.6 for T = 6, against a
chi-square(1) critical value of 3.841. Measured median statistic 3.548, and
size 32.5% against a nominal 5%. It is now 5.8%, with power 100% against random
intercepts.
The documentation was wrong in the same place. The help page was titled a test
for random effects but described a test for "heteroscedasticity in random
effects models", and the roxygen claimed it tested whether the effect's
variance varies across units. It tests whether an individual effect is present
at all, and does not respond to heteroscedasticity: against errors with
sd = x^2 the corrected statistic rejects 3.2% of the time, which is the
nominal level rather than power.
performPesaranTest() never rejected¶
The statistic was sqrt(N(N-1)/(2T)) times the mean pairwise residual
correlation, against Pesaran's (2004, 2015)
CD = sqrt( 2T / (N(N-1)) ) * sum_{i<j} rho_ij
which is the same quantity multiplied by T. The implemented version was
therefore exactly 1/T too small: at T = 8 its standard deviation was 0.133
where the published form gives 1.067, and since the reference distribution is
standard normal, size was 0.0%. It is now 6.5%, with power 100% against a
common time factor.
The pairwise correlations are also computed with a single cor() call rather
than a double loop that grew a vector by c().
Also¶
performRESETTestwas checked in the same sweep and is correct: size 5.2%, power 100% against an omitted quadratic term.performGQTestreproduceslmtest::gqtest()exactly at matchedfraction-- the difference is zero at 0.1, 0.2 and 0.3. It cannot reproduce that function's default, becauselmtestomits no central observations by default andperformGQTest()requiresfractionstrictly between 0 and 1.tests/testthat/test-panel-statistics.Rpins both corrections by simulated size rather than by a stored value, since a stored value would not have caught either fault.
0.10.0¶
rbootstrap_test_statistic() now resamples under the null. Its p-value
changes, and so does the p.value_bootstrap field of the two robust tests,
hence the minor version.
The bootstrap p-value could not detect anything¶
The helper resampled rows of the data with replacement, refitted, and recomputed the statistic. Those replicates follow the statistic's distribution under whatever variance structure the data actually have, not under the null: they are centred on the observed statistic, so comparing the two returns roughly one half however strong the heteroscedasticity is.
Measured over 400 samples at n = 120, B = 199, testing Koenker's statistic:
| procedure | size, homoscedastic | power, sd = x^2 |
|---|---|---|
| pairs resampling (before) | 0.0% | 0.0% |
| null-imposed (now) | 5.0% | 100% |
performWildBootstrapTest() |
2.5% | 100% |
| Koenker, asymptotic | 5.0% | 100% |
Zero rejections under a strong alternative. The p-value was exported directly
and also reached users as p.value_bootstrap from performWhiteTestRobust()
and performBPTestRobust() when called with bootstrap = TRUE.
resample now takes "null", which is the default. The response is
regenerated as fitted + e*, with e* drawn with replacement from the
residuals after dividing by sqrt(1 - h_i) and centring, so the regenerated
data satisfy homoscedasticity by construction. Size is 5.0% (Monte Carlo
standard error 1.1%) and power is 99.2% against sd = 0.5x and 100% against
sd = x^2. Size holds at 5.0% under homoscedastic t_5 errors as well, so
the calibration does not depend on Gaussian tails -- which matters here,
because Bartlett's and Hartley's tests reject about 24% of the time against
the same heavy-tailed null (see inst/validation/README.md).
The leverage correction is not decorative. OLS residuals have variance
sigma^2 (1 - h_i), so resampling them unscaled under-disperses the
regenerated errors; a version without it rejected about 13% of the time under
the null. This is the construction performWildBootstrapTest() already used.
resample = "pairs" is still available and still returns the replicates and
the interval, which describe the statistic's variability perfectly well. It now
returns NA for p_value rather than a number that cannot be interpreted.
Null-imposed resampling is restricted to a Gaussian lm whose response is a
plain variable, and errors otherwise. Two cases previously ran and returned a
plausible p-value from replicates that meant nothing: a transformed response
such as log(y) ~ x, where fitted() is on the log scale and the data column
holds y, so refitting took the logarithm twice and returned p = 1; and a
glm, where the refit used safe_lm() and dropped the family and link,
returning p = 0.952 for a Poisson model. resample = "pairs" resamples rows
and is unaffected by the transformed-response case, so it remains available
there.
A glm is now refused by both strategies. Each replicate is refitted with
least squares whichever strategy is used, so on a Poisson fit of counts the
coefficients move from (0.457, 0.406) on the log link to (-0.931, 2.281) on the
identity scale: the replicates describe a different model from the one passed.
The helper documented glm support it never had.
Percentile intervals are not confidence intervals¶
The ci component was documented as a "percentile confidence interval for the
statistic". It is a percentile interval of the bootstrap distribution: it
summarises where the resampled statistic falls, is not an interval for a
parameter, and carries no coverage guarantee. The documentation says so. This
closes the corresponding roadmap item.
0.9.0¶
fitWLS() now estimates its weights from a variance model. The numbers it
returns change, hence the minor version.
fitWLS() produced unusable standard errors¶
The weights were the inverse squared residuals of the initial fit,
w_i = 1 / e_i^2. A squared residual is a one-degree-of-freedom estimate of
sigma_i^2, far too noisy to invert, and inverting it hands the weight to
whichever observations the first fit happened to reproduce most closely. On
quakes that meant five of a thousand points carrying 99.5% of the total
weight, with a largest-to-median weight ratio of about 1e+07.
Two consequences, measured over 2000 replications of
y = 1 + 2x + e, sd(e) = 0.5x, at n = 200:
| estimator | SD(beta1) | coverage of nominal 95% |
|---|---|---|
| OLS | 0.1027 | 93.7% |
fitWLS() before |
0.1022 | 10.4% |
fitWLS() now |
0.0837 | 96.1% |
| oracle weights | 0.0827 | 95.1% |
The point estimate was unbiased but no more efficient than OLS, so the function
delivered none of the efficiency that is the purpose of weighting; and its
intervals covered the truth about a tenth of the time, because the weighted
residual sum of squares collapses towards n and the reported sigma is
approximately sqrt(n / (n - p)) whatever the data.
fitWLS() now regresses log(e^2) on the model's own design matrix and weights
by 1 / exp(fitted), the standard feasible-GLS recipe. That recovers the
efficiency (SD 19% below OLS, close to the oracle) and calibrates the
intervals. Zero residuals are floored before the logarithm by the same helper
the log-variance tests use, and an unusable variance model falls back to equal
weights, which degrades to the original OLS fit rather than failing.
The estimated variances are attached to the result as the "variance_model"
attribute.
Knock-on corrections¶
compareModelDiagnostics()can now run White's test on a WLS fit. It previously reportedNAthere, correctly, because the degenerate weighting made the weighted fit perfectly explained; that cell has now held three different things across three releases and the test records all three.autoCompareRemediations()compares AIC and RMSE across OLS, WLS and robust fits. The WLS row was computed from the degenerate fit and is now meaningful.- The WLS section of
real_world_case_studies.Rmdno longer needs its warning that the weighted variance profile was flat by arithmetic rather than by fit. Onquakesthe profile now falls from a 6.6-fold spread across thirds of the fitted range to 2.95-fold, which reflects the data.
Metadata¶
- The maintainer affiliation is now recorded identically in
CITATION.cff,inst/CITATION,.zenodo.json,README.mdand the paper. Only.zenodo.jsoncarried the current name; the rest still had the former one.
0.8.1¶
CRAN preparation. No change to any statistic or to the public API.
The bundled Boston dataset is gone¶
boston_housing was a byte-identical copy of MASS::Boston -- all 506 rows and
all 14 columns. Shipping it under this package's Apache License (>= 2.0) put a
verbatim copy of GPL-2 | GPL-3 material under Apache terms, which is at best a
question a CRAN reviewer would ask, and cran-comments.md volunteered it.
The justification recorded there -- that the copy let examples run "without
attaching MASS" -- did not hold either: MASS is a hard Imports dependency, so
it is installed regardless.
Examples, vignettes and the tutorial notebooks now use R's built-in quakes
data, which lives in datasets and is attached by default. Nothing is
redistributed, so the licensing question disappears rather than being argued.
The two datasets the package still ships, diagnostic_data and hetero_data,
are simulated.
The replacement was chosen on evidence, not convenience. Against
stations ~ mag + depth (n = 1000, R-squared 0.74) White gives p = 2e-25,
Breusch-Pagan p = 2e-42 and Koenker p = 7e-25, and the HC3 standard error for
mag is 1.34 times the OLS one -- enough for the notebooks' robust-versus-
classical comparison to keep its point. The response is a count, so its variance
rises with its mean: the textbook mechanism, which makes it a better teaching
example than the original. Candidates with stronger p-values (ChickWeight,
Theoph, DNase, Loblolly, Orange) were rejected because they are repeated-measures
data, where residuals are correlated within subject and these tests' independence
assumption does not hold.
Dropping the dataset also drops the black column, the transformed
racial-composition variable that led scikit-learn to remove this data in 1.2.
Packaging¶
- Three references used
\url{https://doi.org/...}where CRAN asks for\doi{}, the form the rest of the package already used. Fixed inperformStudentizedBPTest,performSzroeterTestandperformWhiteTestBootstrap, and in the roxygen sources so a regeneration produces the same form. - README no longer lists two limitations that were fixed in 0.8.0: the
renv.lockdrift and the bundled dataset.
0.8.0¶
The API review that the validation effort was sequenced towards. Six exports are removed. This is a breaking change, hence the minor version.
Three returned nothing but a migration error, having been withdrawn once their statistics were shown not to work:
| removed | use instead |
|---|---|
performRiceTest() |
performSzroeterTest(), performGQTest() |
performCurryWalshTest() |
performSpatialHeteroTest() |
performHCCovarianceTest() |
performBPTest(), performKoenkerTest(), performWhiteTest(); sandwich::vcovHC() for robust covariance |
Three computed a valid statistic under a name that promised a different method, duplicating a test that remains:
| removed | identical to |
|---|---|
performOrderedLMTest() |
performKoenkerTest() -- sorting the rows and refitting returns the same residuals permuted, so its order_by argument could not affect the result |
performCameronTrivediTest() |
the e^2 ~ yhat + yhat^2 auxiliary F test, covered by performWhiteTest() and performNCVTest(); it was never Cameron and Trivedi's information-matrix test |
performModifiedBartlettTest() |
performBartlettTest() -- the correction factor its documentation called a modification is part of the standard definition |
The package has not been released on CRAN, so no published contract is broken.
tests/testthat/test-public-api.R records the removals and checks that every
replacement named above is still exported.
Two dead fallback routes went with them: hc_covariance was registered as a
recovery path for quantile_regression and high_dimensional, and
hc_covariance_hc0 as a custom fallback, both calling a function that only
ever raised an error.
Packaging¶
- The pkgdown workflow now deploys. It called
build_site_github_pages(), which writesdocs/, and then stopped: there was no deploy step and nogh-pagesbranch, so the job reported success while nothing was published. That is why the URLs inDESCRIPTIONandinst/CITATIONreturned 404. renv.locknow records all 38 declared non-base dependencies, up from 14.renv::restore()previously pinned about a third of them, and the Docker build only worked becauseremotes::install_local()resolves the remainder from CRAN at whatever version is current. The R version moves to 4.5.1, which is where the recorded versions come from.
0.7.2¶
Pass C of the statistical validation matrix: the methods with the least reference coverage. Six were sound and one needed a fix. Two are withdrawn because their statistics cannot detect heteroscedasticity at all, and two more are valid statistics under names that promise something else.
Withdrawn¶
-
performRiceTest()compared Rice's (1984) difference-based variance estimator with the mean squared residual and referred the ratio to an F distribution. For independent residualsE[(e_i - e_{i-1})^2]iss2_i + s2_{i-1}, so the numerator estimates the mean of the variances and so does the denominator: the ratio sits at one under any variance pattern. Simulated means were 1.00 under homoscedasticity, undersigma = 0.2 + 1.2x, undersigma = exp(x)(a fiftyfold spread) and under a twentyfold step change, and empirical size was 0.000. Adding an ordering argument does not repair it. UseperformSzroeterTest()orperformGQTest()instead. -
performCurryWalshTest()passed an unstandardised Moran's I to2 * (1 - pnorm(abs(I))). Moran's I has null expectation-1/(n-1)and a weights-dependent variance, and is bounded near one, so it essentially never reached 1.96: it rejected in 0.0% of samples under strong heteroscedasticity and returnedp = 0.875on clustered variance.performSpatialHeteroTest()already does this correctly viaspdep::moran.mc().
Both retain their exports and signal a migration error, following the withdrawn HC covariance precedent.
Corrected¶
performDavidianCarrollTest()failed withNA/NaN/Inf in 'y'when a residual was exactly zero. It now uses the shared log-residual floor, as Harvey and Park do.
Documented rather than removed¶
performOrderedLMTest()sorts the rows and refits, which returns the same residuals permuted, soorder_bycannot change the statistic and the result is identical toperformKoenkerTest(). It now warns that the argument has no effect.performCameronTrivediTest()is exactly thee^2 ~ yhat + yhat^2F test, not the information-matrix test of Cameron and Trivedi (1990), which concerns overdispersion in count models. The help page now says what it computes.
Removing either redundant export is left to the API review.
Validated without change¶
performRankPermutationTest(), performHighDimensionalTest(),
performWildBootstrapTest(), performQuantileRegressionTest(),
performSpatialHeteroTest() and performPesaranTest() hold their nominal level
and have power.
0.7.1¶
Pass B of the statistical validation matrix: the group-variance tests. This section was missing from the changelog when 0.7.1 shipped and is recorded here.
Statistical corrections¶
-
Bug fix (critical):
performOBrienTest()rejected on every input. O'Brien's transformation produces one score per observation, whose group mean is that group's variance. The implementation computed a single value per group and repeated it across that group's observations, so the scores had no within-group variability: the one-way ANOVA had a residual sum of squares of zero, givingFon the order of1e30andp = 0for any data, homoscedastic or not. Empirical size is now 4.8%. -
Bug fix (critical):
performHartleyFmaxTest()used the wrong null distribution. The statistic is the largest ofkgroup variances over the smallest, but the p-value came frompf(F, df, df), the distribution of a single variance ratio; the number of groups was computed and never used. Empirical size at a nominal 5%, groups of 30, was 0.0998 at two groups, 0.2238 at three, 0.3497 at four and 0.5626 at six. It now uses the maximum-F-ratio distribution and holds 0.05 throughout. -
performOBrienTest()also calledsafe_lm()without a data argument, so every call errored once the transformation was corrected.
Validated without change¶
performLeveneTest(), performBrownForsytheTest(), performBartlettTest() and
performFlignerKilleenTest() reproduce car::leveneTest() with mean and median
centring, stats::bartlett.test() and stats::fligner.test() to within 1e-8.
Documentation¶
performModifiedBartlettTest()is Bartlett's test under another name: the correction factor its documentation described as a modification is part of the standard definition. It is now documented as a compatibility alias.- Bartlett's per-group minimum was relaxed from three observations to two, which is what a sample variance requires.
0.7.0¶
This release is the first pass of a package-wide statistical validation effort.
Pass A covers the classical regression diagnostics. Every test in this group was
checked against an established implementation where one exists, and otherwise
against a reconstruction of the primary reference, followed by Monte Carlo size
and power measurement. Reference equivalence is asserted in
tests/testthat/test-pass-a-reference.R; the simulation evidence is reproducible
via inst/validation/pass-a-size-power.R and recorded in
inst/validation/pass-a-size-power.csv.
Several of the corrections below change the value a test returns. They are breaking changes, and the minor version has been bumped accordingly.
Statistical corrections¶
-
Bug fix (critical):
performSzroeterTest()was standardised incorrectly and had no power. Szroeter's (1978) rank-weighted statistich = sum(i * e_(i)^2) / sum(e_(i)^2)has null mean(n + 1) / 2and null variance(n^2 - 1) / (6n). The previous implementation rescaledhby the mean rank and then divided the centred statistic by a furthersqrt(n), using an ad-hoc variance(n + 1)(2n + 1) / (12n). The net effect was to shrink the standardised statistic by a factor of roughly2 / sqrt(n)— a factor of 5 atn = 100. In 5,000 replications the test rejected in 0.0% of samples under the null and under every alternative tried: it could not reject at any sample size or effect size. The statistic is now Szroeter'sQ, which attains 4.5% size and 87–91% power in the same designs. The test also gained analternativeargument ("greater", the default, for variance increasing with the ordering variable), and reportshinestimate. -
Bug fix:
performNCVTest()did not compute the NCV test. It regressed the absolute residuals on the fitted values and reported a t statistic — a Glejser-type diagnostic — while documenting itself as equivalent tocar::ncvTest(). It now computes the Cook–Weisberg score statistic: half the explained sum of squares from regressinge^2 / (RSS / n)on the variance model, referred to a chi-squared distribution. It reproducescar::ncvTest()to within1e-8. A newvar_formulaargument selects a variance model other than the fitted values, matchingcar::ncvTest(var.formula = ). -
Bug fix:
performCookWeisbergTest()returned the Koenker statistic. It computedn R^2from regressing the raw squared residuals on the fitted values — that is, Stata'sestat hettest, iid fitted— while documenting itself as Stata'sestat hettest, fitted. It now computes the Cook–Weisberg score statistic and delegates toperformNCVTest()so the two exports cannot drift apart. Note the trade-off this exposes: the score test assumes normal errors, and in simulation it over-rejects under heavy tails where the previous (mislabelled) statistic did not.performKoenkerTest()remains the kurtosis-robust choice and is now cross-referenced from both help pages. -
Change:
performHarveyTest()now follows Harvey (1976). The variance regressors default to the model's own explanatory variables rather than the fitted values and their square, and the statistic is the classicalESS / (pi^2 / 2)referred to a chi-squared distribution, wherepi^2 / 2is the null variance oflog(chi^2_1). The previous behaviour is available asauxiliary = "fitted", and the studentized F form — which estimates that variance from the data instead of assuming normality, and holds size better under heavy tails — asstudentize = TRUE. -
Harvey and NCV now take their degrees of freedom from the realised rank of the auxiliary design, so a rank-deficient variance model is not credited with degrees of freedom for columns that
lm()aliased away. -
performHarveyTest()andperformParkTest()now warn when a residual is numerically zero and has been floored before taking logarithms. A floored observation contributeslog(.Machine$double.eps) ~ -36, roughly sixteen standard deviations below the mean of the nulllog(chi^2_1)distribution, and can dominate the auxiliary regression on its own. -
performParkTest(),performGlejserTest(),performNCVTest()andperformCookWeisbergTest()now name theirparameterelement, soprint.htest()labels the degrees of freedom instead of printing= 118.
Validated without change¶
performArchLMTest() and performMcLeodLiTest() were audited and required no
change. ARCH LM reproduces Engle's T R^2 auxiliary regression with T = n - q
(the FinTS::ArchTest() convention) and McLeod–Li reproduces the Ljung–Box
statistic on squared residuals with df = m, both to within 1e-10, with
empirical size of 4.4% and 5.6% and power of 99% and 97% against an ARCH(1)
alternative.
Diagnostic workflow¶
-
Bug fix:
compareModelDiagnostics()could report one test's statistic under another test's name. When a requested diagnostic fails,runHeteroTests()substitutes a fallback diagnostic and tags the result with the substitute's name.compareModelDiagnostics()read the statistic without checking that tag, so a comparison table column headedwhitecould contain, for example, an NCV statistic. It now reportsNAand warns when a substitution occurred, since a numeric table has nowhere to carry the caveat.This masking is also why two existing tests changed expectations. Both were passing because the previously unvalidated
performNCVTest()succeeded on degenerate models — including a perfectly explained fit — and so served as a fallback that always worked. With NCV validating its input, the underlying failures are now surfaced rather than papered over.
Source integrity¶
- Removed two shadowed duplicate definitions.
performHCCovarianceTest()andperformQuantileRegressionTest()each had two definitions in the source tree. The obsolete implementations inR/modern_diagnostics.Rwere loaded first and then silently overwritten by the corrected versions inR/zz_deprecated_hc_covariance.RandR/quantile_regression_joint.R. Runtime behaviour was correct only because of collation order; a partial sourcing, a file rename or a collation change would have resurrected the withdrawn HC pseudo-test and the superseded quantile comparison. The obsolete definitions and their roxygen blocks are gone, along with the now-unreachable.hc_adjustment()helper whose only caller was the deleted HC implementation.
Resampling inference¶
- Bootstrap and permutation p-values now consistently use the finite-simulation
convention
(1 + #{T_b >= T_obs}) / (B_eff + 1), whereB_effcounts the replicates that converged.performWhiteTestBootstrap(),performWhiteTestEnhanced()and the sharedrbootstrap_test_statistic()engine previously used the raw proportion#/B, which can return exactly zero — not an attainable p-value from a finite number of replicates — and is anti-conservative in the tail.performWildBootstrapTest()andperformRankPermutationTest()already used the corrected form.tests/testthat/test-bootstrap-pvalue-convention.Rpins all of them.
Packaging and continuous integration¶
- Fixed the vignette build, which was breaking
R CMD buildand therefore every platform's check job. Three vignettes failed to build:bibliography.Rmdcalledalgorithms_bibliography(), which has never existed as a function (the name belongs to a documentation-only help topic);comprehensive_guide.Rmdcalledsimulate_hetero(n = 50)without the requiredbeta0,beta1andsigma_funcarguments; andusing_heteroTests.Rmdpassedmtcarsas the data for a model fitted onboston_housing, referenced the non-existentperformGoldfeldQuandtTest(), contained an illustrative chunk using an undefinedyour_dataplaceholder, and had a malformed chunk fence that left ten lines of code outside any chunk.R CMD buildnow completes and all eight vignettes render. inst/CITATIONmodernised: 25citEntry()calls converted tobibentry()(deprecated since R 4.2), a duplicatecitHeader()that silently replaced the first removed, and the package version now read frommeta$Version. The previousutils::packageDescription("heteroTests")$Versionraised "$ operator is invalid for atomic vectors" during CRAN incoming checks when the package was not installed.-
The CI R-version matrix was rebuilt around R-devel, R-release and R-oldrel-1, with macOS and Windows on R-release, replacing the fixed 4.0/4.1/4.2/4.3 set. The minimum version declared in
Depends:now has its own Ubuntu 22.04 job that installs hard dependencies only, so it reports on the package's own R 4.0 compatibility rather than on whether modernSuggestsstill support R 4.0. -
Fixed a broken example that failed
R CMD check.performQuantileRegressionTest()documented itself withmtcars, which has 32 observations, while the test requires at least 40. The example now usesboston_housing. - Non-ASCII characters removed from
R/messages.RandR/validation.R. The two occurrences inside string literals are now written as the escapes\u00b2and\u2022; the en dashes in roxygen comments became ASCII hyphens. - Resolved 18 duplicated
\aliasentries.man/simulate_hetero.Rdis a complete hand-written family page forsimulate_hetero(),simulate_arch1()and everysigma_*()helper, but roxygen2 also generated a thinner page per helper, so each name was documented twice. The generated stubs are removed — several were incomplete, for examplesigma_poly.Rdomitteda,bandc— and the corresponding roxygen blocks are marked@noRdso they do not reappear. The same applied togenerate_benchmark_report, which is documented on therun_benchmark_suitepage. - Reconciled
\usagesections that had drifted from their function signatures:performWhiteTest()(max_interactions),plot.HeteroDiagnostic()(plots) andsummary.HeteroDiagnostic()(tests). - Documented previously undocumented arguments in
generateHeteroRecommendations.Rd,plot.power_analysis.Rd,print.htest_enhanced.Rd,test.HeteroDiagnostic.Rdandsimulate_hetero.Rd, and merged the duplicatednentry on the last of these. -
.Rbuildignorenow excludes.zenodo.json,CITATION.cffandCONTRIBUTING.md, which are repository metadata rather than package content.inst/validation/is deliberately shipped so the size and power evidence travels with the package. -
Fixed a latent runtime failure in the dashboard.
launchDiagnosticDashboard()used the magrittr pipe%>%in four places without importing it.DTandplotlyareSuggestsand are called with::rather than attached, so the pipe was only ever resolved if one of them happened to be attached by something else. The calls are now nested. - A second example failed
R CMD check:performWhiteTestBootstrap()usedmtcars(32 observations) while the bootstrap requires 50. It now uses a simulated 120-observation dataset, with the heavier replication counts moved under\donttest{}. - Fixed 15 broken
\link[stats:htest]{htest}cross-references. There is nohtesttopic instats, so every one of these was a dangling link; they are now plain\code{htest}. - Completed the
NAMESPACEimports.AIC,cooks.distance,cov,lm.fit,model.frame,model.response,nobs,quantile,resid,termsandunirootfromstats, andhead,object.sizeandtailfromutils, were used without being imported. Column names referenced insideaes()are declared throughutils::globalVariables(). -
Removed the
Author:andMaintainer:fields fromDESCRIPTION, which had drifted from the values derived fromAuthors@R(the derived form carries the ORCID). Unescaped 15\&sequences in.Rdfiles, and added_pkgdown.ymlandLICENSEto.Rbuildignore. -
Fixed three defects in the benchmarking system, all of which surfaced once its example was actually reachable under
R CMD check --run-donttest.run_benchmark_suite()aborted with "no rows to aggregate" whenever every measured difference wasNA, becauseaggregate()dropsNAresponses and the guard only tested for zero rows.generate_benchmark_report()then failed with "argument 1 is not a vector" when called withprofile_memory = FALSE, since it passed aNULLcolumn toorder(). Andht_extract_metrics()had no branch forcar::ncvTest(), which returnsChiSquare/Df/prather than thehtestfield names, so every accuracy row measured againstcarwas silentlyNA. With that branch in place the benchmark now reports a p-value difference of about1e-15betweenperformNCVTest()andcar::ncvTest()-- an independent confirmation of the correction described above. -
Fixed the Breusch-Pagan benchmark baseline.
performBPTest()is the classical statistic, but the benchmark compared it againstlmtest::bptest()with its defaultstudentize = TRUE, i.e. the Koenker form. The pairing is nowstudentize = FALSE. With this and thecar::ncvTest()extractor in place, all three baselines agree with our implementations to machine precision (0,1.7e-16and1.2e-15), so the benchmark now doubles as a live reference-equivalence check. - Fixed
simulate_power_analysis(), whose effect size had no effect. It built the variance function aseffect_size * sigma_func(x), but heteroscedasticity tests are invariant to multiplying every standard deviation by a constant, so every effect size produced statistically identical data and the reported power differed only by Monte Carlo noise. The effect size now interpolates between a constant variance and the supplied pattern, so0is homoscedastic and1is the pattern itself. Power againstsigma_linearwith White's test now runs 0.04 / 0.07 / 0.33 / 0.96 across effect sizes 0 / 0.1 / 0.4 / 1.0, recovering the nominal level at zero effect. -
The Type I error test now sizes its tolerance from the Monte Carlo standard error rather than a fixed 0.03 margin, which at 200 replications was under two standard errors and failed by chance in roughly one run in seven.
-
Raised the declared minimum R version to 4.1.
Depends: R (>= 4.0.0)was unsatisfiable:ggplot2andscalesare hardImportsand both now require R >= 4.1, so the package could not be installed on R 4.0 whatever its own code did. The minimum-version CI job checks 4.1 accordingly. -
Fixed two CI jobs that had never run successfully. The spell check called
spelling::spell_check_package(ignore = ), an argument that does not exist, so the step errored every time;inst/WORDLISTis read viause_wordlistinstead. The benchmark job gated on heteroTests being no more than 50% slower than barelmtest/carcalls, which it cannot satisfy -- it runs a validation layer those do not, and at n = 100 the baseline time rounds to zero, making the ratio infinite. The timing table is now reported as an artefact and the job gates on accuracy, which is the property that must not regress. -
Fixed the pkgdown deploy, which had been failing on main since before this release.
man/algorithms_bibliography.Rdcarried\usage{NULL}, which is what roxygen emits for a documentation-only topic and which pkgdown cannot parse. A topic that documents no function needs no usage section. This is why the package site was never published, and so why the URLs inDESCRIPTIONandinst/CITATIONreturned 404. -
The automated release workflow now installs dependencies before
R CMD build, which renders the vignettes and therefore needs the vignette builder. That step is gated on the version changing, so the omission had never fired until the 0.7.0 bump. -
Fixed
generateDiagnosticReport(), which could not generate a report. The YAML front matter it writes put two mappings on one line (output: html_document:), so rmarkdown aborted with "mapping values are not allowed in this context".html_documentis now nested underoutput. The example is wrapped in\dontrun{}, so only a check passing--run-dontrunexecuted it, which is why it went unnoticed. - The CRAN submission job now installs a LaTeX toolchain, and the pkgdown job
installs
rsconnect. Both were failing on missing dependencies rather than on anything in the package: the former reportedpdflatex is not available, which R CMD check surfaces as a probable Rd problem, and the latter needs rsconnect to inspectinst/tutorials.
Documentation¶
performGlejserTest()documents that the test is not asymptotically valid under asymmetric errors (Godfrey 1996; Im 2000), and that scanning several transformations and reporting the smallest p-value invalidates the nominal level.performParkTest()documents that its t statistic is valid only asymptotically, the auxiliary error being a strongly skewedlog(chi^2_1)variate, and that a non-significant result speaks only to the single variable tested.
0.6.5¶
- Bug fix (statistical correctness):
performBPTest()/performBreuschPaganTest()now compute the classical Breusch-Pagan statistic (half the explained sum of squares from the scaled squared residuals), matchinglmtest::bptest(studentize = FALSE). It previously returned the studentizedn R^2statistic despite being documented as the classical, normality-based test. - Bug fix (statistical correctness):
performKoenkerTest()now implements Koenker's (1981) studentized Breusch-Pagan statistic asn R^2from regressing the squared residuals on the regressors, matchinglmtest::bptest(studentize = TRUE)andperformStudentizedBPTest(). It previously regressed the absolute residuals, which is not a recognized test and produced p-values without a valid chi-squared justification. - The streaming variants
performBPTestStreaming()andperformKoenkerTestStreaming()were updated to stay algebraically equivalent to their exact counterparts. - Bug fix: resolved a duplicate
%||%operator definition whose two copies had conflicting semantics. Depending on collation order this could causerunHeteroTests()to abort with a "length > 1" coercion error when given a multi-column data frame. Both copies now share a single vector-safe definition. - Bug fix (statistical correctness):
performWildBootstrapTest()now generates its bootstrap reference distribution under the homoscedastic null by resampling leverage-standardised, centred residuals i.i.d. before applying the wild multiplier. The previous implementation multiplied the original (heteroscedastic) residuals by the wild weights, so the bootstrap distribution inherited the heteroscedasticity being tested for and the test had essentially no power (returning p ~ 0.5 on strongly heteroscedastic data). It now controls size and detects heteroscedasticity reliably; regression tests cover both. - Packaging:
R6moved fromSuggeststoImports(it is used at package load time),paralleladded toImports(used via::), and the optionaldigestdependency (used for cache hashing) is now declared inSuggests. These resolveR CMD check"undeclared dependency" and load-time failures. - Bug fix:
performWhiteTest()now drops perfectly collinear auxiliary columns (via QR pivoting) and reportsdfequal to the realised rank, instead of aborting. White's test consequently works on models containing factors, where a squared dummy column is collinear with the dummy itself. - Bug fix:
performVIFDiagnostic()now computes variance inflation factors from the inverse correlation matrix of the design, so it works for models with factor predictors (it previously built auxiliary formulas from contrast-expanded column names that do not exist in the model frame, e.g.cyl6, and errored). ht_data_cleaning_suggestions()now flags infinite values found in the supplied data, not only those mentioned in the error message.- Bug fixes surfaced by
R CMD check: corrected invalid namespace qualifications (stats::all.vars,stats::summary,utils::closeare base functions), fixed asuggestRemediation()call ingenerateDiagnosticReport()that passed unsupported arguments, and declared the (guarded) optionalmgcvdependency. - Data sets
boston_housing,diagnostic_dataandhetero_dataare now shipped as.rdafiles (generated by scripts underdata-raw/) rather than R scripts, removing theLazyDatanote and making the bundled data reproducible. - Testing: the suite is consolidated onto a single
test-*.Rnaming convention, pinsConfig/testthat/edition: 3, drops the deprecatedcontext()calls, removes three stale top-level-skip()ped files that provided no coverage, and adds behavioural regression tests for the fixes above.
0.6.4¶
- Added "Comprehensive Guide to Heteroscedasticity Testing" and "Troubleshooting Common Issues" vignettes.
- Implemented
performWhiteTestStreamingfor large datasets. - Added wild bootstrap, HC covariance, quantile regression, permutation, high-dimensional, and spatial heteroscedasticity diagnostics.
- Integrated broom tidiers, ggplot2 theming/autoplot support, tidymodels workflows, survey designs, grouped data pipelines, and data.table/dtplyr compatibility for diagnostic workflows.
- Introduced an intelligent recommendation engine that profiles datasets, interprets diagnostics with confidence levels, and delivers remediation guidance plus decision-tree style reports for non-statisticians.
- Implemented robust error handling with adaptive fallbacks, actionable cleaning suggestions, and configurable logging for complex diagnostic workflows.
0.6.3¶
- Added automated report generation via
generateDiagnosticReport.
0.6.2¶
- Added studentized Breusch-Pagan, bootstrap White and Szroeter tests.
0.6.1¶
- Introduced
TestFactoryR6 class with metadata support. - Added exported
test_factoryinstance registering core tests. - DESCRIPTION now lists R6 in Suggests.