What every test returns¶

Source: man/test_results.Rd · Help: ?test_results

Every exported perform*Test() function returns a base-R htest object, so a result prints, subsets and composes like the result of stats::t.test() or lmtest::bptest(). This page lists every field such a result can hold: the five every test returns, the two optional htest fields some return, and the fields a few tests add. tests/testthat/test-return-contract.R checks every exported test against it.

Fields of every test¶

statistic: the test statistic: a numeric value of length one, named after the statistic ("X-squared", "F", "t", "GQ", "LM", "z", ...).
parameter: the parameters of the null distribution: a numeric vector whose elements are named, usually degrees of freedom ("df", or "df1" and "df2"). The resampling tests store the number of replications here ("B", "permutations"), and the tests on ordered observations the sizes they use ("n", "m"). It is NULL for performPesaranTest(), whose statistic is referred to the standard normal distribution.
p.value: the p-value: a single number between 0 and 1.
method: a character string naming the test. When an argument switches between two statistics, such as studentize in performHarveyTest(), the string says which one was computed.
data.name: a character string describing what was tested: the formula of the model, followed for some tests by the variance regressors. For performBoxMTest(), which tests data rather than a model, it names the data and the grouping.

Optional htest fields¶

alternative: a character string describing the alternative hypothesis, returned by the tests that have a direction or a stated alternative.
estimate: estimates that accompany the statistic: a named numeric vector (performSzroeterTest(), performSpearmanTest(), performVarianceFormTest()), or, in performQuantileRegressionTest(), a matrix of slope estimates with one column per quantile and named rows and columns.

Fields some tests add¶

p.value_bootstrap: performWhiteTestRobust() and performBPTestRobust() with bootstrap = TRUE: the bootstrap p-value.
robust_details: performWhiteTestRobust() and performBPTestRobust(): a list with the confidence interval, effect size, bootstrap results, power and sample size, and method or studentized, the setting that chose the statistic.
boot_statistics: performWhiteTestBootstrap(): the bootstrap replicates of the statistic.
bootstrap: performWildBootstrapTest(): a list with the replicates, the number of usable ones, the multiplier distribution and the observed statistic.
permutation: performRankPermutationTest(): a list with the replicates, the number of usable ones and the ordering variable.
projection: performHighDimensionalTest(): a list with the number of principal components kept, the share of the variance they explain and the threshold that chose them.
variance_function: performVarianceFormTest(): a list with the form of the variance function, its regressors and the added terms that were tested.
quantiles, se_type, iid: performQuantileRegressionTest(): the quantiles compared and the settings of the standard errors.
chunks_processed, chunk_size, r_squared: performWhiteTestStreaming(), performBPTestStreaming() and performKoenkerTestStreaming(): how the data were read and the $R^2$ of the auxiliary regression.
res: performSpatialHeteroTest(): the Monte Carlo replicates of Moran's I from spdep::moran.mc(), whose class "mc.sim" the result keeps.

Attributes¶

performWhiteTest() and performWhiteTestRobust() attach the attributes auxiliary_regressors, original_regressors, cross_products_included and r_squared_auxiliary, which describe the auxiliary regression. runHeteroTests() adds attributes of its own to each result it returns.

Inputs every test refuses¶

Since 0.15.0 every test on a model refuses four inputs, each with one message: a glm that is not a least-squares fit, that is, of any family but gaussian or with any link but the identity ("needs a model fitted by stats::lm()"); a model with a coefficient that is not finite, infinite, NaN, or NA without the fit being rank-deficient ("Model coefficients must be finite"); data that lack rows the model was fitted to; and data that lack a variable of the model ("The following variables are missing from"). Until 0.16.0 the NA of an aliased term was refused too; since then a rank-deficient fit is tested on the estimable part of its design (see Supported models). The model frame is accepted as data as it is. A glm with the Gaussian family and the identity link is accepted by every perform*Test() function and gives the result of the lm up to rounding error, when the two decide the rank of the design alike (see Supported models); performWhiteTestStreaming(), performHarrisonMcCabeTest() and performVarianceFormTest() refused every glm until 0.16.0, and bootstrapTestStatistic() until 0.17.0, which bootstraps it as the lm it is. With positive prior weights it gives the result of lm(..., weights = w): performGQTest(), performRESETTest(), performWhiteTestBootstrap(), performWildBootstrapTest() and performHarrisonMcCabeTest(), which read the weights of an lm, read the prior weights of the glm, and performQuantileRegressionTest(), which refuses a weighted lm, and performVarianceFormTest(), which refuses weights that did not come from fitWLS(), refuse it with the same message. With a zero prior weight it is refused as the lm with a zero weight is ("The model was fitted with zero or non-finite weights"), since 0.16.0. A survey::svyglm() fit is refused by every test ("The model is a survey::svyglm() fit"). tests/testthat/test-input-validation.R gives every test these inputs. Since 0.17.0 a MASS::rlm() fit is refused by every test too, with class heteroTests_model_context_error (see Supported models), which tests/testthat/test-model-context.R checks.

The tests that read data also refuse data whose rows cannot be confirmed to hold the observations of the fit ("cannot match the rows of data to the observations of the model"). They match the rows of data to the residuals by row name, and a tibble, a data.table or a data frame whose row names were reset, as by dplyr::arrange(), has row names that only number its rows. Before a test reads data, the rows the row names pick are compared with the model frame of the fit, row by row: every column data holds under its own name, value by value, and the others (log(x), poly(x, 2), offset(o), the weights and an offset given as arguments) computed from data as the fit computed them. A term computed from the rows it is given, such as I(x - mean(x)), and a column that names a variable data does not hold, are not compared; for a term of variables of data, the compared columns must then tell the rows of the fit apart. Data that cannot be confirmed so are refused, even the data of the fit: when no column can be compared, as with lm(d$y ~ d$x), or when the compared columns tie on rows that differ in such a term. Pass the full data the model was fitted on, or the variables as columns of data. Data without row names of their own in another order than the fit, or with rows in front of those of the fit, are refused, as are repeated row names, and a model frame rebuilt for a fit with model = FALSE from data that changed after the fit. Data with row names of their own are matched by name, in any order; until 0.16.0 performPesaranTest(), performBPRandomEffectsTest() and performScatterDiagnostic() read data by position, and refused such data in another order than the fit. tests/testthat/test-data-alignment-guard.R checks these refusals.

Since 0.16.0 performWhiteTest(), performBPTest(), performKoenkerTest(), their streaming forms, performWhiteTestStreaming(), performBPTestStreaming() and performKoenkerTestStreaming(), which read the design of the fit chunk by chunk where each chunk built its own from the rows of data, performRankPermutationTest(), performHighDimensionalTest(), the studentized Breusch-Pagan statistic (performStudentizedBPTest(), performBPTestRobust() with studentize = TRUE, the registered "student_bp"), performRESETTest(), performGQTest(), performQuantileRegressionTest(), performGlejserTest(), performParkTest(), performSzroeterTest() with the registered "szroeter", performHarrisonMcCabeTest(), performHartleyFmaxTest(), performSpatialHeteroTest(), the five group tests, performLeveneTest(), performBrownForsytheTest(), performFlignerKilleenTest(), performBartlettTest() and performOBrienTest(), the two panel tests, performPesaranTest() and performBPRandomEffectsTest(), and so runPanelTests(), and performScatterDiagnostic(), and the tests computed from the model alone, performSpearmanTest(), performHarveyTest(), performArchLMTest(), performMcLeodLiTest(), performCookWeisbergTest() and performNCVTest(), performSpreadLevelTest(), performDavidianCarrollTest() and performVarianceFormTest(), with fitWLS(), read the response, the design, the residuals, the weights and the offset from the fit, through the model context of the package, and read from data only the rows the comparison above confirmed; the other tests follow in later steps (the design page of the site, https://diogoribeiro7.github.io/heteroTests/design/model-context/, sets out the plan). RESET, the tests computed from the model alone and fitWLS() without var_formula read nothing from data, and match it as the other tests do, so that every test takes and refuses the same data; until 0.16.0 they gave the value of the fit on numbered data in another order, rows in front of those of the fit and repeated row names, and refused numbered data in the order of the fit whose row numbers are not the fit's (fitWLS() ignored data). A second formula, the var_formula of the Cook-Weisberg test, of the variance form test and of fitWLS(), and the against of the variance form test, is evaluated on the confirmed rows of data, or, without data, on the model frame; a variable that neither holds is refused, where it was found in the workspace. The Goldfeld-Quandt, Szroeter and Harrison-McCabe tests read their order_by, Glejser's and Park's their variable and Hartley's and the group tests their group from the confirmed rows of data, or, without data, from the model frame when it holds the variable under its own name ("The following variables are missing from the model frame of model" otherwise). Hartley's test refuses a group that is missing at an observation of the fit; the group tests leave such observations out, with the warning "Removed N observations due to missing values in ...", as before 0.16.0. The panel tests read their id and time, and the scatter diagnostic its vars, from the confirmed rows of data; without data the panel tests read them from the model frame, when it holds them under their own names, and the scatter diagnostic needs data. Pesaran's test refuses a pair of id and time that holds two observations of the fit and a unit or a period missing at an observation of the fit, and computes the statistic of an unbalanced panel, whose units are not all observed at the same periods, from the periods each pair of units shares, as plm::pcdtest() does. The registered "szroeter" orders by the fitted values, which it computes from the fit, and the spatial test reads no column of data: its listw lists the observations of the fit in its order. Harrison-McCabe takes the rank of the fit, decided at the fit's tolerance (1e-7 for an lm, 1e-11 for a glm). These tests match numbered rows by the row numbers of the fit, or, when data holds one row per observation and its numbers are not those of the fit, in the order of the fit, once those rows reproduce the model frame; they drop nothing before the match, so data that hold the rows a fit with na.omit dropped, or only the rows it kept, numbered again, are accepted. Since 0.17.0 a fit with na.action = na.exclude is tested as the same fit with na.omit, which drops the same rows, with the same values, refusals and warnings: before, the tests refused it ("Model residuals must be finite"), RESET stopped on it ("NA/NaN/Inf in 'x'"), and the panel tests and the scatter diagnostic read its residuals, padded with NA at the rows it dropped, against the rows of data by position, which gave a statistic that was NA, or computed with the NA. An output with one value per row, as the estimated variances fitWLS() attaches to its fit, is padded with NA at the rows the fit dropped, as residuals() and weights() of such a fit are. Named data in any order, numbered data in the order of the fit: numbered data in another order are refused, as by the other tests. They need no data for a model that transforms its variables, has an offset() term, or takes its weights or offset from a column of the data, and accept its model frame as data. They also refuse a multivariate lm ("needs a model with one response"), a fit whose coefficients, residuals or weights were changed after it was fitted ("does not reproduce its residuals from its design and coefficients", "The weights of the model are not those its coefficients were fitted with"), a fit whose terms do not rebuild the design of its coefficients, as a fitWLS() fit of an mgcv::gam() fit with smooth terms (the first of these messages, since 0.17.0), and a fit with model = FALSE whose model frame can no longer be rebuilt, unless data gives its response and fitted values, on which it is then tested. The tests that fit the model again by least squares, or whose null distribution is derived for the least-squares residuals of its design, refuse an mgcv::gam() fit with penalised smooth terms, which is not the least-squares fit of its design, and the others test it on that design, the basis of its smooths, since 0.17.0 (see Supported models below). The residuals of a gam carry no names, and these tests match data to its observations by the row names of its model frame; the tests that are not built on the context yet and read a column of data match them by position. The check of an edited fit is a guard against accidental edits, not a proof that a fit is unedited: an edited coefficient leaves the residuals unchanged, so it is seen only at 1e-7 of the largest term of the response, the design times the coefficients and the residuals, and an edit of 0.5 goes undetected once the response is above about 1e7. These refusals have class heteroTests_model_context_error, which runHeteroTests() and runDiagnostics() keep, as they keep that of a refusal of data, without running another test under the name of the one refused; since 0.17.0 a grouped run stops with such a refusal in a group too, where the group recorded it as a failed test of NA. A test that reads a column of data the model frame does not hold, as the rank permutation, Goldfeld-Quandt, Szroeter and Harrison-McCabe tests do with their order_by, Glejser's and Park's with their variable, Hartley's and the group tests with their group, the panel tests with their id and time and the scatter diagnostic with its vars, refuses numbered rows taken in the order of the fit if two rows of the fit tie on every compared column. On a call with more than one defect, the first one reported may differ from before 0.16.0, since the rows of data are now matched before the checks of the model. tests/testthat/test-model-context.R checks them.

Supported models¶

The tests examine the residuals of a least-squares fit for a variance that is not constant, so they accept the models whose residuals are those of a least-squares fit, and refuse the others early, with the messages above. Which test accepts which class, measured by running every test on each, is the table of the Supported models page of the documentation site; tests/testthat/test-model-eligibility.R and test-input-validation.R check it.

An lm is accepted by every test.
An lm(..., weights = w) is accepted: lm() states that the error variance is $\sigma^2 / w_i$, and the tests read the Pearson residuals $\sqrt{w_i}\, e_i$, which have constant variance when the weights are right. performQuantileRegressionTest(), which refits without them, refuses it, and performVarianceFormTest() refuses weights that did not come from fitWLS().
A zero weight: the observation is not part of the fit, and its residual is an artificial zero that a test would count. Refused by every test but performRESETTest(), an F test of two nested weighted least-squares fits, in which a zero weight drops the observation from both; it gives the value of the fit to the observations with positive weight. Keeping that exception was decided on 2026-10-17.
An offset, given as offset = o or as an offset() term: the residuals of the fit include it, and the tests that read them account for it. Four left it out of a refit and returned a value that was not that of the model; they refused it in the development of 0.16.0 (decided 2026-10-17), and since the offset fix of 0.16.0 they read the offset from the fit and carry it in every refit: performGQTest(), performQuantileRegressionTest() and the studentized Breusch-Pagan statistic (performStudentizedBPTest(), performBPTestRobust() with studentize = TRUE, the registered "student_bp") give the value of the model fitted to the response minus the offset, and performRESETTest() the nested F test of the model with its offset against the model augmented with the powers of its fitted values, $X\hat\beta + o$ (decided 2026-10-18).
A glm with the Gaussian family and the identity link is a least-squares fit. With prior weights $w_i$ it is fitted by maximum likelihood, which for normal errors of variance $\phi / w_i$ maximises $-\sum_i w_i (y_i - x_i^\top \beta)^2 / (2\phi)$: weighted least squares. Iteratively reweighted least squares, the algorithm of glm(), then has the working response $y_i$ and the working weights $w_i$ whatever the current estimate, so its first step reaches the solution of lm(..., weights = w) (Nelder and Wedderburn, 1972; McCullagh and Nelder, 1989, sections 2.5 and 3.6). It is Fisher scoring, and for the identity link, the canonical link of the Gaussian family, the information does not depend on $y$, so scoring is Newton-Raphson, which converges in one step when the log-likelihood is quadratic in $\beta$ (Green, 1984, section 1.2). Its design, coefficients, fitted values, residuals and weights are those of the lm, up to rounding, so a test whose statistic and null distribution are functions of those, under the assumptions of the linear model, which the glm states, gives the result of the lm: every perform*Test() function does, weighted or not (tests/testthat/test-input-validation.R and test-model-eligibility.R compare them). That holds when the two decide the rank of the design alike: lm() aliases a column with the tolerance 1e-7 of its QR decomposition, glm.fit() with min(1e-7, epsilon / 1000), 1e-11 by default, so on a column within that margin of a combination of the others, such as x3 = x1 + 1e-9 * rnorm(n), the glm keeps it and gets the values of lm(..., tol = 1e-11), not those of the default lm. performWhiteTestStreaming(), performHarrisonMcCabeTest() and performVarianceFormTest() refused every glm until 0.16.0, and fitWLS() refits it as that lm since then. bootstrapTestStatistic() refused every glm until 0.17.0, as it refits each replicate by least squares, which would drop a family and a link; it bootstraps a Gaussian glm with the identity link as that lm, with its prior weights, and refuses any other glm, as before. It refused every mgcv::gam() fit until 0.17.0, and refuses one with penalised smooth terms since then (see the item on the gam below). The checks of the model read its R-squared, which summary() of a glm does not report, as that of the lm: summary.lm() applied to the glm, whose fitted values, residuals, weights and offset are those of the lm, so that a near-perfect fit is refused as the lm is, since 0.16.0. The Supported models page of the documentation site sets out the argument; the references are below, and the section numbers of McCullagh and Nelder (1989) are those of its table of contents. The tests read its prior weights, model$prior.weights, and never model$weights, which holds the working weights of the last iteration of iteratively reweighted least squares, $w_i (d\mu_i/d\eta_i)^2 / V(\mu_i)$. For this family and link the two are equal up to rounding; for any other they depend on the fitted means. A glm fitted with weights that are all 1 has the prior.weights of one fitted without weights, so the tests take it for an unweighted fit, where lm(..., weights = rep(1, n)) is a weighted one: performVarianceFormTest() and performQuantileRegressionTest(), which refuse a weighted fit, accept the glm and refuse the lm.
Any other glm, such as a binomial or a Poisson one, is refused: its residuals have a variance that depends on the mean by construction, so a test of constant variance tests nothing the help pages describe.
A survey::svyglm() fit is refused: its residuals are scaled by the sampling weights, which are not precision weights. See runSurveyHeteroTests().
Panel models: performPesaranTest() and performBPRandomEffectsTest(), and so runPanelTests(), take a pooled lm and the columns of data that name the units and the periods. A plm::plm() fit is not an lm: every test refuses it, as it refuses anything but an lm or a glm, with one message, which names the class received and points to the pooled lm().
A rank-deficient fit: a term that is a linear combination of others, or the dummy of a cell of an interaction that no observation falls in, leaves an NA coefficient. Its residuals and fitted values are the projection on the column space of the design, which the aliased columns do not change. Since 0.16.0 every test that accepts the model without the aliased columns accepts it, computed on the estimable part of the design: the columns that were not aliased, with degrees of freedom from the rank, so that it gives the statistic and p-value of the model fitted without them. A test that refuses that model refuses the rank-deficient one with the same message. The model without them is the one lm() keeps: its decomposition keeps a column unless it is a combination of the columns before it, so of an aliased set the column that comes last, in the order of the terms, is left out. Most statistics depend only on the space the columns span, and are the same whichever column is left out. Two depend on which: performWhiteTest(), and its robust, streaming and bootstrap forms, when the cross products are left out (cross_products = FALSE, method = "reduced", or more regressors than max_interactions), as the squares are those of the columns kept; and performHighDimensionalTest() when it keeps fewer principal components than columns. With form = "power", performVarianceFormTest() takes the logarithms of every column of the design, the aliased ones included, so its value is not that of the model without them (and does not depend on the order of the terms); with the exponential form, the default, it is.
A fitWLS() fit is a weighted lm, with estimated weights: accepted, but the reference distributions take the weights as known (performVarianceFormTest() accounts for the estimation).
A MASS::rlm() fit, such as fitRobust() returns, is an lm by class and an M-estimate, whose residuals are not least-squares residuals. Since 0.17.0 every test refuses it, with an error of class heteroTests_model_context_error that names the lm() fit of the same formula to test instead, and so do fitWLS(), the registered names and the runners, which keep the refusal final. Until then 37 tests took it for an lm and tested its M-estimation residuals as if they were least-squares ones, the studentized Breusch-Pagan statistic tested the least-squares refit of its design, and performQuantileRegressionTest() and performVarianceFormTest() refused it as a weighted fit, its case weights, all 1 unless given, being read as precision weights.
An mgcv::gam() fit with the Gaussian family and the identity link, a glm by class, is tested on its stored design, model.matrix(): its parametric columns and the basis of its smooths. A smooth term is penalised unless it is fixed, s(x, fx = TRUE), and the residuals of a penalised fit are not those of the least-squares fit of that design. Since 0.17.0 the tests that read only its residuals, its fitted values, columns of data and the columns of the design as auxiliary regressors test a penalised fit, the streaming tests on the basis too, as the exact tests do, where they read the variables of the formula. The tests that fit it again by least squares, or whose null distribution is derived for the least-squares residuals of its design, refuse it with class heteroTests_model_context_error, and the runners keep the refusal final: performGQTest(), performRESETTest(), performQuantileRegressionTest(), the studentized Breusch-Pagan statistic, performWhiteTestBootstrap(), performWildBootstrapTest(), bootstrapTestStatistic() and the robust forms with bootstrap = TRUE, performVarianceFormTest() and fitWLS(), which refitted it without its penalty, and performHarrisonMcCabeTest() and performSzroeterTest(), which compared its residuals with the null distribution of least-squares ones. A gam that is the least-squares fit of its design, without smooth terms or with fixed ones, is tested by every test as the lm() of that design, the refitting and resampling tests included, where the fixed smooths were refused as penalised ones. Whether a gam is that fit is decided from the fit: its design refitted by least squares, with its weights and offset, gives its residuals to 1e-8 of the largest residual of that refit, plus 1e3 times the machine epsilon of the largest value of the response and of the offset, the rounding of a fit on that scale, so that an offset or a trend does not change the verdict, up to the scale at which that rounding hides a penalised fit (the largest value of the response or of the offset about 1e12 times its residuals). A gam without a penalised term that fails the check is refused as one whose residuals are not those of that fit, not as a penalised one: mgcv's own fit is not the least-squares fit of an ill-conditioned design, as one with a regressor far from zero (centre the regressor, or fit the model with lm()), or the fit was changed after it was fitted. Decisions 37 and 38 of the design of the model context (decided on 2026-10-29) set this out.

checkModel() checks the class of a model only, lm or glm; it does not check whether a test accepts it.

References¶

Green, P. J. (1984). Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives. Journal of the Royal Statistical Society, Series B, 46(2), 149–192. doi:10.1111/j.2517-6161.1984.tb01288.x

McCullagh, P., & Nelder, J. A. (1989). Generalized Linear Models (2nd ed.). Chapman & Hall. The section numbers cited are those of its table of contents.

Nelder, J. A., & Wedderburn, R. W. M. (1972). Generalized linear models. Journal of the Royal Statistical Society, Series A, 135(3), 370–384. doi:10.2307/2344614