A Wrong Forward Model Can Produce a Precise Wrong Inverse

Inverse problems are often introduced with an equation of the form

where $F$ is the forward model, $\theta$ is the unknown parameter or latent field, and $\varepsilon$ represents measurement noise. Much of inverse-problem theory then studies whether $\theta$ is identifiable, whether the inverse map is stable, and how regularisation or prior information should be introduced when the observations do not constrain every parameter direction equally. Those questions are fundamental, but the formulation contains a strong assumption that is easy to overlook: the forward model itself is treated as correct.

In real scientific systems, that assumption is often the least defensible part of the analysis. A differential equation omits mechanisms that are believed to be secondary. A constitutive law is used outside the regime in which it was derived. A computational fluid-dynamics model closes unresolved turbulence with an approximation. A pharmacokinetic model treats heterogeneous tissue as a small number of well-mixed compartments. A finite-element solver discretises a continuous PDE on a finite mesh. A climate model works at a spatial resolution much coarser than the process being measured. A simulator is replaced by an emulator. In each case the observation differs from the forward model not only because of measurement noise but because the model itself cannot reproduce the true system exactly.

A more realistic observation equation is therefore

where $\delta$ represents model discrepancy. The distinction between $\delta$ and $\varepsilon$ is not cosmetic. Measurement noise describes uncertainty in how the system is observed. Model discrepancy describes a systematic difference between the physical process and the mathematical model used to represent it. The two terms can have different dependence structures, different scales, different interpretations and, most importantly, different consequences for parameter estimation.

The difficult feature of model discrepancy is that an inverse algorithm does not know which part of a mismatch belongs to the parameters and which part belongs to the model. If the discrepancy resembles the effect of changing a parameter, the calibration procedure can absorb the error into that parameter and still produce an excellent fit. The resulting inverse may be stable, reproducible and accompanied by narrow uncertainty intervals. None of those properties guarantees that the inferred parameter retains its intended scientific meaning.

Discrepancy can be absorbed exactly into the parameter estimate

The geometry is clearest in the linear case. Suppose the true data-generating mechanism is

but the fitted inverse model omits $\delta$ and assumes

If $A$ has full column rank, ordinary least squares gives

Substituting the true data-generating equation yields

Taking expectation over zero-mean measurement noise gives

The second term is the parameter bias created by model discrepancy. It is not reduced by collecting repeated observations at the same design points if the discrepancy persists. The estimator converges more precisely to the wrong parameter value because the omitted structural term is systematically projected into the parameter space.

The decomposition becomes even more informative if the discrepancy is split into components parallel and orthogonal to the column space of $A$,

where

is the projection matrix onto $\operatorname{col}(A)$. The component $P_A\delta$ is observationally indistinguishable from a change in the parameter vector because it lies in exactly the space of output changes that the forward model can generate. The component $(I-P_A)\delta$ cannot be reproduced by parameter changes and therefore appears in the residual.

The fitted residual is

which, ignoring measurement noise for the moment, becomes

This identity contains a central warning for inverse modelling. Residual diagnostics can only see the part of model discrepancy that lies outside the forward model's output space. The component of discrepancy that projects into $\operatorname{col}(A)$ has already been converted into parameter bias and disappears from the residual entirely.

A perfect residual therefore does not imply a correct forward model. If

then there exists some vector $\Delta\theta$ such that

The data satisfy

so the misspecified inverse model fits the observations exactly with parameter

There is no residual signal from which the discrepancy can be detected. The inverse problem has produced a perfectly fitting but scientifically shifted parameter vector.

This is not merely a pathological algebraic construction. It describes the general mechanism by which calibration parameters compensate for missing physics. If increasing a heat-transfer coefficient produces the same observational pattern as an omitted heat source, the calibrated coefficient can absorb the source. If a compartment model lacks a pathway that accelerates clearance, another clearance-rate parameter may shift to reproduce the same trajectory. If a material model omits nonlinear stiffness, an effective linear modulus may drift with the loading regime because it is being asked to represent both the intended property and the missing constitutive behaviour.

The fitted parameter then ceases to be a direct estimate of the physical quantity named in the model. It becomes an effective parameter, one that compensates for the particular discrepancy present under the particular experimental conditions used for calibration.

A simple omitted-curvature example shows why good fit is not enough

Consider observations generated by

but suppose the fitted model assumes only

The omitted quadratic term

is model discrepancy. The least-squares estimate of $\theta$ is

Substituting the true model gives

The expected bias is therefore

The estimate of the linear coefficient is contaminated by the omitted curvature. More observations at the same distribution of $x$ reduce the random term but do not remove the structural bias. Indeed, as sample size grows, the confidence interval around $\hat\theta$ can become arbitrarily narrow while remaining centred on the wrong effective slope.

The amount of bias depends on the design points. If calibration is performed only over a narrow interval of $x$, the quadratic term may be approximated extremely well by a linear adjustment, and residuals can look innocuous. If the model is then used at values of $x$ far outside the calibration region, the compensation fails because the effective slope that mimicked curvature locally no longer represents the system globally.

This is one reason calibrated mechanistic parameters often fail to transport across experimental conditions. The parameter was not estimated in isolation from model error. It was estimated jointly with whatever discrepancy happened to be present in the calibration regime. A new regime changes the discrepancy, so the effective parameter can change even if the underlying physical property remains constant.

The phenomenon is particularly dangerous when the fitted parameter is given a strong mechanistic interpretation. A narrow standard error can create the impression that a physical rate constant or material property has been measured precisely, while in fact the estimate is partly a surrogate for omitted dynamics. Statistical precision around an effective parameter is not the same as physical accuracy.

Treating discrepancy as noise can hide structure rather than remove it

One common response to unexplained residual variation is to increase the measurement-noise variance. Suppose the inverse model is written as

and the fitted $\sigma^2$ becomes large enough to cover discrepancies between model and data. This can make uncertainty intervals wider and reduce apparent overconfidence, but it changes the interpretation of the unexplained variation. Systematic model error is being represented as independent observational noise.

If discrepancy is smooth over time or space, residual errors are likely to be correlated. If the model persistently underpredicts one region and overpredicts another, the mean error is not zero locally. If the discrepancy depends on input or state, its covariance structure is not constant. Replacing all of this with iid noise can misrepresent both parameter information and predictive uncertainty.

The consequences are not limited to underestimated or overestimated standard errors. The assumed error covariance determines which directions in observation space are considered informative. In a Gaussian inverse problem with covariance $\Sigma$, the objective involves

Changing $\Sigma$ changes the metric in which model-data agreement is evaluated. Treating correlated structural error as independent noise can therefore alter the calibrated parameter itself.

Conversely, adding a highly flexible discrepancy process can create the opposite problem. If both $F(\theta)$ and $\delta$ are allowed to explain the same patterns, then the decomposition

may not identify $\theta$ uniquely. A change in $\theta$ can be offset by a change in $\delta$. Model-discrepancy analysis therefore creates a new identifiability problem rather than providing a universal correction for misspecification.

This tension lies at the centre of statistical calibration. Ignoring discrepancy biases parameters when the forward model is wrong. Allowing discrepancy to be completely flexible can make the calibration parameter unidentifiable. A useful discrepancy model must therefore encode enough structure to represent plausible model inadequacy without being able to mimic every parameter-induced change arbitrarily.

Bayesian calibration makes the confounding explicit

Kennedy and O'Hagan formulated a widely influential Bayesian approach to computer-model calibration in which the physical process is represented as

where $f(x,\theta)$ is the simulator and $\delta(x)$ is a discrepancy function, often modelled with a Gaussian process. Observations then satisfy

The formulation has an important conceptual advantage: it refuses to treat the simulator as exact. Prediction uncertainty can account both for uncertainty about the calibration parameters and for the residual difference between simulator and reality.

The difficulty is that $\theta$ and $\delta(x)$ can be strongly confounded. If changing $\theta$ produces a smooth change in the simulator output and the discrepancy prior also allows similar smooth functions, the same data can be explained by shifting the parameter, shifting the discrepancy, or some combination of both. The posterior for $\theta$ is then influenced strongly by prior assumptions on the scale and smoothness of $\delta$.

Brynjarsdóttir and O'Hagan later emphasised that this is not a small technical inconvenience. Without scientifically meaningful prior information about the discrepancy, calibration parameters can be biased and difficult to interpret. A flexible discrepancy term can improve prediction of the physical system while simultaneously weakening inference about the physical meaning of $\theta$.

That distinction should be made explicit in applied work. If the objective is prediction, a calibrated simulator plus discrepancy process may perform very well even when $\theta$ is not recovered accurately. If the objective is scientific parameter estimation, predictive adequacy is not enough. One must ask whether parameter changes and discrepancy functions are distinguishable under the design and priors.

The same issue appears in deterministic regularisation. A penalty can force discrepancy into one function class and parameters into another. Bayesian calibration simply makes the assumptions probabilistic and therefore easier to inspect. Neither framework eliminates the fundamental confounding.

One useful strategy is orthogonalisation. If discrepancy is constrained to lie partly outside the parameter-sensitivity space, it becomes harder for it to imitate parameter changes. In a local linearisation,

the columns of $J$ span the observation-space directions generated by small parameter changes. Discrepancy components inside this span are locally confounded with parameters. Restricting discrepancy away from those directions can improve identifiability, although the restriction itself is a modelling assumption and may be scientifically wrong if real discrepancy genuinely resembles parameter effects.

The geometry is therefore the same as in the linear projection argument. Parameter interpretation depends on which part of model error is allowed to overlap with the forward model's sensitivity space.

Numerical error is also model discrepancy

Not all discrepancy comes from missing science. Numerical approximation can create the same problem. Suppose the physical forward operator is

but the inverse uses a discretised solver

where $h$ denotes mesh width, time step or another resolution parameter. The numerical discrepancy is

If $h$ is sufficiently small relative to observational uncertainty, the error may be negligible. If not, calibration can compensate for numerical error exactly as it compensates for missing physics.

This produces a subtle failure mode. A modeller may refine the optimisation algorithm, compute gradients accurately and obtain apparently converged parameters while the forward solver itself remains too coarse for the requested inference. The inverse is then solving the discretised problem correctly, but the discretised problem is not close enough to the intended physical problem.

A good inverse analysis should therefore include numerical-convergence checks on the quantities that matter for calibration. It is not enough to show that the forward solution looks visually unchanged under mesh refinement. One should examine whether inferred parameters, likelihood values, sensitivities or posterior summaries remain stable as the numerical approximation improves.

The issue becomes especially important when observational noise is small. High-quality data expose discrepancies that would have been negligible under noisier measurements. As instruments improve, forward-model and discretisation errors can become the dominant uncertainty source.

The same reasoning applies to surrogate models and emulators. If an expensive simulator is approximated by a neural network, Gaussian-process emulator or reduced-order model, then emulator error contributes another discrepancy term. Calibration against the emulator without propagating emulator uncertainty can produce overconfident parameter estimates even if the original simulator would not.

The inverse crime makes synthetic validation look better than reality

A particularly misleading form of model agreement occurs when synthetic data are generated with exactly the same forward model used for inversion. Suppose data are created as

and the inverse then estimates $\theta$ using the same discretisation $F_h$. The synthetic experiment contains no model discrepancy by construction. If the inverse method is otherwise implemented correctly, parameter recovery can look excellent.

This is known as an inverse crime. The term does not imply misconduct. It describes a validation design that makes the inverse problem artificially favourable by allowing the inversion model to share exactly the same modelling assumptions, discretisation and numerical representation as the data generator.

The danger is strongest when synthetic validation is used to claim robustness to real data. Real observations are generated by the physical system, not by the code. They contain discrepancy from omitted physics, imperfect boundary conditions, calibration drift, numerical approximations and unmodelled heterogeneity. A method tested only on self-generated data may therefore demonstrate algorithmic correctness without demonstrating scientific robustness.

A stronger simulation study deliberately separates the data-generating process from the inversion model. Synthetic truth can be generated on a finer mesh than the one used for inversion, with additional physical terms, alternative boundary conditions or different parameterisations. Noise can be correlated rather than iid. The aim is to test the inverse under controlled misspecification rather than only under exact model agreement.

This is analogous to training and testing a predictive model under the same artificial data-generating assumptions. Success proves that the algorithm behaves as expected under those assumptions. It does not show that the assumptions resemble reality closely enough for the scientific application.

Posterior predictive fit is necessary but not sufficient

Posterior predictive checks are often used to assess whether a Bayesian model can reproduce important features of the observed data. If replicated data

systematically fail to resemble the observations, the model is inadequate in some relevant sense. This is valuable information.

The converse is weaker. A model that reproduces the observations well may still contain biased calibration parameters because discrepancy has been absorbed into them. Good predictive fit can coexist with wrong mechanism. The linear projection example already demonstrated the extreme case: when discrepancy lies inside the column space of $A$, the misspecified model can fit perfectly.

Posterior predictive checks should therefore be designed around features that are difficult for parameter compensation to reproduce. Temporal derivatives, phase relationships, responses to controlled perturbations, cross-variable constraints or out-of-regime predictions can be more informative than reproducing the calibration trajectory itself.

External validation is particularly important. If parameters are physically meaningful and the model is mechanistically adequate, the same calibrated values should often retain explanatory value across new experimental conditions. If parameter estimates shift substantially whenever forcing, boundary conditions or observation locations change, that instability can indicate structural discrepancy rather than ordinary sampling variation.

The strongest test is often intervention. A parameter and a discrepancy term may be confounded under passive observation but respond differently when the system is deliberately excited. Experimental design can therefore separate model error from parameter uncertainty by choosing inputs that generate different signatures for competing explanations.

Experimental design can reveal compensation

Suppose two parameter values and one discrepancy mechanism produce almost indistinguishable outputs under the current experiment. Repeating the same experiment more precisely may reduce measurement noise without resolving the ambiguity. A new experiment that changes the input, boundary condition or observation location can alter the sensitivity geometry and make the explanations separable.

Let the local parameter sensitivities be the columns of

and let a discrepancy basis be represented by columns of $B$. If

and

overlap strongly, parameter and discrepancy effects are difficult to distinguish. An experimental design that changes $J_\theta$ relative to $B$ can reduce the overlap.

This is one reason collecting more data is not synonymous with collecting more information. One thousand repeated measurements along an uninformative trajectory can leave the same confounding intact. A handful of measurements under strategically chosen perturbations can be much more useful.

Design should therefore consider model inadequacy explicitly. Traditional optimal design often seeks to maximise Fisher information for $\theta$ under the assumption that the forward model is correct. If discrepancy is plausible, a design that maximises parameter sensitivity may still be poor if those sensitivities are indistinguishable from likely discrepancy patterns. Robust design requires considering both.

The same principle applies to sensor placement. If model error is spatially structured, observing only locations where parameter and discrepancy effects align can produce biased calibration. Sensors placed where candidate mechanisms predict different spatial patterns provide more information about model adequacy than sensors placed solely where the response magnitude is largest.

A precise inverse can still be scientifically wrong

The strongest warning from model discrepancy is that conventional indicators of successful inference can all be present simultaneously. An optimiser can converge. The Hessian can be well conditioned. The posterior can be narrow. Residuals can be small. Cross-validation inside the calibration regime can look excellent. Repeated runs can reproduce the same parameter estimate. None of those facts proves that the parameter corresponds to the physical quantity named in the model.

Under misspecification, increasing the amount of data can make the problem more deceptive. Sampling uncertainty shrinks while structural bias remains. The posterior or confidence interval contracts around a pseudo-true parameter, the value that makes the wrong model approximate the data-generating system as closely as possible under the chosen loss or likelihood. The inverse becomes more statistically certain and not necessarily more physically correct.

This distinction is familiar in statistical misspecification theory, but it has a particularly concrete interpretation in mechanistic inverse problems. The calibrated parameter may be the value that compensates most effectively for the forward model's deficiencies. It is then a property of the model-system pair under the calibration design, not necessarily a property of the physical system alone.

That does not make such parameters useless. Effective parameters can be highly valuable for prediction within a restricted regime. Engineering models routinely use calibrated coefficients that summarise unresolved processes. The scientific mistake is to interpret an effective parameter as though it had been measured independently of the approximations that created it.

A disciplined inverse analysis should therefore separate at least four sources of uncertainty: measurement noise, parameter uncertainty conditional on the model, numerical approximation error and structural model discrepancy. In many applications these categories interact and cannot be identified perfectly. Making the distinction explicit is still better than collapsing all unexplained variation into one noise term.

Model discrepancy also changes how regularisation should be interpreted. A regulariser can stabilise the inversion of a wrong forward model just as effectively as it stabilises a correct one. Stability therefore says nothing about model adequacy. An inverse can be both well regularised and systematically biased.

The logical sequence for inverse modelling should consequently extend beyond identifiability and stability. First ask whether the observations can distinguish the target parameter under the assumed forward model. Then ask whether the inverse is stable at realistic noise levels. Finally ask whether the forward model is accurate enough that the recovered parameter still has the intended scientific meaning. The first two questions concern the mathematical inverse of the model. The third concerns whether the model itself deserves to be inverted.

A wrong forward model can produce a precise wrong inverse because parameter estimation is willing to use parameters to explain whatever the model cannot represent separately. The inverse problem does not know which part of the fit is mechanism and which part is compensation. That distinction has to be created by model criticism, external validation and experimental design.

References

Brynjarsdóttir, J., & O'Hagan, A. (2014). Learning about physical parameters: the importance of model discrepancy. Inverse Problems, 30(11), 114007. https://doi.org/10.1088/0266-5611/30/11/114007

Kaipio, J., & Somersalo, E. (2005). Statistical and Computational Inverse Problems. Springer.

Kennedy, M. C., & O'Hagan, A. (2001). Bayesian calibration of computer models. Journal of the Royal Statistical Society: Series B, 63(3), 425–464. https://doi.org/10.1111/1467-9868.00294

Oberkampf, W. L., & Roy, C. J. (2010). Verification and Validation in Scientific Computing. Cambridge University Press.

Pernot, P., & Cailliez, F. (2017). A critical review of statistical calibration/prediction models handling data inconsistency and model inadequacy. AIChE Journal, 63(11), 4642–4665. https://doi.org/10.1002/aic.15781

Stuart, A. M. (2010). Inverse problems: A Bayesian perspective. Acta Numerica, 19, 451–559. https://doi.org/10.1017/S0962492910000061