Confidence Sets Are Not Just Intervals

A confidence interval is one of the most familiar objects in statistics.

We learn to report

and to interpret every value between the two endpoints as belonging to the uncertainty region.

That geometry becomes so familiar that it is easy to forget the more general object:

A confidence set need not be one interval.

It can be disconnected.

It can be empty.

It can contain isolated points on a finite grid.

It can have several separated components.

And after projection from a multidimensional parameter space, the set of accepted target values can have gaps that should not be filled simply because drawing one interval looks cleaner.

The distinction is not cosmetic.

If the statistical procedure rejects a value in the middle of two accepted regions, replacing the set by its convex hull changes the inferential statement.

Test Inversion Comes First

Suppose a pointwise test evaluates

for every candidate value (\theta_0).

Let

be the rejection decision, where (1) means reject.

The inverted confidence set is

Equivalently, if the pointwise test reports a p-value (p(\theta)),

up to whatever tie convention the procedure defines.

There is nothing in that definition requiring (\mathcal C_{1-\alpha}) to be connected.

The interval shape appears only in special problems where the test statistic has suitable monotonicity or convexity properties.

Those properties are common enough to shape intuition.

They are not universal.

A Toy Example With Two Components

Consider a scalar parameter (\theta) and an estimator

where

Suppose we observe

For a candidate (\theta_0), define the pointwise z-statistic

The two-sided p-value is

Now invert the 5% test.

The model mean

is small near both

and

It is much larger around

So values near both (-1) and (+1) can be compatible with the observation even though values in between are rejected.

A short computation makes this visible.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
import numpy as np
from scipy.stats import norm

alpha = 0.05
y_obs = 0.04
sigma = 0.08

grid = np.linspace(-1.8, 1.8, 721)
mu = (grid**2 - 1.0) ** 2
z = (y_obs - mu) / sigma
p = 2.0 * norm.sf(np.abs(z))
accepted = p > alpha

accepted_grid = grid[accepted]

To recover connected components on the represented grid:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
components = []
start = None
previous = None

for theta, keep in zip(grid, accepted):
    if keep and start is None:
        start = theta
    if keep:
        previous = theta
    elif start is not None:
        components.append((start, previous))
        start = None
        previous = None

if start is not None:
    components.append((start, previous))

print(components)

The result consists of two separated regions, approximately

The exact endpoints vary slightly with grid resolution, but the geometry does not.

There are two components.

The Convex Hull Gives a Different Answer

The smallest ordinary interval containing the confidence set is

That looks tidy.

It also includes values around zero.

But at (\theta=0), the model mean is

which is far from the observed (0.04).

The pointwise test strongly rejects it.

So replacing

by

changes the statement from

these two separated parameter regions were not rejected

into

every parameter value between the extreme endpoints belongs to the reported uncertainty region.

Those are different statements.

The convex hull is therefore not a harmless formatting choice.

It is a new set:

Why Disconnected Sets Arise

The toy model is deliberately simple, but the mechanism is general.

Disconnected confidence sets appear whenever the mapping from parameters to observables is non-monotone or many-to-one.

Suppose the data identify some function

well, but (g) is not injective.

Then several separated parameter values can imply similar observable behaviour.

If

for distant (\theta_1) and (\theta_2), pointwise inversion can preserve both regions.

This is not a numerical pathology.

It is information about the identification geometry.

The set shape is telling us that the data distinguish some parameter values but not others in a globally simple ordering.

Intervals Encode More Than Uncertainty

Reporting one interval implicitly says something geometric:

That is a connectedness assumption.

Sometimes the statistical problem guarantees it.

Sometimes software silently imposes it after the fact.

Those are very different situations.

A useful discipline is therefore:

Do not decide in advance that the result must be an interval.

Finite-Grid Inversion Is Still Set Inversion

In computational work, the continuous parameter space is often replaced by a grid

Then the exact object computed by the software is

This is a finite represented set.

It is not automatically an approximation to one single continuous interval.

If the accepted grid points are

then those are the accepted represented values.

A reporting layer can identify two adjacent runs,

and

but the notation should not imply that every real number inside those coarse intervals was explicitly tested.

Grid inversion answers a finite question unless extra theory or numerical refinement justifies more.

Adjacency Is a Computational Concept

On an ordered grid

it is natural to call consecutive accepted grid values one component.

Suppose the decisions are

where (1) means accepted.

Then there are two accepted runs.

This is useful bookkeeping.

But an important boundary remains:

if two adjacent coarse-grid points have the same decision, a coarse inversion has learned nothing about possible hidden changes between them.

The statement

does not logically imply

That implication requires smoothness or shape information the finite grid may not provide.

Refinement Sharpens Detected Boundaries

Suppose adjacent grid points have opposite decisions:

Then there is an observed decision transition between (a) and (b) on the coarse grid.

A natural numerical refinement repeatedly evaluates midpoints and narrows the bracket.

If

is the current bracket, evaluate

Keep the half-bracket whose endpoints retain opposite decisions.

Continue until

This can localize a detected boundary much more precisely.

It does not discover boundaries hidden inside coarse intervals whose endpoints had the same decision.

That is a different search problem.

So numerical refinement changes

not automatically

A Boundary Bracket Is Not an Exact Endpoint

If a refinement routine returns

with

it is tempting to report the midpoint as the confidence-set endpoint.

That can overstate what was established.

What the computation actually proved is closer to:

the implemented decision changes somewhere inside a bracket this narrow, under the same Monte Carlo and test configuration.

If the pointwise p-value itself is Monte Carlo estimated, then numerical refinement does not remove Monte Carlo uncertainty.

One can have a very narrow numerical bracket around a noisy decision boundary.

Those are separate error sources:

Floating-Point Arithmetic Adds Another Boundary

Bisection sounds infinitely refinable in real arithmetic.

Computers do not use real arithmetic.

Eventually, for adjacent representable floating-point numbers (a<b), the computed midpoint can equal one endpoint:

or

Then the algorithm cannot subdivide the bracket further.

That state is not convergence merely because the code stopped.

A careful implementation distinguishes

from

If the final width is still larger than the requested tolerance but no representable midpoint exists, the honest status is

Set-valued inference makes these numerical semantics visible because boundaries are part of the returned scientific object.

Now Add a Nuisance Parameter

The geometry becomes more interesting with a parameter vector

where (\psi) is the target and (\lambda) is nuisance.

Suppose we can compute a pointwise p-value

for every complete parameter vector.

The joint inverted set is

The projected confidence set for (\psi) is conceptually

Equivalently, if a valid profile p-value is available,

then

This is where computational claims need care.

Finite-Grid Projection Is a Finite Maximum

Suppose the software evaluates only a finite parameter grid

Then for a represented target value (\psi), the computable profile quantity is

That is an exact maximum over represented rows.

It is not automatically

over the continuous nuisance space.

The distinction matters because missing the true supremum can make the profile p-value too small.

A smaller profile p-value can incorrectly reject a target value.

So calling a finite-grid maximum a continuous profile supremum is not merely imprecise language. It can hide an anti-conservative approximation.

A Simple Projection Example

Consider this finite joint grid:

At level (\alpha=0.05), the finite-grid profile p-values are

and

The projected represented set is therefore

Its convex hull is

But the represented target values

all have profile p-values below 0.05.

Filling the gap would erase exactly the information the projection produced.

Projection Can Create or Preserve Disconnectedness

A joint accepted set can have complicated geometry.

Projecting it onto one coordinate can simplify that geometry, but it need not make it connected.

Imagine two separated accepted islands in ((\psi,\lambda))-space:

If their (\psi)-ranges are separated, the projection remains disconnected:

Again, the gap is information.

It says that no accepted nuisance configuration on the represented grid rescued those intermediate target values.

Witnesses Are Useful

For each projected target value (\psi), it is often useful to retain one nuisance configuration attaining the finite-grid maximum:

This row is a witness.

It answers:

Which represented nuisance configuration made this target value look most plausible?

Witnesses are particularly useful for debugging and interpretation.

But they have their own invariance rule.

If the source grid rows are permuted, the absolute row number of the witness may change.

The scientific result should not.

The correct invariant is therefore not

but

That is a good example of testing semantics rather than representation details.

Row Order Must Not Change the Set

A finite parameter grid is a mathematical set represented as a table.

Permuting table rows should not change

• represented target values,
• profile p-values,
• acceptance decisions,
• connected components,
• projected accepted values.

Formally, if (\pi) is any permutation of the source rows,

up to ordering conventions and witness row labels.

This is exactly the sort of invariant that catches implementation bugs ordinary example tests may miss.

Scalar Transformations Need the Same Discipline

Sometimes the target is not one named coordinate but a transformation

For example,

On a finite grid, several source rows may map to the same scalar target value.

Then the finite-grid profile p-value is

One subtle issue appears immediately: how are equal transformed values defined?

If the implementation groups by exact numerical equality, then

and

may be distinct represented target values.

A tolerance-based grouping rule would be a different inferential/computational contract.

Neither should be introduced accidentally.

The grouping semantics need to be explicit because they determine the set being reported.

A Descriptive Hull Can Still Be Useful

Sometimes users genuinely want a quick range summary.

For a finite accepted scalar set

one can report

That can be useful as a descriptive hull.

The important word is descriptive.

It should not be interpreted as

or

or

A good software interface can return both the exact represented accepted values and the hull while naming them differently.

The problem is not computing the hull.

The problem is silently replacing the set with it.

Empty Sets Are Legitimate Results

Another habit inherited from interval thinking is discomfort with empty confidence sets.

But inversion can produce

On a finite grid, this simply means every represented candidate was rejected.

Possible interpretations include:

• the parameter grid missed the plausible region,
• the model is badly incompatible with the data,
• the test is too aggressive under the current approximation,
• the represented bounds are too narrow,
• the assumptions defining the model are violated.

The software should not repair emptiness by automatically returning the full range or the nearest rejected point.

An empty set is diagnostic information.

One-Point Sets Are Legitimate Too

At the other extreme, a confidence set may contain exactly one represented value:

There are no transition brackets to refine unless neighbouring represented points exist and disagree.

A robust implementation should handle this without pretending there must be two endpoints.

Again, the set abstraction is cleaner than the interval abstraction.

The set can contain zero, one or many components without requiring special conceptual exceptions.

Coverage Claims Must Match the Computed Object

Suppose a joint finite-grid confidence set has the property that, if the true parameter vector is represented, it is included with probability at least (1-\alpha):

Then projection gives the represented-grid implication

But this does not establish coverage for an off-grid (\psi_0) or for arbitrary nuisance values absent from the grid.

The difference between

and

must remain visible in the claim.

Why Generic Continuous Optimization Can Be Dangerous

A natural response is:

Why not just optimize over the nuisance parameter continuously?

Sometimes that is exactly the right solution.

But it needs justification.

Suppose the profile p-value is

If the objective is smooth, well behaved and globally optimizable, continuous profiling may work very well.

In more complicated inferential procedures the objective can contain

• moment-selection discontinuities,
• finite-Monte-Carlo step functions,
• piecewise statistics,
• boundary effects,
• multiple local maxima.

A local optimizer returning one maximum does not certify the global supremum.

If it misses a higher nuisance configuration, the reported profile p-value is too small.

That error goes in the dangerous direction:

can make the projected set too narrow.

A finite-grid maximum is limited, but honest:

is a precise computational statement.

Partial Identification Makes the Set View Natural

In point-identified problems, we often think first about an estimator

and then attach an interval around it.

In partially identified problems, the population object itself may be a set

The data-generating distribution does not identify one unique parameter vector even with infinite data.

Moment inequalities provide a common example. If

then the identified set is

This population set can itself be non-convex or disconnected depending on the moment functions and parameterization.

In that setting, forcing inference into one interval is especially unnatural.

Set-valued inference is not a complication added by software.

It reflects the object being learned.

The Geometry Is Part of the Result

Suppose two analysts report uncertainty for the same parameter.

The first reports

The second reports

The endpoint range is identical.

The inferential content is not.

The second result says that values near zero are inconsistent with the data under the test.

That is substantive information.

A downstream scientific decision may depend on it.

For example, perhaps negative and positive parameter values imply two qualitatively different mechanisms, while values near zero imply no meaningful effect. A disconnected set can say:

the data support either mechanism, but not the near-zero explanation.

The convex hull erases that distinction entirely.

What Software Should Return

For one-dimensional inversion, I would want a result object to preserve at least

• the represented grid,
• pointwise p-values or decisions,
• accepted values,
• connected components,
• the confidence level,
• test method and Monte Carlo settings,
• whether the set is empty,
• parameter bounds,
• any refined transition brackets,
• convergence and numerical-stall diagnostics.

For projection, I would add

• represented target values,
• finite-grid profile p-values,
• accepted projected values,
• witness rows or witness parameters,
• connected components where order is meaningful,
• an explicitly labelled descriptive hull if one is provided.

That may seem like more information than two endpoints.

It is exactly the information needed to avoid pretending the geometry is simpler than it is.

Plot the Decisions, Not Just the Hull

A useful visualization is often extremely simple.

For one-dimensional inversion, plot

against (\theta) with a horizontal line at (\alpha).

The confidence set is the region where the p-value lies above the line.

Disconnectedness becomes visually obvious.

On a finite grid, plotting accepted and rejected points directly can be even more honest than drawing filled intervals.

For projections, plot the finite-grid profile p-values

against represented target values.

Then the reader can see both the accepted components and the gaps.

The Main Statistical Lesson

The phrase "confidence interval" is so common that it can quietly turn into a modelling assumption about the geometry of uncertainty.

The safer hierarchy is

Not

When the accepted parameter region is disconnected, the gaps are part of the evidence.

When the computation is performed on a finite grid, the grid is part of the inferential boundary.

When nuisance parameters are projected out, a finite maximum over represented rows is not the same thing as a continuous supremum.

And when a hull is useful for presentation, it should be labelled as a hull rather than silently substituted for the set.

The broader principle is simple:

$$ \boxed{ \text{uncertainty has geometry, and the geometry should survive the software.} }