Necessity

A condition is necessary for an outcome when the outcome is a subset of the condition: wherever the outcome appears, the condition appears too. It is the mirror of sufficiency, and it answers a different question.

	Question	Set relation	Consistency
Sufficiency	Is this enough to produce the outcome?	X ⊆ Y	Σ min(X,Y) / Σ X
Necessity	Can the outcome occur without this?	Y ⊆ X	Σ min(X,Y) / Σ Y

The two are duals: necessity of X for Y is sufficiency of Y for X with the roles exchanged. Neither implies the other, and a condition can be both, either, or neither.

Screening

from setqca import necessity_analysis

analysis = necessity_analysis(
    data,
    outcome="SURV",
    conditions=["DEV", "URB", "LIT", "IND", "STB"],
    consistency_threshold=0.90,
)
print(analysis)
print(analysis.to_frame())

Every condition is screened in both directions by default. A condition's absence can be necessary when its presence is not, and screening only presence is a common way to miss the finding.

Results are typed objects, not just a frame:

analysis.necessary  # consistent and non-trivial
analysis.trivial  # consistent but uninformative
analysis.candidates  # everything screened

The trivialness problem

This is the part that matters most.

A condition present in almost every case is a superset of almost anything. It will show near-perfect necessity consistency while telling you nothing — you cannot explain a rare outcome with a ubiquitous condition.

data = pd.DataFrame({"ubiquitous": [1.0, 1.0, 1.0, 1.0], "Y": [0.8, 0.7, 0.2, 0.1]})

ubiquitous scores consistency 1.000 — apparently a perfect necessary condition. It is an artefact of prevalence.

Relevance of necessity is what exposes it:

RoN = Σ (1 − X) / Σ (1 − min(X, Y))

The more prevalent X is, the smaller the numerator, and the closer RoN falls to zero. In the example above RoN is exactly 0.

setqca therefore reports both, and separates the two lists:

Necessary:
  ~C [cons=1.000, cov=0.900, RoN=0.909]
Consistent but trivial (prevalent enough to be uninformative):
  B [cons=1.000, RoN=0.000, prevalence=1.000]

A candidate is necessary only when it clears both thresholds. Clearing consistency alone makes it trivial, which is reported rather than left for the reader to notice.

Consistency alone is not evidence of necessity

A high necessity consistency with a low RoN is the single most common way a QCA writes up a finding that isn't there. Always report both.

Compound conditions

Only disjunctions are screened, and this is a mathematical result rather than a limitation:

consistency(A*B) ≤ min(consistency(A), consistency(B))    because min(A,B,Y) ≤ min(A,Y)
consistency(A+B) ≥ max(consistency(A), consistency(B))    because min(max(A,B),Y) ≥ min(A,Y)

A conjunction can never be more necessary than its own components, so testing conjunctions adds nothing. A union can be necessary when neither part is — the SUIN condition of the literature, a sufficient part of an insufficient but necessary condition.

analysis = necessity_analysis(
    data,
    outcome="SURV",
    conditions=["DEV", "URB", "LIT"],
    max_disjunction_size=2,
)

Beware that the number of unions grows quickly: with k literals and size n there are C(k, n) of them, and screening many raises the chance that one clears the threshold by luck. Treat unions as hypotheses to examine, not findings to report.

Necessity is not causation

Necessity is a statement about set relations in the data you have. It does not establish that the condition produces the outcome, that removing it would prevent the outcome, or that the relation holds outside your cases. A constant-across-cases condition is necessary in the data and may be causally irrelevant, and vice versa.

setqca.analysis.necessity

Systematic analysis of necessary conditions.

A condition is necessary for an outcome when the outcome is a subset of the condition: wherever the outcome is present, the condition is present too. This is the mirror image of sufficiency, and the two answer different questions — necessity asks what cannot be missing, sufficiency asks what is enough.

Necessity is easy to claim and easy to overclaim. A condition present in almost every case is a superset of almost anything, so it will show near-perfect necessity consistency while explaining nothing. That is trivial necessity, and it is reported here rather than left for the reader to notice.

Which compounds are worth testing

Only disjunctions. For the minimum/maximum operators:

• consistency(A*B) <= min(consistency(A), consistency(B)), because min(A, B, Y) <= min(A, Y). A conjunction can therefore never be more necessary than its own components, and testing conjunctions adds nothing.
• consistency(A+B) >= max(consistency(A), consistency(B)), because min(max(A, B), Y) >= min(A, Y). A union can be necessary when neither part is, which is the SUIN condition of the literature — a sufficient part of an insufficient but necessary condition.

So conjunctions are excluded on mathematical grounds, not for lack of effort.

NecessityCandidate dataclass

NecessityCandidate(
    expression: str,
    fit: NecessityFit,
    prevalence: float,
    consistent: bool,
    relevant: bool,
)

One candidate necessary condition, with everything needed to judge it.

Attributes:

Name	Type	Description
expression	str	The candidate in standard QCA notation, e.g. "~DEV" or "DEV+URB".
fit	NecessityFit	Consistency, coverage and relevance of necessity.
prevalence	float	Mean membership of the candidate across cases. A prevalence near 1 is what makes a necessity claim trivial.
consistent	bool	Whether consistency reached the threshold.
relevant	bool	Whether relevance of necessity reached its threshold.

necessary property

necessary: bool

Return whether the candidate is both consistent and non-trivial.

trivial property

trivial: bool

Return whether the candidate is consistent but irrelevant.

This is the dangerous combination: the numbers look like necessity, but the condition is so prevalent that the claim carries no information.

NecessityAnalysis dataclass

NecessityAnalysis(
    outcome: str,
    consistency_threshold: float,
    relevance_threshold: float,
    candidates: tuple[NecessityCandidate, ...],
)

The result of screening candidates for necessity.

necessary property

necessary: tuple[NecessityCandidate, ...]

Return candidates that are consistent and non-trivial.

trivial property

trivial: tuple[NecessityCandidate, ...]

Return candidates that pass on consistency but fail on relevance.

to_frame

to_frame() -> DataFrame

Return a tidy table, sorted by consistency then relevance.

Returns:

Type	Description
DataFrame	Columns condition, consistency, coverage, RoN, prevalence, necessary and trivial.

Source code in src/setqca/analysis/necessity.py

def to_frame(self) -> pd.DataFrame:
    """Return a tidy table, sorted by consistency then relevance.

    Returns
    -------
    pandas.DataFrame
        Columns ``condition``, ``consistency``, ``coverage``, ``RoN``,
        ``prevalence``, ``necessary`` and ``trivial``.
    """
    frame = pd.DataFrame(
        {
            "condition": [item.expression for item in self.candidates],
            "consistency": [item.fit.consistency for item in self.candidates],
            "coverage": [item.fit.coverage for item in self.candidates],
            "RoN": [item.fit.ron for item in self.candidates],
            "prevalence": [item.prevalence for item in self.candidates],
            "necessary": [item.necessary for item in self.candidates],
            "trivial": [item.trivial for item in self.candidates],
        }
    )
    return frame.sort_values(
        ["consistency", "RoN"], ascending=False, kind="stable"
    ).reset_index(drop=True)

necessity_analysis

necessity_analysis(
    data: DataFrame,
    *,
    outcome: str,
    conditions: list[str] | tuple[str, ...],
    consistency_threshold: float = DEFAULT_CONSISTENCY,
    relevance_threshold: float = DEFAULT_RELEVANCE,
    include_absence: bool = True,
    max_disjunction_size: int = 1,
) -> NecessityAnalysis

Screen conditions, and optionally their disjunctions, for necessity.

Parameters:

Name	Type	Description	Default
data	DataFrame	Calibrated memberships in [0, 1]. Works for crisp and fuzzy alike, since crisp membership is the {0, 1} special case.	required
outcome	str	Name of the outcome column.	required
conditions	list of str or tuple of str	Condition columns to screen.	required
consistency_threshold	float	Minimum necessity consistency for a candidate to count as consistent.	0.90
relevance_threshold	float	Minimum relevance of necessity. Candidates that pass on consistency but fail here are reported as trivial rather than necessary.	0.50
include_absence	bool	Also screen the negation of every condition. A condition's absence can be necessary when its presence is not.	True
max_disjunction_size	int	Largest union to test. 1 screens single conditions only; 2 adds every pair, and so on. Conjunctions are never tested — see the module docstring for why they cannot help.	1

Returns:

Type	Description
NecessityAnalysis	Every candidate screened, with those that are necessary and those that are merely trivial identified separately.

Raises:

Type	Description
ValueError	If a threshold is out of range, max_disjunction_size is below 1, or the data are not calibrated.
KeyError	If a named column is absent.

Examples:

>>> analysis = necessity_analysis(
...     data, outcome="SURV", conditions=["DEV", "URB", "LIT"]
... )
>>> analysis.to_frame()

Source code in src/setqca/analysis/necessity.py

def necessity_analysis(
    data: pd.DataFrame,
    *,
    outcome: str,
    conditions: list[str] | tuple[str, ...],
    consistency_threshold: float = DEFAULT_CONSISTENCY,
    relevance_threshold: float = DEFAULT_RELEVANCE,
    include_absence: bool = True,
    max_disjunction_size: int = 1,
) -> NecessityAnalysis:
    """Screen conditions, and optionally their disjunctions, for necessity.

    Parameters
    ----------
    data : pandas.DataFrame
        Calibrated memberships in ``[0, 1]``. Works for crisp and fuzzy alike,
        since crisp membership is the ``{0, 1}`` special case.
    outcome : str
        Name of the outcome column.
    conditions : list of str or tuple of str
        Condition columns to screen.
    consistency_threshold : float, default 0.90
        Minimum necessity consistency for a candidate to count as consistent.
    relevance_threshold : float, default 0.50
        Minimum relevance of necessity. Candidates that pass on consistency but
        fail here are reported as trivial rather than necessary.
    include_absence : bool, default True
        Also screen the negation of every condition. A condition's absence can
        be necessary when its presence is not.
    max_disjunction_size : int, default 1
        Largest union to test. ``1`` screens single conditions only; ``2`` adds
        every pair, and so on. Conjunctions are never tested — see the module
        docstring for why they cannot help.

    Returns
    -------
    NecessityAnalysis
        Every candidate screened, with those that are necessary and those that
        are merely trivial identified separately.

    Raises
    ------
    ValueError
        If a threshold is out of range, ``max_disjunction_size`` is below 1, or
        the data are not calibrated.
    KeyError
        If a named column is absent.

    Examples
    --------
    >>> analysis = necessity_analysis(  # doctest: +SKIP
    ...     data, outcome="SURV", conditions=["DEV", "URB", "LIT"]
    ... )
    >>> analysis.to_frame()  # doctest: +SKIP
    """
    if not 0.0 <= consistency_threshold <= 1.0:
        raise ValueError("consistency_threshold must be in [0, 1].")
    if not 0.0 <= relevance_threshold <= 1.0:
        raise ValueError("relevance_threshold must be in [0, 1].")
    if max_disjunction_size < 1:
        raise ValueError("max_disjunction_size must be at least 1.")

    names = validate_columns(data, conditions)
    if not names:
        raise ValueError("At least one condition is required.")
    validate_columns(data, [outcome])

    y = validate_membership(data[outcome].to_numpy(), name=outcome)

    # Literals first: each condition present, and optionally absent.
    literals: list[tuple[str, FloatArray]] = []
    for name in names:
        values = validate_membership(data[name].to_numpy(), name=name)
        literals.append((name, values))
        if include_absence:
            literals.append((f"~{name}", 1.0 - values))

    candidates = [
        _candidate(expression, membership, y, consistency_threshold, relevance_threshold)
        for expression, membership in literals
    ]

    # Unions of literals. A union can be necessary when no part of it is, which
    # is the only compound worth screening.
    for size in range(2, max_disjunction_size + 1):
        for chosen in combinations(literals, size):
            expression = "+".join(name for name, _ in chosen)
            membership = np.maximum.reduce([values for _, values in chosen])
            candidates.append(
                _candidate(expression, membership, y, consistency_threshold, relevance_threshold)
            )

    return NecessityAnalysis(
        outcome=outcome,
        consistency_threshold=consistency_threshold,
        relevance_threshold=relevance_threshold,
        candidates=tuple(candidates),
    )