API reference¶

The package exports the classifier and its explanation types from subspaceknn. Plotting lives in subspaceknn.plotting and needs the optional plot extra.

SubspaceKNNClassifier ¶

SubspaceKNNClassifier(
    *,
    n_neighbors: int = 5,
    subspace_size: int | Sequence[int] = 2,
    n_subspaces: int | None = 5,
    selection: Selection = "complementary",
    max_votes: int = 50,
    balance_classes: bool = True,
    max_candidates: int | None = 1000,
    voting: Voting = "soft",
    weighting: Weighting = "score",
    cv: Literal["loo"] | int | BaseCrossValidator = "loo",
    scoring: str | Callable[..., float] = "f1_macro",
    knn_weights: Literal["uniform", "distance"] = "uniform",
    metric: str = "minkowski",
    n_jobs: int | None = None,
)

Weighted vote of k-nearest-neighbour classifiers fitted on small feature subspaces.

The estimator fits a k-nearest-neighbour model on every subset of subspace_size features (or every subset of each size when a sequence of sizes is given) and computes each subset's out-of-fold class probabilities on the training data, by exact leave-one-out by default. It then chooses at most n_subspaces of them to vote on new samples.

The default, complementary selection, builds the ensemble greedily: at every step it adds the subspace that most reduces the class-balanced Brier score of the ensemble's out-of-fold probabilities, allowing a subspace to be added again, and keeps the best ensemble found. A subspace therefore earns its place by what it adds to the others, not by how well it does alone, and its weight is the number of times it was chosen. Ranked selection instead keeps the subspaces with the best individual scores, as ikNN does.

Because the members of the ensemble live in spaces of one, two or three features, every prediction can be explained by looking at the neighbourhoods that produced it; see explain.

Parameters:

Name	Type	Description	Default
`n_neighbors`	`int`	Number of neighbours used by every subspace model.	`5`
`subspace_size`	`int or sequence of int`	Number of features in each subspace. A sequence enumerates subspaces of every listed size. Sizes larger than the number of features are ignored.	`2`
`n_subspaces`	`int or None`	Maximum number of distinct subspaces used for prediction, that is, the number of pictures an explanation contains. Complementary selection may use fewer when more would not improve the ensemble. `None` removes the limit.	`5`
`selection`	`('complementary', 'ranked')`	`"complementary"` builds the ensemble by greedy forward selection on out-of-fold probabilities, as described above. `"ranked"` keeps the `n_subspaces` subspaces with the highest individual `scoring` and weights them according to `weighting`.	`"complementary"`
`max_votes`	`int`	Number of greedy steps of complementary selection. Each step casts one vote and the best ensemble over all steps is kept, so the weights are multiples of `1 / n_votes` for some `n_votes <= max_votes`. Ignored by ranked selection.	`50`
`balance_classes`	`bool`	Whether the Brier score minimised by complementary selection weights each sample inversely to its class frequency, so that every class counts equally. `False` weights samples equally. Ignored by ranked selection.	`True`
`max_candidates`	`int or None`	Soft cap on the number of candidate subspaces. When the number of subsets exceeds it, features are first screened by the `scoring` of their one-dimensional model and only the best-scoring features are combined, as many as keep the candidate count within the cap. `None` evaluates every subset, which grows combinatorially with the number of features.	`1000`
`voting`	`('soft', 'hard')`	`"soft"` averages the class probabilities of the subspace models, `"hard"` averages their one-hot predictions. Complementary selection optimises whichever of the two is used for prediction.	`"soft"`
`weighting`	`('score', 'uniform')`	Weights of ranked selection: `"score"` weights each subspace by its individual score (negative scores are clipped to zero), `"uniform"` gives every subspace the same weight. Ignored by complementary selection, whose weights are its vote counts.	`"score"`
`cv`	`"loo", int or cross-validation generator`	How the out-of-fold probabilities are computed. `"loo"` is exact leave-one-out, obtained from a single neighbour query per subspace. An integer selects stratified k-fold with that many splits, reduced automatically when a class has fewer samples than splits; a splitter must partition the samples. When the training set is too small for the chosen scheme, the probabilities are computed on the training data itself.	`"loo"`
`scoring`	`str or callable`	Any scikit-learn scorer, evaluated on each subspace's out-of-fold predictions. It screens features, ranks subspaces under ranked selection, and is reported as each subspace's score. Higher must be better.	`"f1_macro"`
`knn_weights`	`('uniform', 'distance')`	Neighbour weighting passed to the subspace models.	`"uniform"`
`metric`	`str`	Distance metric passed to the subspace models.	`"minkowski"`
`n_jobs`	`int or None`	Parallelism of the neighbour queries, or of the cross-validation when `cv` is not `"loo"`.	`None`

Attributes:

Name	Type	Description
`classes_`	`ndarray of shape (n_classes,)`	Class labels.
`n_features_in_`	`int`	Number of features seen during `fit`.
`feature_names_in_`	`ndarray of shape (n_features_in_,)`	Feature names, only when `X` had string column names.
`screened_features_`	`ndarray of shape (n_screened,)`	Indices of the features retained after screening, all features when no screening was necessary.
`feature_screening_scores_`	`ndarray of shape (n_features_in_,) or None`	Out-of-fold score of each feature's one-dimensional model, only when screening took place.
`candidate_subspaces_`	`list of tuple of int`	Every subspace that was evaluated, in enumeration order.
`candidate_scores_`	`ndarray of shape (n_candidates,)`	Out-of-fold score of each candidate subspace on its own.
`subspaces_`	`list of tuple of int`	Subspaces used for prediction, from the highest to the lowest weight.
`subspace_scores_`	`ndarray of shape (n_selected,)`	Individual scores of the selected subspaces.
`subspace_weights_`	`ndarray of shape (n_selected,)`	Normalised voting weights of the selected subspaces; they sum to one.
`selection_path_`	`list of tuple or None`	Under complementary selection, one `(subspace, loss)` pair per vote of the final ensemble, in the order the votes were added, where `loss` is the ensemble's out-of-fold Brier score after that vote, class-balanced when `balance_classes` is true. `None` under ranked selection.
`estimators_`	`list of TieSharingKNeighborsClassifier`	Fitted subspace models, aligned with `subspaces_`. Points tied at the distance of the k-th neighbour share the remaining votes, so predictions do not depend on the order of the training data or on the platform.
`feature_scores_`	`ndarray of shape (n_features_in_,)`	Mean score of the selected subspaces containing each feature, zero for features that appear in none. A coarse measure of feature relevance.

Examples:

>>> from sklearn.datasets import load_iris
>>> from subspaceknn import SubspaceKNNClassifier
>>> X, y = load_iris(return_X_y=True)
>>> clf = SubspaceKNNClassifier(subspace_size=(1, 2)).fit(X, y)
>>> clf.subspaces_[0]
(2, 3)
>>> clf.explain(X[:1])[0].votes[0].feature_names
('x2', 'x3')

fit ¶

fit(X: ArrayLike, y: ArrayLike) -> SubspaceKNNClassifier

Evaluate every candidate subspace and select the ensemble.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_samples, n_features)`	Training data.	required
`y`	`array-like of shape (n_samples,)`	Class labels.	required

Returns:

Type	Description
`self`	The fitted estimator.

predict_proba ¶

predict_proba(X: ArrayLike) -> NDArray[float64]

Return the weighted average of the subspace models' class probabilities.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_samples, n_features)`	Samples to classify.	required

Returns:

Type	Description
`ndarray of shape (n_samples, n_classes)`	Class probabilities aligned with `classes_`; each row sums to one.

predict ¶

predict(X: ArrayLike) -> NDArray[Any]

Return the class with the highest ensemble probability for each sample.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_samples, n_features)`	Samples to classify.	required

Returns:

Type	Description
`ndarray of shape (n_samples,)`	Predicted class labels.

explain ¶

explain(X: ArrayLike) -> list[Explanation]

Explain the prediction for every sample in X.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_samples, n_features)`	Samples to explain.	required

Returns:

Type	Description
`list of Explanation`	One explanation per sample, each listing the vote of every subspace used by the ensemble, ordered from the highest to the lowest weight.

Explanation `dataclass` ¶

Explanation(
    prediction: Any,
    probabilities: NDArray[float64],
    classes: NDArray[Any],
    votes: tuple[SubspaceVote, ...],
)

How the ensemble arrived at the prediction for one sample.

Attributes:

Name	Type	Description
`prediction`	`Any`	Class label predicted by the ensemble.
`probabilities`	`ndarray of shape (n_classes,)`	Ensemble class probabilities, the weighted average of the votes.
`classes`	`ndarray of shape (n_classes,)`	Class labels, in the order used by `probabilities`.
`votes`	`tuple of SubspaceVote`	One entry per subspace used for prediction, ordered from the highest to the lowest weight.

agreement ¶

agreement() -> float

Return the weighted fraction of subspaces that voted for the ensemble prediction.

to_records ¶

to_records() -> list[dict[str, Any]]

Return the votes as plain dictionaries, one per subspace.

The result is suitable for pandas.DataFrame.from_records and contains, for every vote, the feature names, the subspace score and weight, the subspace prediction, whether it agrees with the ensemble, and one p(<class>) column per class.

SubspaceVote `dataclass` ¶

SubspaceVote(
    features: tuple[int, ...],
    feature_names: tuple[str, ...],
    score: float,
    weight: float,
    probabilities: NDArray[float64],
    prediction: Any,
)

The contribution of one feature subspace to a single prediction.

Attributes:

Name	Type	Description
`features`	`tuple of int`	Column indices of the features that span the subspace.
`feature_names`	`tuple of str`	Names of those features, taken from `feature_names_in_` when the estimator was fitted on a data frame and `x<i>` otherwise.
`score`	`float`	Cross-validated score of the subspace on the training data.
`weight`	`float`	Normalised weight of the subspace in the ensemble vote. Weights sum to one across the subspaces used for prediction.
`probabilities`	`ndarray of shape (n_classes,)`	Class probabilities produced by this subspace alone, aligned with `Explanation.classes`. Under hard voting this is a one-hot vector.
`prediction`	`Any`	Class label predicted by this subspace alone.

plot_subspaces ¶

plot_subspaces(
    estimator: SubspaceKNNClassifier,
    X: ArrayLike,
    y: ArrayLike,
    *,
    sample: ArrayLike | None = None,
    n_subspaces: int | None = None,
    grid_resolution: int = 100,
    panel_size: float = 4.0,
) -> Figure

Draw the training data in the best subspaces, with decision regions where possible.

Parameters:

Name	Type	Description	Default
`estimator`	`SubspaceKNNClassifier`	A fitted estimator.	required
`X`	`array-like of shape (n_samples, n_features)`	Data to draw, usually the training data.	required
`y`	`array-like of shape (n_samples,)`	Class labels of `X`.	required
`sample`	`array-like of shape (n_features,)`	A single sample to highlight with a star in every panel, for example a test point whose prediction is being explained.	`None`
`n_subspaces`	`int`	Number of panels, from the best subspace onwards. Defaults to all subspaces used for prediction.	`None`
`grid_resolution`	`int`	Number of grid points per axis for the decision regions of one- and two-dimensional subspaces.	`100`
`panel_size`	`float`	Width and height of each panel in inches.	`4.0`

Returns:

Type	Description
`Figure`	The figure; it is not shown or saved.

Raises:

Type	Description
`ImportError`	If matplotlib is not installed.

Subspace models¶

Every entry of estimators_ is one of these. It is not exported from the package: create a SubspaceKNNClassifier instead.

TieSharingKNeighborsClassifier ¶

TieSharingKNeighborsClassifier(
    n_neighbors: int = 5,
    *,
    weights: Literal["uniform", "distance"] = "uniform",
    metric: str = "minkowski",
    n_jobs: int | None = None,
)

k-nearest-neighbour classifier whose votes do not depend on the order of ties.

When several training points lie exactly at the distance of the k-th neighbour, a plain kNN keeps whichever of them its neighbour search returns first, and that order depends on the platform. Here, with d* the k-th smallest distance, every point strictly closer than d* gets a full vote and the points at exactly d* share the remaining votes equally, so the total is still n_neighbors. With weights="distance" each vote is further divided by the point's distance, and points at distance zero, if there are any, take all the weight.

Training points with identical coordinates are stored once, as a cell with a count per class, so repeated values cost nothing extra. For the Euclidean, Manhattan and Chebyshev metrics, distances are recomputed from the coordinates with operations that round identically on every platform, and ties are equalities of those distances; the neighbour search only proposes candidates. Other metrics use the search's own distances.

This is the model behind every subspace of SubspaceKNNClassifier.

Parameters:

Name	Type	Description	Default
`n_neighbors`	`int`	Number of votes per prediction.	`5`
`weights`	`('uniform', 'distance')`	Whether votes are equal or divided by the distance.	`"uniform"`
`metric`	`str`	Distance metric passed to the neighbour search.	`"minkowski"`
`n_jobs`	`int or None`	Parallelism of the neighbour search.	`None`

fit ¶

fit(
    X: ArrayLike, y: ArrayLike
) -> TieSharingKNeighborsClassifier

Store the training data as cells of identical points and build the search.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_samples, n_features)`	Training data.	required
`y`	`array-like of shape (n_samples,)`	Class labels.	required

Returns:

Type	Description
`self`	The fitted model.

predict_proba ¶

predict_proba(X: ArrayLike) -> NDArray[float64]

Return the class probabilities of each sample.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_queries, n_features)`	Samples to classify.	required

Returns:

Type	Description
`ndarray of shape (n_queries, n_classes)`	Vote shares aligned with `classes_`.

predict ¶

predict(X: ArrayLike) -> NDArray[Any]

Return the class with the largest vote share; ties go to the first class.

Parameters:

Name	Type	Description	Default
`X`	`array-like of shape (n_queries, n_features)`	Samples to classify.	required

Returns:

Type	Description
`ndarray of shape (n_queries,)`	Predicted class labels.

leave_one_out_proba ¶

leave_one_out_proba() -> NDArray[float64]

Return each training sample's probabilities with the sample left out.

This equals refitting without each sample and predicting it. Samples that share a cell and a class have the same answer, so it is computed once per cell and class.

Returns:

Type	Description
`ndarray of shape (n_samples, n_classes)`	Leave-one-out vote shares aligned with `classes_`.

API reference¶

SubspaceKNNClassifier ¶

fit ¶

predict_proba ¶

predict ¶

explain ¶

Explanation dataclass ¶

agreement ¶

to_records ¶

SubspaceVote dataclass ¶

plot_subspaces ¶

Subspace models¶

TieSharingKNeighborsClassifier ¶

fit ¶

predict_proba ¶

predict ¶

leave_one_out_proba ¶

Explanation `dataclass` ¶

SubspaceVote `dataclass` ¶