Regression¶
Predict each incomplete column from the other columns.
Regression-based imputers that predict each column from the others.
RegressionImputer
¶
Bases: BaseImputer
Impute missing values via linear regression.
Each incomplete column is regressed on all other columns, using the rows where it is observed. Gaps in the predictor columns are mean-filled first.
Initialize the imputer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
on_error
|
OnError
|
What to do if the model can't be fitted: |
None
|
Source code in src/imputation_methods/regression.py
impute
¶
Predict missing entries using other columns as features.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Dataframe with potential NaN values. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Imputed dataframe with missing values filled by regression |
DataFrame
|
predictions. |
Source code in src/imputation_methods/regression.py
StochasticRegressionImputer
¶
StochasticRegressionImputer(random_state: int | None = None)
Bases: BaseImputer
Impute missing values with regression plus random noise.
Like :class:RegressionImputer, but adds Gaussian noise with the standard
deviation of the regression residuals, which preserves the variance of the
imputed column instead of shrinking it towards the regression line.
Initialize the imputer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
random_state
|
int | None
|
Seed controlling the noise generation. |
None
|
Source code in src/imputation_methods/regression.py
impute
¶
Predict missing entries and add Gaussian noise.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Dataframe with potential NaN values. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Imputed dataframe with stochastic regression predictions. |
Source code in src/imputation_methods/regression.py
PMMImputer
¶
Bases: BaseImputer
Impute missing values using predictive mean matching (PMM).
For each column, a linear regression on the other columns scores every row.
Each missing entry is then replaced by the observed value of a donor drawn
at random from the n_neighbors rows whose predicted scores are closest, so
imputed values are always values that actually occur in the data.
References
Little, R. J. A. (1988). Missing-data adjustments in large surveys. Journal of Business & Economic Statistics, 6(3), 287-296.
Initialize the imputer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_neighbors
|
int
|
Number of donor candidates to consider. |
5
|
random_state
|
int | None
|
Seed for donor selection randomness. |
None
|
on_error
|
OnError
|
What to do if the model can't be fitted: |
None
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If n_neighbors is not a positive integer. |
Source code in src/imputation_methods/regression.py
impute
¶
Impute data using predictive mean matching.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Dataframe with missing values. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Imputed dataframe. Columns with no observed values are left as-is. |
Source code in src/imputation_methods/regression.py
BayesianRidgeImputer
¶
BayesianRidgeImputer(max_iter: int = 300, tol: float = 0.001, alpha_1: float = 1e-06, alpha_2: float = 1e-06, lambda_1: float = 1e-06, lambda_2: float = 1e-06)
Bases: BaseImputer
Bayesian ridge regression imputation for missing values.
Uses Bayesian ridge regression to predict missing values based on other features. Provides probabilistic estimates and handles multicollinearity well.
Algorithm Overview: Bayesian ridge regression treats the regression coefficients as random variables with Gaussian priors. It iteratively estimates both the coefficients and the precision (inverse variance) parameters using an Expectation-Maximization approach. This provides automatic relevance determination - features with low relevance are automatically down-weighted.
Key Differences from Standard Ridge Regression: - Standard ridge: Fixed regularization parameter λ (must be tuned) - Bayesian ridge: Learns optimal α (precision of weights) and λ (precision of noise) - Provides uncertainty estimates through posterior distributions - More robust to overfitting with automatic parameter adaptation
Hyperparameter Priors: - alpha ~ Gamma(alpha_1, alpha_2): Controls precision of weights (inverse variance) - lambda ~ Gamma(lambda_1, lambda_2): Controls precision of noise - Small values (1e-6) create weak priors, letting data dominate - Larger values create stronger priors, enforcing more regularization
When to Use: - Small to medium datasets with multivariate relationships - When feature relevance is unknown (automatic feature selection) - When uncertainty quantification is valuable - When you want to avoid manual hyperparameter tuning
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
max_iter
|
int
|
Maximum iterations for optimization. Default: 300 |
300
|
tol
|
float
|
Convergence tolerance. Default: 1e-3 |
0.001
|
alpha_1
|
float
|
Hyper-parameter for Gamma prior over alpha. Default: 1e-6 |
1e-06
|
alpha_2
|
float
|
Hyper-parameter for Gamma prior over alpha. Default: 1e-6 |
1e-06
|
lambda_1
|
float
|
Hyper-parameter for Gamma prior over lambda. Default: 1e-6 |
1e-06
|
lambda_2
|
float
|
Hyper-parameter for Gamma prior over lambda. Default: 1e-6 |
1e-06
|
Examples:
>>> import pandas as pd
>>> import numpy as np
>>> from imputation_methods import BayesianRidgeImputer
>>> df = pd.DataFrame({
... 'a': [1, 2, np.nan, 4, 5],
... 'b': [2, 4, 6, np.nan, 10]
... })
>>> imputer = BayesianRidgeImputer()
>>> imputed = imputer.impute(df)
References
Bayesian approach to ridge regression with automatic relevance determination. MacKay, D. J. C. (1992). Bayesian interpolation.
Initialize the Bayesian ridge imputer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
max_iter
|
int
|
Maximum iterations |
300
|
tol
|
float
|
Convergence tolerance |
0.001
|
alpha_1
|
float
|
Shape parameter of the Gamma prior over |
1e-06
|
alpha_2
|
float
|
Rate parameter of the Gamma prior over |
1e-06
|
lambda_1
|
float
|
Shape parameter of the Gamma prior over |
1e-06
|
lambda_2
|
float
|
Rate parameter of the Gamma prior over |
1e-06
|
Source code in src/imputation_methods/regression.py
impute
¶
Impute using Bayesian ridge regression.
Implements a column-by-column imputation strategy where each column with missing values is predicted using all other columns as features. The Bayesian ridge model automatically learns optimal regularization parameters during the iterative fitting process.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Dataframe with missing values. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Imputed dataframe. |
Source code in src/imputation_methods/regression.py
HuberImputer
¶
Bases: BaseImputer
Robust regression imputation using Huber loss.
Uses Huber regression which is robust to outliers in both features and target values.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
epsilon
|
float
|
Huber loss parameter (controls outlier threshold). Default: 1.35 |
1.35
|
max_iter
|
int
|
Maximum iterations. Default: 100 |
100
|
alpha
|
float
|
Regularization strength. Default: 0.0001 |
0.0001
|
Examples:
>>> import pandas as pd
>>> import numpy as np
>>> from imputation_methods import HuberImputer
>>> df = pd.DataFrame({
... 'a': [1, 2, np.nan, 100, 5], # 100 is outlier
... 'b': [2, 4, 6, 200, np.nan]
... })
>>> imputer = HuberImputer()
>>> imputed = imputer.impute(df)
References
Huber, P. J. (1964). Robust estimation of a location parameter.
Initialize the Huber imputer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
epsilon
|
float
|
Huber loss parameter |
1.35
|
max_iter
|
int
|
Maximum iterations |
100
|
alpha
|
float
|
Regularization strength |
0.0001
|
Source code in src/imputation_methods/regression.py
impute
¶
Impute using Huber regression.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Dataframe with missing values. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Imputed dataframe. |
Source code in src/imputation_methods/regression.py
RANSACImputer
¶
RANSACImputer(min_samples: int | None = None, residual_threshold: float | None = None, max_trials: int = 100, random_state: int | None = None, on_error: OnError = None)
Bases: BaseImputer
RANSAC robust regression for outlier-resistant imputation.
Uses RANSAC (Random Sample Consensus) to fit regression models that are robust to outliers in the training data.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
min_samples
|
int | None
|
Minimum samples for model. Default: None (auto) |
None
|
residual_threshold
|
float | None
|
Threshold for inliers. Default: None (auto) |
None
|
max_trials
|
int
|
Maximum RANSAC iterations. Default: 100 |
100
|
random_state
|
int | None
|
Random seed. Default: None |
None
|
Examples:
>>> import pandas as pd
>>> import numpy as np
>>> from imputation_methods import RANSACImputer
>>> df = pd.DataFrame({
... 'a': [1, 2, np.nan, 100, 5], # 100 is outlier
... 'b': [2, 4, 6, 200, np.nan]
... })
>>> imputer = RANSACImputer()
>>> imputed = imputer.impute(df)
References
Fischler & Bolles (1981). Random sample consensus.
Initialize the RANSAC imputer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
min_samples
|
int | None
|
Minimum samples for model |
None
|
residual_threshold
|
float | None
|
Inlier threshold |
None
|
max_trials
|
int
|
Maximum iterations |
100
|
random_state
|
int | None
|
Random seed |
None
|
on_error
|
OnError
|
What to do if the model can't be fitted: |
None
|
Source code in src/imputation_methods/regression.py
impute
¶
Impute using RANSAC regression.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Dataframe with missing values. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Imputed dataframe. |
Source code in src/imputation_methods/regression.py
GaussianProcessImputer
¶
GaussianProcessImputer(kernel: RBF | None = None, alpha: float = 1e-10, random_state: int | None = None, on_error: OnError = None)
Bases: BaseImputer
Impute missing values using Gaussian Process regression.
Initialize the imputer.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
kernel
|
RBF | None
|
Kernel used by the Gaussian process. Defaults to |
None
|
alpha
|
float
|
Added noise term to ensure numerical stability. |
1e-10
|
random_state
|
int | None
|
Random seed for reproducibility. |
None
|
on_error
|
OnError
|
What to do if the model can't be fitted: |
None
|
Source code in src/imputation_methods/regression.py
impute
¶
Predict missing entries with a Gaussian Process model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
df
|
DataFrame
|
Dataframe containing missing values. |
required |
Returns:
| Type | Description |
|---|---|
DataFrame
|
Dataframe where NaNs are replaced by GP predictions. |