Transformers in Scikit-Learn

Transformers serve as the foundational components for feature engineering pipelines. They standardize, normalize, or encode raw data into formats suitable for model training. The core interface revolves around three primary methods:

• fit(): Computes internal parameters (e.g., mean and standard deviation for scaling) from the provided dataset.
• transform(): Applies the computed paramteers to modify the dataset.
• fit_transform(): Combines both steps efficiently, typically used on training data.

Standardization follows the formula: $z = \frac{x - \mu}{\sigma}$

from sklearn.preprocessing import StandardScaler
import numpy as np

# Training dataset
train_data = np.array([[2.0, 4.0, 6.0],
                       [8.0, 10.0, 12.0]])

# Initialize and apply fit_transform on training data
scaler = StandardScaler()
scaled_train = scaler.fit_transform(train_data)
print(scaled_train)
# Output:
# [[-1. -1. -1.]
#  [ 1.  1.  1.]]

# Separate fit and transform calls yield identical results
scaler_alt = StandardScaler()
scaler_alt.fit(train_data)
scaled_train_alt = scaler_alt.transform(train_data)
print(np.array_equal(scaled_train, scaled_train_alt))  # True

The distinction becomes critical when processing unseen data. Applying transform() to new samples uses the statistics learned from the training set, whereas fit_transform() recalculates statistics based on the new data, which leads to data leakage in production pipelines.

# Unseen test dataset
test_data = np.array([[14.0, 16.0, 18.0],
                      [20.0, 22.0, 24.0]])

# Correct approach: use training statistics
scaled_test = scaler.transform(test_data)
print(scaled_test)
# Output:
# [[3. 3. 3.]
#  [5. 5. 5.]]

# Incorrect approach for test data: recalculates mean/std
leaked_scaled = scaler.fit_transform(test_data)
print(leaked_scaled)
# Output:
# [[-1. -1. -1.]
#  [ 1.  1.  1.]]

Estimators: The Core Algorithm API

Estimators represent the actual machine learning alogrithms in scikit-learn. Every model, whether supervised or unsupervised, implements a consistent estimator interface.

Common Estimator Categories:

• Classification: sklearn.neighbors.KNeighborsClassifier, sklearn.naive_bayes.GaussianNB, sklearn.linear_model.LogisticRegression, sklearn.ensemble.RandomForestClassifier
• Regression: sklearn.linear_model.LinearRegression, sklearn.linear_model.Ridge, sklearn.tree.DecisionTreeRegressor
• Clustering (Unsupervised): sklearn.cluster.KMeans, sklearn.cluster.DBSCAN

Standard Estimator Workflow

1. Instantiation: Configure hyperparameters during object creation.
2. Training: Call fit(X_train, y_train) to learn patterns. The model is ready immediately after this call.
3. Evaluation & Inference: Generate predictions and measure performance against ground truth.

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression

# Generate synthetic dataset
X, y = make_classification(n_samples=200, n_features=4, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# 1. Instantiate estimator with hyperparameters
clf = LogisticRegression(max_iter=200, solver='lbfgs')

# 2. Train the model
clf.fit(X_train, y_train)

# 3. Inference and Evaluation
predictions = clf.predict(X_test)

# Manual accuracy calculation
manual_accuracy = sum(predictions == y_test) / len(y_test)
print(f"Manual Accuracy: {manual_accuracy:.4f}")

# Built-in scoring method
built_in_accuracy = clf.score(X_test, y_test)
print(f"Built-in Score: {built_in_accuracy:.4f}")

The predict() method outputs class labels or continuous values depending on the algorithm type, while score() automatically computes the default metric (accuracy for classifiers, R² for regressors) by comparing predictions against the provided test labels.