scikit-learn interview questions

What a strong answer covers

• Explain `fit()`: learns parameters from training data (e.g., mean/std for scaler, coefficients for model).
• Explain `transform()`: applies learned parameters to data, typically for preprocessing (e.g., scaling features).
• Explain `predict()`: uses learned model parameters to make predictions on new, unseen data.
• Provide a concrete example for each, such as `StandardScaler.fit(X_train)`, `StandardScaler.transform(X_test)`, and `LogisticRegression.predict(X_new)`.

Where people lose the point

• ×Applying `fit()` to test data, leading to data leakage.
• ×Confusing `transform()` with `predict()` or vice-versa.
• ×Not understanding that `fit()` is for learning, `transform()`/`predict()` are for applying.

What a strong answer covers

• Explain that `train_test_split` prevents overfitting by evaluating the model on unseen data, providing an unbiased performance estimate.
• Describe the `test_size` parameter for specifying the proportion of the dataset to include in the test split.
• Explain the `random_state` parameter for reproducibility of the split.
• Mention the `stratify` parameter, crucial for maintaining class proportions in classification tasks, especially with imbalanced datasets.

Where people lose the point

• ×Not splitting data at all, leading to an overly optimistic performance estimate.
• ×Forgetting `random_state`, making results non-reproducible.
• ×Neglecting `stratify` for classification, especially with imbalanced classes, leading to skewed splits.

What a strong answer covers

• Explain that `StandardScaler` is used to standardize features by removing the mean and scaling to unit variance.
• State its primary use case: when features have different scales or units, which can negatively impact algorithms sensitive to feature magnitudes (e.g., SVMs, K-Nearest Neighbors, neural networks).
• Describe the mathematical operation: `z = (x - u) / s`, where `u` is the mean and `s` is the standard deviation.
• Mention that it assumes data is normally distributed, though it works reasonably well even if not perfectly normal.

Where people lose the point

• ×Applying `StandardScaler` to target variables (y) instead of features (X).
• ×Using `fit_transform` on the test set, causing data leakage.
• ×Not understanding that it transforms data to have a mean of 0 and a standard deviation of 1.

What a strong answer covers

• Explain that `Pipeline` chains multiple processing steps (transformers) and a final estimator into a single scikit-learn object.
• List benefits: reduces code duplication, prevents data leakage (by ensuring `fit` is only on training data), simplifies hyperparameter tuning across steps, and improves code readability and maintainability.
• Provide a concrete example: a pipeline that first scales numerical features, then applies one-hot encoding to categorical features, and finally trains a logistic regression model.
• Mention how `Pipeline` makes cross-validation and hyperparameter tuning more robust.

Where people lose the point

• ×Not understanding how `Pipeline` prevents data leakage.
• ×Thinking `Pipeline` is only for preprocessing, not for combining with an estimator.
• ×Failing to explain how `Pipeline` simplifies hyperparameter tuning (e.g., with `GridSearchCV`).

What a strong answer covers

• Explain `GridSearchCV`: exhaustive search over all specified hyperparameter combinations, guaranteeing the best combination within the defined grid.
• Explain `RandomizedSearchCV`: samples a fixed number of hyperparameter settings from specified distributions, not exhaustive.
• Contrast their computational cost: Grid Search is high (exponential with number of parameters), Random Search is lower (fixed number of iterations).
• Discuss when to choose each: Grid Search for small search spaces or when certainty of finding the global optimum within the grid is critical; Random Search for large search spaces, many hyperparameters, or when computational resources are limited, often finding a 'good enough' solution faster.

Where people lose the point

• ×Believing `RandomizedSearchCV` is always inferior because it's not exhaustive.
• ×Underestimating the computational cost of `GridSearchCV` for large search spaces.
• ×Not understanding that `RandomizedSearchCV` samples from distributions, not just discrete values.

What a strong answer covers

• Define k-fold cross-validation: the dataset is divided into 'k' equal-sized folds. The model is trained 'k' times, each time using 'k-1' folds for training and one fold for validation.
• Explain how performance metrics are averaged across the 'k' iterations to provide a more robust estimate of the model's generalization ability.
• List advantages: reduces variance in performance estimation compared to a single train-test split, makes better use of limited data, and provides a more reliable measure of model performance.
• Mention `StratifiedKFold` for classification tasks to preserve class proportions in each fold.

Where people lose the point

• ×Confusing cross-validation with hyperparameter tuning (though they are often used together).
• ×Not understanding that the model is trained multiple times.
• ×Failing to explain *why* it's better than a single split (reduced variance, better data utilization).

What a strong answer covers

• Define Accuracy: proportion of correctly classified instances. State its limitation with imbalanced datasets.
• Define Precision: proportion of positive identifications that were actually correct. Useful when minimizing false positives is critical (e.g., spam detection).
• Define Recall (Sensitivity): proportion of actual positives that were identified correctly. Useful when minimizing false negatives is critical (e.g., disease detection).
• Define F1-score: harmonic mean of precision and recall. Useful when seeking a balance between precision and recall, especially with imbalanced classes.
• Mention ROC AUC as another robust metric for imbalanced datasets, evaluating classifier performance across all classification thresholds.

Where people lose the point

• ×Relying solely on accuracy for imbalanced datasets.
• ×Confusing precision and recall definitions.
• ×Not understanding the trade-off between precision and recall.

What a strong answer covers

• Explain `ColumnTransformer`: it allows applying different transformers to different columns of a dataset simultaneously.
• Describe its mechanism: takes a list of (name, transformer, columns) tuples, where 'columns' can be indices or names.
• Highlight its benefit: handles mixed data types (numerical, categorical, text) within a single preprocessing step, integrating seamlessly into `Pipeline` objects.
• Provide a scenario: a dataset with numerical features needing scaling, categorical features needing one-hot encoding, and some columns to be dropped or passed through untouched.

Where people lose the point

• ×Thinking `ColumnTransformer` is only for numerical data.
• ×Not understanding how it integrates with `Pipeline`.
• ×Failing to explain its advantage over manually applying transformers to subsets of data.

What a strong answer covers

• Detecting Overfitting/Underfitting: Explain using `train_test_split` and cross-validation to compare training and validation/test scores. High training score, low test score indicates overfitting; low scores on both indicate underfitting.
• Mitigating Overfitting (scikit-learn tools): Discuss regularization (e.g., `C` parameter in `LogisticRegression`, `alpha` in `Ridge`/`Lasso`), reducing model complexity (e.g., `max_depth` in `DecisionTreeClassifier`), ensemble methods (`RandomForestClassifier`), and increasing training data.
• Mitigating Underfitting (scikit-learn tools): Discuss increasing model complexity (e.g., more features, higher polynomial degree), reducing regularization, and using more powerful models.
• Mention `learning_curve` and `validation_curve` from `sklearn.model_selection` as explicit tools for visualizing these phenomena.

Where people lose the point

• ×Confusing the symptoms of overfitting and underfitting.
• ×Suggesting solutions that are not directly scikit-learn related (e.g., collecting more data without mentioning data augmentation).
• ×Not linking specific scikit-learn parameters or models to mitigation strategies.

What a strong answer covers

• Explain that a custom transformer must inherit from `BaseEstimator` and `TransformerMixin` from `sklearn.base`.
• Describe the required methods: `fit(self, X, y=None)` (returns `self`) and `transform(self, X)` (returns transformed `X`).
• Provide a simple example: a transformer that calculates the ratio of two existing columns and adds it as a new feature.
• Emphasize that `fit` should learn any parameters needed for `transform`, and `transform` should apply the transformation without modifying the original data in place.

Where people lose the point

• ×Forgetting to inherit from `BaseEstimator` and `TransformerMixin`.
• ×Not returning `self` from the `fit` method.
• ×Modifying `X` in place within `transform` instead of returning a new array/DataFrame.

What a strong answer covers

• Explain the problem: imbalanced datasets can lead to models biased towards the majority class, performing poorly on the minority class.
• Strategy 1: Resampling techniques. Discuss `imblearn` (a scikit-learn-compatible library) for oversampling (e.g., SMOTE) or undersampling (e.g., RandomUnderSampler) to balance class distributions.
• Strategy 2: Using class weights. Explain how many scikit-learn classifiers (e.g., `LogisticRegression`, `SVC`, `RandomForestClassifier`) have a `class_weight` parameter (e.g., 'balanced') to penalize misclassifications of the minority class more heavily.
• Strategy 3 (Bonus): Choosing appropriate evaluation metrics (e.g., F1-score, precision, recall, ROC AUC) instead of accuracy, as discussed in a previous question.

Where people lose the point

• ×Only mentioning resampling without considering `class_weight`.
• ×Applying resampling *before* `train_test_split`, leading to data leakage.
• ×Not understanding *why* imbalanced data is a problem for standard models.

What a strong answer covers

• Explain that tree-based models (e.g., `DecisionTreeClassifier`, `RandomForestClassifier`, `GradientBoostingClassifier`) in scikit-learn often provide a `feature_importances_` attribute after fitting.
• Describe how `feature_importances_` is calculated: typically based on the reduction in impurity (e.g., Gini impurity or entropy) brought by each feature across all splits in the trees.
• Discuss limitations: can be biased towards high-cardinality features, doesn't account for feature interactions well, and can be unstable (especially for single decision trees).
• Mention alternative methods like permutation importance (available in `sklearn.inspection`) for a more robust and model-agnostic approach.

Where people lose the point

• ×Assuming `feature_importances_` is available for all scikit-learn models (e.g., linear models).
• ×Not understanding the underlying mechanism (impurity reduction).
• ×Failing to mention the limitations or alternative, more robust methods.

What a strong answer covers

• Explain model persistence: the process of saving a trained machine learning model to disk and loading it back later without retraining.
• Describe how to save: using Python's `pickle` module (`pickle.dump(model, file)`) or `joblib` (`joblib.dump(model, filename)`). Recommend `joblib` for large NumPy arrays, which scikit-learn models often contain.
• Describe how to load: using `pickle.load(file)` or `joblib.load(filename)`.
• Explain importance: avoids retraining, allows deployment of models to production, enables sharing of models, and facilitates reproducibility of results.

Where people lose the point

• ×Forgetting to open files in binary write/read mode (`'wb'`, `'rb'`).
• ×Not mentioning `joblib` as the preferred method for scikit-learn models.
• ×Failing to explain *why* persistence is important beyond just 'saving time'.

What a strong answer covers

• Define sparse data: data where most values are zero (e.g., one-hot encoded categorical features with many categories, text data represented by TF-IDF vectors).
• Explain scikit-learn's handling: many estimators and transformers (e.g., `LinearSVC`, `TfidfVectorizer`, `OneHotEncoder`) can directly accept and efficiently process sparse matrices (from `scipy.sparse`).
• Explain importance: reduces memory usage significantly for high-dimensional, sparse datasets, and speeds up computations by avoiding operations on zero values.
• Mention common sparse formats like CSR (Compressed Sparse Row) and CSC (Compressed Sparse Column) and their use cases.

Where people lose the point

• ×Not understanding what sparse data is.
• ×Thinking all scikit-learn estimators handle sparse data equally well (some convert to dense internally).
• ×Failing to explain the memory and computational benefits.

What a strong answer covers

• Ensemble methods: combine multiple base estimators to produce a better predictive model than any single estimator.
• Method 1: Bagging (e.g., `RandomForestClassifier`). Explain that it trains multiple base estimators (e.g., decision trees) independently on different bootstrap samples of the training data and averages their predictions (or takes a majority vote). Reduces variance.
• Method 2: Boosting (e.g., `GradientBoostingClassifier`, `AdaBoostClassifier`). Explain that it trains base estimators sequentially, where each new estimator tries to correct the errors of the previous ones. Focuses on difficult-to-classify instances. Reduces bias.
• Mention that both aim to improve robustness and accuracy over individual models.

Where people lose the point

• ×Confusing bagging and boosting mechanisms.
• ×Not mentioning specific scikit-learn implementations.
• ×Failing to explain *why* combining models helps (reducing variance/bias).