Machine Learning interview questions

What a strong answer covers

• Supervised learning trains on labeled examples (X mapped to a known y) to predict a target: classification (spam or not) or regression (house price).
• Unsupervised learning finds structure in unlabeled data: clustering (customer segments via k-means), dimensionality reduction (PCA), or density estimation.
• Concrete examples on both sides, plus a note that semi-supervised and self-supervised sit between them when labels are scarce or costly.
• A strong answer ties the choice to label availability and cost, not just definitions.

Where people lose the point

• ×Calling clustering a supervised method or confusing it with classification.
• ×Giving only definitions with no concrete example of each.
• ×Not recognizing that reinforcement learning is a separate paradigm, not a subtype of the two.

What a strong answer covers

• Bias is error from wrong assumptions (too simple a model, underfits); variance is sensitivity to the specific training sample (too complex, overfits).
• Expected test error decomposes into bias squared plus variance plus irreducible noise; you cannot drive both to zero at once.
• High bias shows as poor train and test performance; high variance shows as a large gap between strong train and weak test performance.
• Levers: model complexity, regularization strength, and training-set size all shift where you sit on the curve.

Where people lose the point

• ×Defining bias and variance backwards, or conflating variance with the statistical variance of a feature.
• ×Claiming more data always fixes high bias (it mainly reduces variance).
• ×Not connecting the tradeoff to observable symptoms like the train-test gap.

What a strong answer covers

• With rare positives (say 1% fraud), a model that predicts 'never fraud' scores 99% accuracy while catching zero fraud, so accuracy is misleading under class imbalance.
• Use precision (of flagged cases, how many are real fraud) and recall (of real fraud, how many were caught), summarized by F1 or by the area under the precision-recall curve.
• The right operating point depends on the cost of a false negative (missed fraud) versus a false positive (blocking a good customer); tune the decision threshold accordingly.
• Techniques to help: resampling or class weighting, and evaluating with a confusion matrix rather than a single number.

Where people lose the point

• ×Defending raw accuracy or not spotting that the base rate makes it meaningless.
• ×Confusing precision and recall, or treating them as interchangeable.
• ×Recommending ROC-AUC as the fix without noting PR curves are more informative when positives are very rare.

What a strong answer covers

• Regularization adds a penalty on model complexity (typically weight magnitude) to the loss, trading a bit of training fit for lower variance and better generalization.
• L2 (ridge) penalizes squared weights, shrinking them smoothly toward zero but rarely exactly to zero; it handles correlated features gracefully.
• L1 (lasso) penalizes absolute weights, which drives some coefficients exactly to zero and so performs feature selection, yielding sparse models.
• Elastic net blends both; the strength hyperparameter (lambda or alpha) is tuned via cross-validation, and features should be scaled first.

Where people lose the point

• ×Swapping which of L1 and L2 produces sparsity.
• ×Forgetting that features must be standardized before penalizing weights.
• ×Describing regularization as something that always improves training accuracy (it deliberately does not).

What a strong answer covers

• Leakage is when information unavailable at prediction time slips into training, producing validation scores that collapse in production.
• Common forms: fitting a scaler, imputer, or target encoder on the full dataset before splitting; features that are proxies for the label; and time leakage where future rows inform past predictions.
• Symptoms: suspiciously high offline metrics, a single feature dominating importance, or a large train-to-production performance drop.
• Prevention: split first, fit all preprocessing inside a pipeline on the training fold only, use time-based splits for temporal data, and audit top features for future information.

Where people lose the point

• ×Fitting preprocessing on all data and then splitting (the classic leak).
• ×Using random k-fold on time-series data instead of a forward-chaining split.
• ×Only defining leakage abstractly with no concrete example or detection signal.

What a strong answer covers

• Gradient-boosted trees (XGBoost, LightGBM) excel on heterogeneous tabular data: they capture nonlinearities and interactions, handle mixed feature scales, and are robust to outliers with little preprocessing.
• Prefer linear or logistic models when interpretability, calibrated probabilities, or a strong linear signal matter, or when data is small and high-dimensional; prefer deep nets mainly for unstructured inputs like text, images, or audio.
• Validate with cross-validation (time-based if temporal), a held-out test set, and comparison against a simple baseline; check calibration and per-segment performance, not just aggregate score.
• Weigh operational factors: training and inference cost, latency, retraining cadence, and how easily each model can be explained to stakeholders and regulators.

Where people lose the point

• ×Reaching for deep learning by default on modest tabular data where boosted trees usually win.
• ×Comparing models without a baseline or a consistent, leakage-free validation setup.
• ×Ignoring non-accuracy factors like latency, interpretability, and maintenance cost.