Computer Vision interview questions

What a strong answer covers

• Define convolution: applying a learnable filter (kernel) across an input to produce a feature map, highlighting local patterns.
• Explain pooling (e.g., max pooling): downsampling feature maps by taking the maximum/average value in a region, reducing spatial dimensions.
• Contribution to feature extraction: Convolution learns hierarchical features (edges, textures, object parts) automatically.
• Contribution to model efficiency: Pooling reduces parameter count, computation, and helps achieve translation invariance by making the model less sensitive to small shifts in input.

Where people lose the point

• ×Confusing convolution with simple matrix multiplication or not mentioning learnable filters.
• ×Failing to explain how pooling reduces dimensionality and contributes to translation invariance.
• ×Not connecting these operations to the automatic learning of features from raw pixels.

What a strong answer covers

• Gaussian blur: Smooths the image to remove noise, preventing false edges.
• Gradient calculation: Computes intensity gradients (magnitude and direction) using operators like Sobel.
• Non-maximum suppression: Thins edges by keeping only the local maxima of the gradient magnitude along the gradient direction.
• Hysteresis thresholding: Uses two thresholds (high and low) to identify strong edges and connect weak edges that are connected to strong ones, reducing spurious edges.
• Effectiveness: Produces thin, continuous edges with good localization and low error rate, making it robust to noise and variations.

Where people lose the point

• ×Omitting or incorrectly describing non-maximum suppression or hysteresis thresholding.
• ×Not explaining the role of Gaussian blur in noise reduction.
• ×Failing to mention the two thresholds in hysteresis and their purpose.

What a strong answer covers

• SIFT: Detects distinctive keypoints (e.g., corners, blobs) that are scale and rotation invariant, then describes their local neighborhood using gradient orientations.
• HOG: Describes the distribution of gradient orientations within localized regions of an image, typically used in a dense grid, effective for shape and appearance of objects.
• Key differences: SIFT focuses on sparse, distinctive keypoints; HOG is typically dense and describes object contours/shapes. SIFT is more robust to scale/rotation changes due to keypoint detection and orientation assignment.
• Use cases: SIFT for object recognition, image matching, panorama stitching where robust keypoint matching is needed. HOG for pedestrian detection, human pose estimation where overall shape is important.

Where people lose the point

• ×Incorrectly attributing scale/rotation invariance to HOG or overstating SIFT's invariance.
• ×Not clearly distinguishing between keypoint-based (sparse) vs. dense feature descriptions.
• ×Failing to provide concrete examples of when each would be preferred.

What a strong answer covers

• Two-stage detectors: First generate region proposals (potential object locations), then classify and refine these proposals. Examples: R-CNN, Fast R-CNN, Faster R-CNN.
• One-stage detectors: Directly predict bounding boxes and class probabilities in a single pass over the image. Examples: YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector).
• Trade-offs: Two-stage models generally offer higher accuracy and better localization, especially for small objects. One-stage models are significantly faster, suitable for real-time applications.
• Evolution: Started with R-CNN's region proposals, improved with Fast/Faster R-CNN's shared computations and RPN, then optimized for speed with YOLO/SSD's single-pass approach.

Where people lose the point

• ×Confusing the order of operations or components within two-stage detectors (e.g., RPN's role).
• ×Incorrectly classifying a model as one-stage or two-stage.
• ×Not discussing the core trade-off between speed and accuracy for each type.

What a strong answer covers

• Definition: Reusing a pre-trained model (often trained on a large, general dataset like ImageNet) as a starting point for a new, related task.
• Benefits: Reduces need for large datasets, significantly faster training, often leads to better performance (especially with limited data), leverages features learned from vast amounts of data.
• Implementation: Load a pre-trained CNN, freeze the early layers (feature extractors) and replace/retrain the final classification layers for the new task.
• Fine-tuning: Optionally, unfreeze some or all layers and train with a very small learning rate to adapt the pre-trained features to the specific dataset.

Where people lose the point

• ×Describing transfer learning as simply using a pre-trained model without modification.
• ×Not explaining the concept of freezing layers or why it's done.
• ×Failing to mention the benefit of reduced data requirements.

What a strong answer covers

• Semantic Segmentation: Classifies every pixel in an image into a predefined category (e.g., 'car', 'road', 'sky'). All pixels belonging to the same class are given the same label, regardless of individual instances.
• Instance Segmentation: Identifies and segments each individual object instance in an image. It assigns a class label and a unique instance ID to every pixel belonging to an object (e.g., 'car_1', 'car_2').
• Semantic Segmentation Architecture: U-Net is a common example, known for its encoder-decoder structure with skip connections to preserve spatial information.
• Instance Segmentation Architecture: Mask R-CNN is a prominent example, extending Faster R-CNN by adding a mask prediction branch in parallel with bounding box regression and classification.

Where people lose the point

• ×Confusing the output of semantic vs. instance segmentation (e.g., not distinguishing between 'all cars' vs. 'each car').
• ×Incorrectly assigning an architecture to the wrong segmentation type.
• ×Not explaining the 'instance' aspect clearly for instance segmentation.

What a strong answer covers

• Definition: Artificially increasing the size and diversity of a training dataset by applying various transformations to existing images.
• Cruciality: Helps prevent overfitting by exposing the model to more varied data, improves model generalization, and makes models more robust to variations in real-world data (e.g., lighting, pose, scale).
• Common techniques: Random rotations, flips (horizontal/vertical), shifts, scaling, cropping, brightness/contrast adjustments, adding noise, color jittering.
• Benefits: Reduces the need for collecting massive amounts of real-world data, especially for niche applications where data is scarce.

Where people lose the point

• ×Only listing techniques without explaining the 'why' (preventing overfitting, improving generalization).
• ×Confusing data augmentation with synthetic data generation (though related, augmentation uses existing data).
• ×Not emphasizing its importance for deep learning models specifically.

What a strong answer covers

• Intersection over Union (IoU): Measures the overlap between a predicted bounding box and a ground truth bounding box. Calculated as (Area of Intersection) / (Area of Union). Used to determine if a detection is considered a True Positive.
• True Positive (TP), False Positive (FP), False Negative (FN): Define these based on IoU threshold (e.g., IoU > 0.5 for TP).
• Precision and Recall: Precision = TP / (TP + FP), Recall = TP / (TP + FN). These are often plotted as a Precision-Recall curve.
• Mean Average Precision (mAP): A common metric for object detection that averages the Average Precision (AP) over all object classes. AP is the area under the Precision-Recall curve. mAP provides a single score reflecting both localization and classification accuracy across all classes.

Where people lose the point

• ×Incorrectly calculating IoU or misinterpreting its meaning.
• ×Confusing precision and recall or not explaining their trade-off.
• ×Failing to explain that mAP averages AP across classes and that AP is derived from the PR curve.

What a strong answer covers

• Role: Introduce non-linearity into the network, allowing it to learn complex patterns and map non-linear relationships in the data. Without them, a deep network would behave like a single linear layer.
• ReLU (Rectified Linear Unit): Output is `x` if `x > 0`, and `0` otherwise. Simple, computationally efficient, and helps mitigate the vanishing gradient problem.
• Sigmoid: Squashes input to a range between 0 and 1. Historically used but suffers from vanishing gradients for very large/small inputs and not zero-centered.
• Tanh: Squashes input to a range between -1 and 1. Zero-centered, but still suffers from vanishing gradients. ReLU is generally preferred in hidden layers of deep CNNs due to its advantages.

Where people lose the point

• ×Not emphasizing the introduction of non-linearity as the primary role.
• ×Incorrectly describing the output range or mathematical function of ReLU, sigmoid, or tanh.
• ×Failing to mention the vanishing gradient problem or ReLU's advantage in mitigating it.

What a strong answer covers

• Haar-like features: Simple rectangular features that capture differences in intensity between adjacent regions in an image, resembling edges, lines, or four-rectangle features.
• Integral Image: Used to rapidly calculate the sum of pixel intensities within any rectangular region, making feature computation extremely fast.
• Adaboost: A machine learning algorithm used to select a small number of highly discriminative Haar-like features from a very large set and combine them into a strong classifier.
• Cascade Classifier: A series of increasingly complex classifiers. Simple classifiers quickly reject most non-object regions, while more complex ones are applied only to promising regions, achieving high detection rates with low computational cost.

Where people lose the point

• ×Not explaining what a Haar-like feature actually represents (intensity differences).
• ×Failing to mention the integral image's role in speeding up feature calculation.
• ×Incorrectly describing the cascade structure or its purpose in efficiency.

What a strong answer covers

• Overfitting: Model learns the training data too well, including noise and specific patterns, leading to poor generalization on unseen data. High accuracy on training set, low on validation/test set.
• Underfitting: Model is too simple to capture the underlying patterns in the training data, resulting in poor performance on both training and unseen data. Low accuracy on both sets.
• Identification: Monitor training and validation loss/accuracy curves. Overfitting shows validation loss increasing while training loss decreases. Underfitting shows high loss on both.
• Mitigation: Overfitting - data augmentation, regularization (L1/L2, dropout), early stopping, more data. Underfitting - increase model complexity (more layers/neurons), longer training, better features, reduce regularization.

Where people lose the point

• ×Confusing the symptoms of overfitting and underfitting (e.g., high training accuracy for underfitting).
• ×Providing mitigation strategies for the wrong problem.
• ×Not explaining how to identify these issues by monitoring training/validation metrics.

What a strong answer covers

• Residual Connection: A shortcut connection that bypasses one or more layers and adds the input of the skipped layers directly to their output.
• Problem Solved: Vanishing/Exploding Gradients and Degradation Problem. In very deep networks, gradients can become extremely small (vanish) or large (explode) during backpropagation, making training difficult or impossible. Degradation refers to accuracy saturating and then rapidly degrading with increasing depth.
• Mechanism: Allows gradients to flow directly through the network, enabling the training of much deeper architectures without performance degradation.
• Benefit: Makes it easier for the network to learn identity mappings, ensuring that adding more layers will at least not hurt performance, and often significantly improve it by learning more complex features.

Where people lose the point

• ×Incorrectly describing how the skip connection works (e.g., concatenation instead of addition).
• ×Not clearly stating the 'degradation problem' or vanishing gradients as the core issue.
• ×Failing to explain how it enables training of deeper networks.

What a strong answer covers

• RGB (Red, Green, Blue): Additive color model, most common for display and capture. Each pixel has R, G, B intensity values. Intuitive but color information is coupled with intensity.
• Grayscale: Represents image intensity only, no color information. Reduces dimensionality, useful for algorithms sensitive to color noise or when color is not discriminative (e.g., edge detection).
• HSV (Hue, Saturation, Value): Separates color (Hue), color purity (Saturation), and brightness (Value/Intensity). More perceptually uniform than RGB.
• Use Cases: RGB for general display/storage. Grayscale for computational efficiency, some traditional CV tasks (e.g., Canny, SIFT). HSV for color-based segmentation, object tracking, or when illumination changes are a concern (Value channel can be ignored).

Where people lose the point

• ×Incorrectly describing the components of HSV or RGB.
• ×Not providing clear examples of when to use each color space.
• ×Failing to mention the benefit of HSV in separating color from intensity.

What a strong answer covers

• Definition: A collection of images, all originating from a single original image, but successively downsampled to create a multi-resolution representation.
• Purpose: To handle objects of varying scales in an image. Allows algorithms to detect features or objects at different resolutions without explicitly resizing the object itself.
• Construction: Typically involves two main operations: Gaussian pyramid (smoothing and downsampling) and Laplacian pyramid (storing the difference between levels, useful for reconstruction).
• Application: Scale-invariant feature detection (e.g., SIFT uses a difference-of-Gaussians pyramid), object detection (e.g., sliding window approaches at multiple scales), image blending, and image compression.

Where people lose the point

• ×Confusing image pyramids with simple resizing or not mentioning multi-resolution.
• ×Not explaining the 'why' (handling scale variations).
• ×Failing to provide a concrete application where pyramids are essential.

What a strong answer covers

• Process: Detect keypoints and compute descriptors (e.g., SIFT, ORB) for both images. Then, compare descriptors (e.g., using Euclidean distance or Hamming distance) to find potential matches.
• Ratio Test (e.g., Lowe's ratio test): A common method to filter out ambiguous matches by comparing the distance to the nearest neighbor with the distance to the second nearest neighbor.
• RANSAC (Random Sample Consensus): An iterative algorithm used to estimate parameters of a mathematical model from a set of observed data that contains outliers. In feature matching, it's used to find the geometric transformation (e.g., homography) that best aligns the inlier matches.
• Challenges: Outliers (incorrect matches), scale/rotation differences, illumination changes, occlusions, repetitive patterns. Addressed by robust descriptors, ratio tests, and RANSAC for outlier rejection.

Where people lose the point

• ×Only mentioning keypoint detection without descriptor comparison or filtering.
• ×Not explaining the purpose of the ratio test or RANSAC in filtering matches.
• ×Failing to identify common challenges beyond simple noise.