Homework 0: Basic Image Manipulation

This homework focuses on fundamental image processing operations such as resizing, rotating, and basic transformations.

Image Resizing

Resizing an image involves changing its dimensions. One common method is nearest neighbor interpolation, which simply copies the value of the nearest pixel.

def resize_image_nearest_neighbor(source_img, new_height, new_width):
    """Resize an image using nearest neighbor interpolation."""
    original_height, original_width, channels = source_img.shape
    assert channels == 3

    # Create the output image
    resized_img = np.zeros((new_height, new_width, 3), dtype=source_img.dtype)

    # Calculate scaling factors
    row_scale = original_height / new_height
    col_scale = original_width / new_width

    # Populate the output image
    for i in range(new_height):
        for j in range(new_width):
            # Find the nearest pixel in the original image
            src_row = int(i * row_scale)
            src_col = int(j * col_scale)
            resized_img[i, j] = source_img[src_row, src_col]

    return resized_img

Image Rotation

Rotating an image involves applying a rotation matrix to each pixel. The rotation is typically performed around the image center.

def rotate_2d_point(point, angle_rad):
    """Rotate a 2D point by a given angle in radians."""
    x, y = point
    cos_theta = np.cos(angle_rad)
    sin_theta = np.sin(angle_rad)
    new_x = x * cos_theta - y * sin_theta
    new_y = x * sin_theta + y * cos_theta
    return np.array([new_x, new_y])

def rotate_image(source_img, angle_deg):
    """Rotate an image by a given angle in degrees."""
    original_height, original_width, channels = source_img.shape
    assert channels == 3

    # Convert angle to radians
    angle_rad = np.radians(angle_deg)

    # Create an output image with the same shape
    rotated_img = np.zeros_like(source_img)

    # Calculate the center of the image
    center_x = original_width / 2
    center_y = original_height / 2

    # Iterate over each pixel in the output image
    for i in range(original_height):
        for j in range(original_width):
            # Translate the pixel to the origin
            translated_point = np.array([j - center_x, i - center_y])
            # Rotate the point
            rotated_point = rotate_2d_point(translated_point, angle_rad)
            # Translate back
            new_x = rotated_point[0] + center_x
            new_y = rotated_point[1] + center_y

            # Check if the rotated point is within the image bounds
            if 0 <= new_x < original_width and 0 <= new_y < original_height:
                rotated_img[i, j] = source_img[int(new_y), int(new_x)]

    return rotated_img

Homework 1: Convolution and Filtering

This homework explores convolution, cross-correlation, and separable filters.

Convolution vs. Cross-Correlation

Convolution and cross-correlation are fundamental operations in signal processing and image enalysis. Convolution involves flipping the kernel before applying it, while cross-correlation does not.

Convolution: (f * g)[m, n] = Σi Σj f[i, j] * g[m - i, n - j]

Cross-Correlation: (f ⋆ g)[m, n] = Σi Σj f[i, j] * g[i - m, j - n]

Separable Filters

A separable filter is a filter that can be expressed as the outer product of two vectors. This allows for more efficient computation.

For example, a 2D Gaussian filter can be separated into two 1D Gaussian filters.

# Example of a separable filter (Gaussian blur)
def gaussian_blur_separable(image, sigma):
    """Apply Gaussian blur using separable filters."""
    # Create 1D Gaussian kernel
    size = int(6 * sigma + 1)
    kernel = np.linspace(-(size // 2), size // 2, size)
    kernel = np.exp(-0.5 * (kernel / sigma) ** 2)
    kernel = kernel / np.sum(kernel)

    # Apply horizontal blur
    blurred_image = np.zeros_like(image)
    for i in range(image.shape[0]):
        for j in range(image.shape[1]):
            for c in range(image.shape[2]):
                blurred_image[i, j, c] = np.sum(image[i, j - size // 2:j + size // 2 + 1, c] * kernel)

    # Apply vertical blur
    final_image = np.zeros_like(blurred_image)
    for i in range(blurred_image.shape[0]):
        for j in range(blurred_image.shape[1]):
            for c in range(blurred_image.shape[2]):
                final_image[i, j, c] = np.sum(blurred_image[i - size // 2:i + size // 2 + 1, j, c] * kernel)

    return final_image

Homework 2: Edge Detection

This homework focuses on edge detection techniques, including Canny edge detection and Hough transform.

Canny Edge Detection

Canny edge detection is a multi-stage algorithm that identifies edges in images.

1. Gaussian Smoothing: Apply a Gaussian filter to reduce noise.
2. Gradient Calculation: Compute the gradient magnitude and direction.
3. Non-Maximum Suppression: Thin the edges by suppressing non-maximum pixels.
4. Double Thresholding: Identify strong and weak edges.
5. Edge Tracking by Hysteresis: Connect weak edges to strong edges.

Hough Transform

The Hough transform is used to detect lines in an image.

1. Canny Edge Detection: First, apply Canny edge detecsion to the image.
2. Region of Interest (ROI):strong> Extract the region of interest.
3. Hough Transform: Map edge points to the Hough space and find lines.

Homework 3: Image Stitching

This homework involves creating a panorama by stitching multiple images to gether.

Feature Detection and Description

Feature detection and description are crucial for image stitching.

1. Corner Detection: Use methods like Harris corner detection to find key points.
2. Feature Description: Describe the key points using descriptors like SIFT or ORB.
3. Feature Matching: Match features between images using distance metrics like Euclidean distance.

Homography Estimation

Estimate the homography matrix to transform one image to align with another.

def estimate_homography(src_points, dst_points):
    """Estimate the homography matrix using RANSAC."""
    # Implement RANSAC to find the best homography
    # ...
    return homography_matrix

Image Blending

Blend the images together to create a seamless panorama.

def blend_images(img1, img2, mask1, mask2):
    """Blend two images using their masks."""
    # Implement image blending
    # ...
    return blended_image

Homework 6: Object Detection

This homework focuses on object detection techniques, including sliding window and deformable parts models.

Sliding Window Detection

Sliding window detection involves scanning an image with a window of a fixed size and classifying each window.

1. Image Pyramid: Create a pyramid of images at different scales.
2. Sliding Window: Slide a window across each scale and classify.
3. Non-Maximum Suppression: Remove overlapping detections.

Deformable Parts Model (DPM)

DPM is an extension of sliding window detection that models objects as a collection of parts.

1. Part Detection: Detect individual parts of the object.
2. Part Assembly: Assemble the parts to form the object.
3. Scoring: Score the assembled object based on part locations.

Homework 7: Optical Flow

This homework explores optical flow, which is the pattern of apparent motion of objects, surfaces, and edges in a visual scene caused by the relative motion between an observer and the scene.

Lucas-Kanade Method

The Lucas-Kanade method is a widely used optical flow algorithm.

1. Feature Detection: Detect features in the first frame.
2. Optical Flow Calculation: For each feature, calculate the optical flow using the Lucas-Kanade method.
3. Tracking: Track features across subsequent frames.

Iterative Lucas-Kanade

An iterative version of the Lucas-Kanade method can handle larger motions.

def iterative_lucas_kanade(image1, image2, window_size, max_iterations):
    """Iterative Lucas-Kanade optical flow."""
    # Initialize flow
    flow = np.zeros((image1.shape[0], image1.shape[1], 2))

    # Iterate
    for _ in range(max_iterations):
        # Compute spatial gradients
        # ...
        # Compute temporal difference
        # ...
        # Update flow
        # ...

    return flow

Homework 8: Camera Calibration

This homework involves camera calibration and understanding the camera projection matrix.

Camera Projection Matrix

The camera projection matrix relates 3D world coordinates to 2D image coordinates.

def camera_projection_matrix(focal_length, principal_point, rotation_matrix, translation_vector):
    """Construct the camera projection matrix."""
    # Construct the intrinsic matrix
    K = np.array([
        [focal_length, 0, principal_point[0]],
        [0, focal_length, principal_point[1]],
        [0, 0, 1]
    ])

    # Construct the extrinsic matrix
    R = rotation_matrix
    t = translation_vector
    T = np.column_stack((R, t))

    # Combine to form the projection matrix
    P = K @ T

    return P

VANISHING POINTS

Vanishing points are points in the image where parallel lines in the 3D world appear to converge.

1. Detect Lines: Detect lines in the image using edge detection and Hough transform.
2. Find Intersections: Find the intersections of parallel lines to determine vanishing points.

Optical Center and Focal Length

The optical center and focal length can be estimated from vanishing points.

1. Optical Center: The optical center is the intersection of the lines connecting the vanishing points.
2. Focal Length: The focal length can be calculated using the distances between the vanishing points and the optical center.