Computer Vision Algorithms: Everything You Wanted To Know [2024]

Introduction

The word computer vision means the ability of a computer to see and perceive the surrounding. A lot of application holds for computer vision to cover — Object detection and recognition, self driving cars, facial recognition, ball tracking, photo tagging, and many more. Before diving in the technical jargons, first let’s discuss the entire computer vision pipeline. 

The entire pipeline is divided into 5 basic steps, each with a specific function. Firstly, the input is needed for the algorithm to process that can be in the form of an image or stream of image (image frames). The next step is pre-processing. In this step, functions are applied to the incoming image(s) so that the algorithm can better understand the image.

Some of the functions involve noise reduction, image scaling, dilation, and erosion, removing color spots, etc. The next step is selecting the area of interest or the region of interest. Under this lies the object detection and image segmentation algorithms. Further, we have feature extraction that means retrieving relevant information/features from the images that are necessary for accomplishing the end goal.

Get Machine Learning Certification from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

The final step is recognition or prediction, where we recognize objects in a given frame of images or predict the probability of the object in a given image frame.

Example

Let’s look at a real world application of the computer vision pipeline. Facial expression recognition is an application of computer vision that is used by a lot of research labs to get an idea of what effect a particular product has on its users. Again, we have input data to which we apply the pre-processing algorithms.

The next step involves detecting faces in a particular frame and cropping that part of the frame. Once this is achieved, facial landmarks are identified like mouth, eyes, nose, etc. — key features for emotion recognition.

In the end, a prediction model( trained model) classifies the images based on the features extracted in the intermediary steps.

Algorithms

Before I start mentioning the algorithms in computer vision, I want to stress the term ‘Frequency’. The frequency of an image is the rate of change of intensity. High-frequency images have large changes in intensity. A low-frequency image is relatively uniform in brightness or the intensity changes slowly.

On applying Fourier transform to an image we get a magnitude spectrum that yields the information of the image frequency. Concentrated point in the center of the frequency domain image means a lot of low frequency components are present in the image. High frequency components include — edges, corners, stripes, etc. We know that an image is a function of x and y f(x,y). To measure the intensity change, we just take the derivative of the function f(x,y). 

Object Detection Algorithms: From R-CNN to YOLO

Object detection is a fundamental task in computer vision algorithms that involves locating and classifying objects within an image or a video. Over the years, several object detection algorithms have been developed, each with its own strengths and limitations. In this article, we will explore two popular object detection algorithms: R-CNN and YOLO.

R-CNN (Region-based Convolutional Neural Network) was one of the pioneering algorithms in object detection. It introduced the concept of region proposals, which involves generating a set of potential object-bounding box regions within an image. R-CNN then applies a pre-trained CNN to each proposed region, extracting features and performing classification to determine the presence of objects. While R-CNN achieved good accuracy, its drawback was its slow inference speed due to its sequential processing of region proposals.

To address the speed issue, subsequent improvements were made, leading to the development of Fast R-CNN and Faster R-CNN. These algorithms introduced region of interest (ROI) pooling and shared convolutional layers, which significantly sped up the detection process. By sharing convolutional computation across regions, these algorithms achieved faster inference times while maintaining high accuracy.

Another breakthrough in object detection came with the introduction of You Only Look Once (YOLO). YOLO revolutionized the field by introducing a single-stage detection approach that achieved real-time object detection. YOLO divides the input image into a grid and predicts bounding boxes and class probabilities directly from each grid cell. This approach eliminates the need for region proposals and allows for simultaneous detection across the entire image. YOLO’s architecture, coupled with its efficient implementation, enables it to achieve impressive speed while maintaining competitive accuracy.

Image Classification Using Convolutional Neural Networks (CNNs)

Image classification is a crucial task in computer vision algorithms and applications that involves assigning labels or categories to input images. Convolutional Neural Networks (CNNs) have emerged as the go-to architecture for image classification, delivering state-of-the-art performance on various datasets.

CNNs excel at capturing hierarchical representations of images, learning both low-level and high-level features. The architecture comprises multiple layers, including convolutional, pooling, and fully connected layers. Convolutional layers use filters to detect local patterns and spatial relationships within the input image. Pooling layers downsample feature maps, reducing computational complexity while retaining important information. Fully connected layers enable the network to make predictions by learning complex relationships between extracted features and their corresponding classes.

Pre-training and fine-tuning CNNs using large-scale datasets, such as ImageNet, have become common practices. Pre-trained models, such as VGGNet, ResNet, and Inception, serve as powerful feature extractors, allowing for transfer learning. By leveraging pre-trained models, even with limited labeled data, one can achieve remarkable accuracy in image classification tasks.

CNNs have also benefited from advancements in regularization techniques, such as dropout and batch normalization, which mitigate overfitting and improve generalization. These techniques, along with data augmentation strategies like rotation, scaling, and flipping, help the network generalize well to unseen images.

Sober Filter 

The Sobel operator is used in image processing and computer vision for edge detection algorithms. The filter creates an image of emphasizing edges. It computes an approximation of the slope/gradient of the image intensity function. At each pixel in the image, the output of the Sobel operator is both the corresponding gradient vector and the norm of this vector.

The Sobel Operator convolves the image with a small integer-valued filter in the horizontal and vertical directions. This makes the operator inexpensive in terms of computation complexity. The Sx filter detects edges in the horizontal direction and Sy filter detects edges in the vertical direction. It is a high pass filter.

Applying Sx to the image

Applying Sy to the image

Read: Machine Learning Salary in India

Averaging Filter

Average filter is a normalized filter which is used to determine the brightness or darkness of an image. The average filter moves across the image pixel by pixel replacing each value in the pixel with the average value of the neighboring pixels, including itself.

The Average (or mean) filtering smoothens the images by reducing the amount of variation in the intensity between the neighboring pixels. 

Average filter, Image source

Gaussian Blur Filter

Gaussian blur filter is a low pass filter and it has the following functions:

1. Smooths an image
2. Blocks high frequency parts of an image
3. Preserves edges

Mathematically, by applying a Gaussian blur to an image we are basically convolving the image with a Gaussian function. 

In the above formula, x is the horizontal distance from the point of origin, y is the vertical distance from the origin point, and σ is the standard deviation of the Gaussian distribution. In two dimension, the formula represents a surface whose profiles are concentric circles with a Gaussian distribution from the point of origin. 

Gaussian Blur Filter, Image source

One thing to note here is the importance of choosing a right kernel size. It is important because if the kernel dimension is too large, small features present in the image may disappear and the image will look blurred. If it is too small, the noise in the image will not be eliminated.

Also Read: Types of AI Algorithm You Should Know

Canny Edge Detector

It is an algorithm that makes use of four filters to detect horizontal, vertical and diagonal edges in the blurred image. The algorithm performs the following functions.

1. It is a widely used an accurate edge detection algorithm
2. Filters out noise using Gaussian Blur
3. Finds the strength and direction of edges using Sobel filter
4. Applies non-max suppression to isolate the strongest edges and thin them to one pixel line
5. Uses hysteresis(double thresholding method) to isolate the best edges

Canny Edge detector on a steam engine photo, Image by Wikipedia

Haar Cascade

This is a machine learning based approach where a cascade function is trained to solve binary classification problems. The function is trained from a plethora of positive and negative images and is further used to detect objects in other images. It detects the following:

1. Edges
2. Lines
3. Rectangular patterns

To detect the above patterns, following features are used:

Convolutional layers

In this approach, the neural network learns the features of a group of images belonging to the same category. The learning takes place by updating the weights of the neurons using back propagation technique and gradient descent as an optimizer.

It is an iterative process that aims to decrease the error between the actual output and the ground truth. The convolution layers/blocks so obtained in the process act as feature layers that are used to distinguish a positive image from a negative one. Example of a convolution layer is given below.

Convolutional Neural Network, Image Source

Must Read: Types of Classification Algorithm in ML

The fully connected layers along with a SoftMax function at the end categorizes the incoming image into one of the categories it is trained on. The output score is a probabilistic score with a range between 0 to 1.

Conclusion

An overview of the most common algorithms used in Computer Vision has been covered in this blog along with a general pipeline. These algorithms form the basis of more complicated algorithms like SIFT, SURF, ORB, and many more.

If you’re interested to learn more about machine learning, check out IIIT-B & upGrad’s PG Diploma in Machine Learning & AI which is designed for working professionals and offers 450+ hours of rigorous training, 30+ case studies & assignments, IIIT-B Alumni status, 5+ practical hands-on capstone projects & job assistance with top firms.