Convolutions are one of the key features behind Convolutional Neural Networks. For the details of working of CNNs, refer to Introduction to Convolution Neural Network. Feature Learning Feature Engineering or Feature Extraction is the process of extracting useful patterns from input data that will help the prediction model to understand better the real nature of the problem. A good feature learning will present patterns in a way that increase significantly the accuracy and performance of the applied machine learning algorithms in a way that would be impossible or too expensive by the machine learning itself. Feature learning algorithms find the common patterns that are important to distinguish between the wanted classes and extract them automatically.
After this process, they are ready to be used in a classification or regression problem. Let us consider a popular image classification problem, classification of images of a face and a non-face object. In the early days of computer vision, scientists tried to solve the problem by hand coding the detection algorithms of possible features of a human face like shape, eyes, nose, lips etc. This approach usually gave poor results because a face may appear in so many varieties, that it was not possible to account for even a significant fraction of the features. Just a simple change in lighting or orientation can bring about change in an image such that the algorithms were no longer able to detect faces. In 1998, Yann Lecun introduced the concept of Convolutional Neural Networks which was capable of classifying images of handwritten characters with about 99% accuracy.
The great advantage of Convolutional Neural Networks is that they are uncommonly good at finding features in images that grow after each level, resulting in high-level features in the end. The final layers (can be one or more) use all these generated features for classification or regression. Convolution Convolution is an operation that is performed on an image to extract features from it applying a smaller tensor called a kernel like a sliding window over the image. Depending on the values in the convolutional kernel, we can pick up specific patterns from the image. In the following example, we will demonstrate detection of horizontal and vertical edges in an image using appropriate kernels.
Convolution is a mathematical operation that is used to combine two functions to form a third function that expresses how the shape of one is modified by the other. In the context of image processing and computer vision, convolutions are used to extract features from images.
In Python, one popular library for image processing and computer vision is OpenCV. OpenCV has the function cv2.filter2D() which can be used to apply a convolution to an image.
Python3
import numpy as np
import matplotlib.pyplot as plt
# let img1 be an image with no features
img1= np.array([np.array([200,200numpy as np1200,200numpy as np5
numpy as np6= np.array([np.array([200,200numpy as np1import3,import3numpy as np5
import7= np.array([np.array([200,import3numpy as np1200,import3numpy as np5
matplotlib.pyplot as plt8= np.array([np.array([# let img1 be an image with no features1,# let img1 be an image with no features1numpy as np1# let img1 be an image with no features5# let img1 be an image with no features1,# let img1 be an image with no features5# let img1 be an image with no features1numpy as np5
img11img12img13img14
img15= np.array([np.array([# let img1 be an image with no features1,# let img1 be an image with no features5# let img1 be an image with no features1numpy as np1# let img1 be an image with no features1,# let img1 be an image with no features5# let img1 be an image with no features1numpy as np5
img11=9np.array([np.array([0img14
np.array([np.array([2
np.array([np.array([3
np.array([np.array([4 np.array([np.array([5
np.array([np.array([6np.array([np.array([7 np.array([np.array([8np.array([np.array([92000
2001
2002
20032004img14
20062007img14
2009
,0
img11,2,3,4
,5,6
img11,2,9,4
,5numpy as np02
numpy as np03
numpy as np04
20032004img14
2006numpy as np09img14
2009
numpy as np12
img11,2,3numpy as np16
,5,6
img11,2,9numpy as np16
,5numpy as np02
numpy as np25
numpy as np26
20032004img14
2006numpy as np31img14
2009
numpy as np34
img11,2,3numpy as np38
,5,6
img11,2,9numpy as np38
,5numpy as np02
Output:
[ [ 2 2] [-2 -2] ] is a kernel for detecting horizontal edges [ [ 2 -2] [ 2 -2] ] is a kernel for detecting vertical edges
Horizontal edge confidence score: 0 Vertical edge confidence score: 0Horizontal edge confidence score: 800 Vertical edge confidence score: 0Horizontal edge confidence score: 0 Vertical edge confidence score: 800
Advantages and Disdvantages:
Advantages of using convolution in image processing and computer vision include:
- The ability to extract features from images: Convolutions can be used to identify patterns and features in an image, such as edges, corners, and textures. This can be useful for tasks such as object detection, image classification, and image segmentation.
- Translation invariance: Convolutions are translation invariant, which means that the same feature can be detected regardless of its position in the image. This is important for tasks such as object recognition, where the object may be in different positions in different images.
- Efficiency: Convolutions can be computed using fast algorithms such as the Fast Fourier Transform (FFT), which makes them efficient to compute even for large images.
- ability to learn features from data: In CNNs, the convolutional layers learn to extract features from the input data, which makes them useful in tasks such as image classification.
Disadvantages of using convolution in image processing and computer vision include:
- Limited ability to process large images: Convolutions are limited by the size of the kernel, which means that they are not well-suited for processing very large images.
- Limited ability to detect non-linear features: Convolutions are linear operations, which means that they are not well-suited for detecting non-linear features in images.
- High computational cost: Convolutional neural networks have a high computational cost which makes them less efficient to train and run.
- Overfitting: CNNs are prone to overfitting, especially when the training dataset is small. It is important to use techniques such as regularization to prevent overfitting.
References:
There are several books that provide in-depth coverage of convolution and its applications in image processing and computer vision. Some popular ones include: