9 of 9 for Introduction to Image Processing

Template Matching

How do we find objects from templates?

For this final blog post for the Introduction to Image Processing course, we will cover template matching and show it in action. It is a robust method utilized in image detection, recognition, and object tracking. The essence of template matching lies in locating a reference image, known as the template image, within a larger image referred to as the source image.

This algorithm follows an intuitive approach, systematically sliding the template image pixel by pixel and comparing it with different sections of the source image. Essentially, it convolutes the template across the image — similar to how kernels are used in convolutional neural networks. Through this process, a new image or matrix is created, where each pixel value represents the similarity between the template and the corresponding region in the source image. By analyzing this resulting image, we can identify peaks that signify the precise locations in the source image where the template image is present. It is important to note that the implementation of template matching can vary, encompassing different similarity measurements and efficient comparison methods across multiple areas.

Let us consider this fruity template 🍎🍋🍊🍓:

Fruiiiiits (Photo from Freepik)

Say we are interested in knowing how many complete apples there are in this image without having to manually count it.

Note that I emphasized “complete”. It is pretty obvious from the lefthand side of the image that there are apples which are cut. We cannot expect the algorithm to match these incomplete apples with the template we have below since this is just naïve matching and there is no deep learning model to learn that this incomplete apple is also similar with the apple from the template.

from skimage.color import rgb2gray

fruits = imread('fruits.jpg')
fruits_gray = rgb2gray(fruits)
template = fruits_gray[1350: 1550, 960: 1150]
imshow(template)
plt.show()

Apple template taken from the original image

Now that we have already defined our template, we can call the match_template function from scikit-image.feature.

from skimage.feature import match_template

result = match_template(fruits_gray, template)
imshow(result, cmap='coolwarm')
plt.show()

Dark red points in the image point areas where it matched template

This heatmap reveals hot areas where there is a match with the template. There are four distinct dark red points visible from the heatmap. To make the representation more obvious and intuitive, we can use the peak_local_max function also from scikit-image.feature instead.

from skimage.feature import peak_local_max

imshow(fruits_gray)
template_width, template_height = template.shape

rect_counter = 0
for i, (x, y) in enumerate(peak_local_max(result, threshold_abs=0.9)):
    rect = plt.Rectangle((y, x), template_height, template_width, color='r', 
                         fc='none')
    rect_counter += 1
    plt.gca().add_patch(rect);
    
plt.axis('off')
plt.show()
print(f'The count of apples is: {rect_counter}')

Amazing, right? Even without deep learning techniques, we were able to use computer vision (CV) to count the number of apples from the image. However, this does come with its inherent limitations. Since it looks at the pixel by pixel representation of the template, any augmented version of apples will not be detected by the algorithm. So for any real world applications involving real-life set-ups where there are no two objects alike, this algorithm will surely fail. We will need to use deep learning techniques for such use cases.

On the flipside, when we are simply dealing with templates in images with repeating down to the pixel, template matching will suffice. needless to mention, this is also less computationally expensive so it will be the more pragmatic choice between the two.

Conclusion

Template matching is a powerful technique used in image detection, recognition, and object tracking. Throughout this blog post, we explored the fundamental principles and practical applications of template matching. By systematically sliding a template image pixel by pixel and comparing it with different sections of a source image, we can identify the presence and locations of the template image within the larger image. This process generates a new image or matrix that represents the similarity between the template and corresponding regions in the source image.

We learned that template matching can be implemented using various similarity measurements and efficient comparison methods. It offers a straightforward and intuitive approach to identify patterns and objects in images. By using template matching, we were able to count the number of apples in an image without relying on deep learning techniques. However, it is essential to note that template matching has its limitations, particularly in scenarios involving augmented or modified objects. In such cases, deep learning techniques may be more suitable.

In our journey through template matching, we gained valuable insights into the world of computer vision and image processing. By understanding this technique, we can leverage its power in various applications, ranging from object detection to image analysis. Template matching provides a computationally efficient solution when dealing with pixel-perfect template repetitions. So, depending on the specific use case, template matching can be a pragmatic choice for pattern recognition and analysis tasks.

References

Colourful different fruits pattern. (n.d.). Freepik. photograph. Retrieved from https://www.freepik.com/free-vector/colourful-different-fruits-pattern_9002480.htm#query=fruit%20pattern&position=5&from_view=keyword&track=ais.