5 of 9 for Introduction to Image Processing

Thresholding and Otsu’s Method for Image Segmentation

How do we segment certain parts of the image?

For this one, we shall try segmenting images using vanilla thresholding and a more advanced technique called the Otsu’s method. We will also explore segmentation across different color channels.

Let us consider the following cute red panda (Ailurus fulgens). Perhaps we want to try segmenting the image such that only the trees and the red panda (foreground) are included.

Red panda at the Everland Resort in Korea

Vanilla thresholding

We can explore different threshold values and see which one gives us the best separation between the foreground and the background.

from skimage.io import imread, imshow
from skimage.color import rgb2gray

fig, ax = plt.subplots(3, 3, figsize=(16, 9))
ax = ax.flatten()
ths = [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]
for i, th in enumerate(ths):
    red_panda_bw = red_panda_gray < th
    ax[i].set_title(f'Threshold: {th}')
    ax[i].imshow(red_panda_bw, cmap='gray')
    ax[i].axis('off')
fig.show()

Choosing the best threshold remains subjective

Subjectively speaking, threshold value around 0.4 seems to provide the best distinction between the foreground and the background. This already shows the weakness of vanilla thresholding. Since it differs from case-to-case, there is no one-size-fits-all solution for every photo out there.

A quick check on the histogram of the intensity values for all pixels in the image reveals that there are two distinguishable peaks.

freq, bins = histogram(red_panda_gray)
plt.step(bins, freq*1.0/freq.sum())
plt.xlabel('intensity value')
plt.ylabel('fraction of pixels')

plt.axvline(x=0.4, color='red', linestyle='--')

plt.show()

Two peaks correspond to the different foreground and the background intensities

The first peak at the left represent the lower intensity foreground which comprise most of the area in the image. The high intensity background pixels comprise the window in the image. Applying a threshold of 0.4 seems to work from our earlier observation since this threshold value separates the background which has high intensities from the foreground with relatively lower intensities.

Otsu’s Method

from skimage.filters import threshold_otsu

rp_thresh = threshold_otsu(red_panda_gray)
red_panda_otsu  = red_panda_gray < rp_thresh
imshow(red_panda_otsu)
plt.axis('off')
plt.show()

Otsu’s method provides a more automated way to determine the appropriate threshold value

Otsu’s method is a technique used to automatically determine an optimal threshold for image segmentation. It works by calculating the variance within two classes of pixels: those that are considered foreground and those that are considered background. The method iteratively computes the threshold that maximizes the weighted sum of variances between these two classes. Essentially, Otsu’s method finds the threshold that maximizes the separation between object and background pixels based on their intensity values.

Otsu’s method assumes only foreground and background are the classes

However, one obvious limitation of the Otsu’s method is that it assumes that there are only two classes in the image: the foreground and the background. This assumption is not always useful in all cases especially those where segmentation needs to be performed within the foreground. For example, if we want to segment the color of the red panda’s fur from the trees in the foreground, Otsu’s method will not work.

Hence, the segmentation may have to be performed on the color channels of the image.

img = red_panda.copy()
r = img[:,:,0]
g = img[:,:,1]
b = img[:,:,2]

threshold = 70
binary_mask = (r > threshold) & (g < threshold) & (b < threshold)
img[~binary_mask] = 0

imshow(img)
plt.axis('off')
plt.show()

Segmenting the red panda using the RGB color space does not seem to work well

It might be tricky to segment by each color channel considering that much of the colors of the objects in the image have significant red channel values.

Most of the objects in the image have significant values across all RGB channels

Hence, it might be possible to instead segment on the HSV channels.

Segmentation across HSV channels reveal better separation between objects in the image

red_panda_hue = red_panda_hsv[:, :, 0]
red_panda_sat = red_panda_hsv[:, :, 1]
red_panda_val = red_panda_hsv[:, :, 2]

hue_threshold_low = 0.02
hue_threshold_high = 0.06
saturation_threshold_low = 0.6
saturation_threshold_high = 1

binarized_hsv = ((red_panda_hue > hue_threshold_low) & (red_panda_hue < hue_threshold_high) &
                 (red_panda_sat > saturation_threshold_low) & (red_panda_sat < saturation_threshold_high))

r = red_panda[:,:,0] * binarized_hsv
g = red_panda[:,:,1] * binarized_hsv
b = red_panda[:,:,2] * binarized_hsv
plt.imshow(np.dstack([r,g,b]))
plt.axis('off')
plt.show()

Red fur of the red panda properly segmented through the HSV channels

While segmentation through the HSV channels did not exactly segment the red panda by itself, it was able to properly segment the red fur of the red panda. The major reason why segmentation via the HSV channels was more successful than through the RGB channels is due to the greater hue separation between objects in the image.

Conclusion

In class, we learned about image segmentation techniques such as thresholding and Otsu’s method. Thresholding involves selecting a threshold value to separate foreground and background pixels, while Otsu’s method automatically determines the optimal threshold by maximizing the variance between these classes. We also explored segmentation of images using RGB and HSV color spaces. While segmenting based on individual RGB channels was not extremely effective, segmenting based on HSV channels provided better separation of the objects in the image. These techniques highlight the importance of understanding image characteristics and choosing suitable segmentation approaches for different scenarios.