8 of 9 for Introduction to Image Processing
New perspectives: Homography Matrix and Texture Metrics
How do we change perspectives and identify textures in images?
In this blog post, we will go over homography matrices which are particularly useful when you want a different perspective of your image. We will also tackle texture metrics, which considers the spatial arrangement of intensity values.
Homography Matrices
Homography is simply the relationship between two images of a flat surface in space. It refers to a transformation matrix in a homogeneous coordinate space. A homography matrix creates a mapping between two planar projections of an image. Basically, it allows you to create a projection of the image from a different perspective.
For this demonstration, let us use this photo of my laptop taken while I was sat on the table. Say I want to have a top-view photo of my laptop, but I do not have access to that laptop currently for a second shot. We can apply homography matrices to get a new perspective of the laptop in this image.
We need to define more than three coplanar points on this image which we can easily define in the top view perspective.
lappy = imread('lappy.jpg')
imshow(lappy)
src = np.array([150, 1030,
2420, 280,
3860, 940,
1270, 2830
]).reshape((4, 2))
plt.scatter(src[:, 0], src[:, 1], color='red', s=20)
plt.show()
Now that we have carefully selected and marked four coplanar points from these image, we want to establish the destination plane. This destination plane is the top view projection of this laptop. A quick Google search reveals that the dimensions of this laptop is 314 mm x 219 mm. Hence, we are going to define our destination dimensions to be 3140 x 2190.
Do not mind the fact that I added an extra 0 (scaled by a factor of 10), I just did this to ensure that the output will still be high resolution since our source image is high resolution.
dst = np.array([0, 0,
3140, 0,
3140, 2190,
0, 2190,
]).reshape((4, 2))
plt.scatter(dst[:, 0], dst[:, 1], color='red', s=3)
plt.show()
We will utilize the transform class from scikit-image to generate the required homography matrix and apply in onto our original image based on the points that we have defined. In essence, it does the calculation of the matrix for us.
from skimage import transform
tform_lappy = transform.estimate_transform('projective', src, dst)
tf_lappy_img = transform.warp(lappy, tform_lappy.inverse, output_shape=(2190, 3140, 4))
fig, ax = plt.subplots()
ax.imshow(tf_lappy_img)
_ = ax.set_title('Projective transformation')
We now have a top view perspective of our laptop, all from our angled original image. Pretty awesome, to be honest!
Texture Metrics
Texture metrics offer a means of capturing the spatial arrangement and patterns of intensity values within an image. They serve as an alternative approach to characterize images, especially when color information lacks consistency or reliability due to factors like white balancing or color variations. Unlike focusing solely on absolute magnitudes, texture metrics are akin to spatial filters. They analyze the distribution of intensity values in an image with emphasis on their arrangement.
The computation of texture metrics typically involves the following steps:
- Defining a moving window, also referred to as a neighborhood or kernel, and traversing it across the image.
- Within each window, generating an occurrence histogram that represents the frequency of each gray level (intensity value) present.
- Calculating a probability array based on the occurrence histogram, which determines the likelihood of each gray level occurring within the window.
- Finally, deriving a specific texture metric based on the probability array. The choice of which metric to use depends on the particular application or analysis being performed.
By employing this method, a texture metric is assigned to the center of each moving window. To gain a comprehensive understanding of the texture distribution throughout the entire image, the moving window iterates over all positions, computing the texture metric at each step.
Texture metrics provide valuable insights into the spatial patterns present in an image. They find utility in a range of applications, including image analysis, computer vision, and texture classification. These metrics enable the quantification and comparison of textures, facilitating tasks such as image segmentation, object recognition, and anomaly detection.
Enough with the explanation, let us get on with the demonstration. Consider the following satellite image of Tacloban City after it was devastated by Typhoon Haiyan (Bagyong Yolanda) in 2013.
We can use texture metrics in segmenting the buildings and the damaged areas from this typhoon-struck satellite image.
We are going to compute for the entropy for each iteration of the value we set for the threshold.
The entropy can be calculated using the formula:
u_P_b represents the probability of one outcome (e.g. “damaged”) in the binary variable or distribution.
u_P_w represents the probability of the other outcome (e.g. "not damaged") in the binary variable or distribution.
sat_c = skimage.color.rgb2gray(sat)
e_sat_lst = []
fig, ax = plt.subplots(5, 4, figsize=(15, 15))
ax = ax.flatten()
for k, i in enumerate(np.linspace(0, 1, 20)):
sat_bw = sat_c > i
ax[k].imshow(sat_bw, cmap='gray')
ax[k].axis('off')
ax[k].set_title(f'Threshold: {i:.2f}', fontsize=15)
u_P_b = sum(sat_bw.flatten()) / len(sat_bw.flatten())
u_P_w = 1 - u_P_b
if u_P_b == 0 or u_P_b == 1:
e_sat = 0
else:
e_sat = - (u_P_b * np.log(u_P_b) + u_P_w * np.log(u_P_w))
e_sat_lst.append(e_sat)
plt.tight_layout()
plt.show()
Similar with the threshold determination criterion for image segmentation, a proper threshold value is still determined by heuristics.
To guide us in finding a reasonable threshold value, we can check which threshold value maximizes entropy.
e_sat_lst = np.array(e_sat_lst)
plt.plot(e_sat_lst, 'red')
plt.title('Entropy Values')
plt.ylabel('Threshold Value');
Now that we have estimated the threshold value to use, we can use this to mask the areas in the image to separate areas of low and high entropies.
threshold = np.linspace(0, 1, 20)[e_sat_lst.argmax()]
print(f'The threshold is: {threshold:.2f}')
sat = imread('sat.png')[:, :, :-1]
sat_bw = skimage.color.rgb2gray(sat) > threshold
satP_b = sum(sat_bw.flatten())/len(sat_bw.flatten())
satP_w = (len(sat_bw.flatten())-sum(sat_bw.flatten()))/len(sat_bw.flatten())
satE_dm = - (satP_b * np.log(satP_b) + satP_w * np.log(satP_w))
print(f'Entropy Value of {satE_dm:.2f}')
fig, ax = plt.subplots(1, 2, figsize=(10,10))
ax[0].set_title('Original Image')
ax[0].imshow(sat)
ax[1].set_title('Damaged Buildings')
ax[1].imshow(sat)
ax[1].imshow(sat_bw, cmap='Reds', alpha=0.3)
plt.show()The threshold is: 0.21Entropy Value of 0.68
In these images, damaged buildings are highlighted in red, while undamaged buildings remain unhighlighted. By utilizing entropy as a texture metric, we successfully segmented the damaged buildings from the undamaged ones. The higher entropy value in damaged areas indicates a greater degree of randomness or unpredictability in texture, likely resulting from the destruction or alteration of the object in the image.
Conclusion
Our exploration of homography matrices and texture metrics has highlighted their importance in image transformation and analysis. Homography matrices allow us to obtain new perspectives of an image, while texture metrics provide insights into spatial patterns and arrangements of intensity values. By applying these concepts practically, we demonstrated their usefulness in tasks such as viewpoint manipulation and image segmentation. Overall, these techniques enhance our understanding of images and enable improved analysis in computer vision and image processing.
On a personal note, homography matrices may be the most wow topic for me in image processing. I am aware how matrices are used for transformations, but only in the concept of vectors and planes. However, once we have covered and applied the topic in images for perspective manipulation, it somewhat amazed me even with my prior knowledge about transformations.
