Pattern Recognition via Texture

When you google the word texture, it returns the following definition:

noun: 1. the feel, appearance, or consistency of a surface or a substance
"skin texture and tone"
synonyms: feel, touch; appearance, finish, surface, grain; quality, consistency

In reality, one could easily identify a texture of a surface by merely evaluating it through our senses such as touch and sight. In machine learning however, we are only limited by the appearance of the image -- that is, the gradient of graylevel values across an area. In this activity, we use texture as a feature in recognizing patterns in an image. There are already several techniques in analyzing textures of images. In particular, we will be using the local binary pattern technique,  which is essentially non-reversible, meaning one cannot use the final output (that is, the histogram) to get back to the original image. 

Now how do we obtain the LBP feature vector? We follow the following algorithm:

1.  Compare the graylevel value of a pixel to its eight nearest neighbors. If the neighbor is greater than the pixel itself, we make it equal to 1. For each pixel, there should be 8 binary numbers.
2. Convert these 8 binary numbers to decimal by multiplying each binary number to their corresponding power of 2 - [1,2,4,8,16,32,64,128]. Sum them all.
3. Do this for all pixels in the image. The end product should be another image set with pixel values ranging from 0 to 255.

This algorithm can be easier understood by following the following diagram:

Figure 1. LBP algorithm to be implemented

Upon writing the code, I tried it first to a random image I found in google. This is an image of a broken glass, which I find fascinating.

Figure 2. Image of the (left) shattered glass and its (right) corresponding LBP.

As one can see, you could still easily detect the location of the fractures, and the relatively smooth part of the mirror. In order to extract the features, one should obtain the histogram of the LBP values. In textures of the same class (fur, canvass, seeds and so on), the histogram should be relatively the same.

Shown below are just some of the texture classes I processed with their corresponding LBP histogram.

Figure 3. Class: Barley Rice

From the images shown above, one can easily see that the images are of similar textures, but different in colors. As expected, the histograms are relatively the same.

(a) Different canvas used

(b) Corresponding LBP images of canvas

(c) Histogram of the LBP for the canvas class

Figure 3. Sample images of the (a) original canvas used and the corresponding (b) lbp images. The histograms are shown in (c)

It is interesting to see that in Fig. 3, the histogram of the LBP for all canvas images are still relatively similar, regardless of the fact that the spatial patterns in the canvas have different directions/orientations.

I have also applied the algorithm to several other classes such as chips and seeds, among many others.

Refereces:

[1] Local Binary Patterns, Retrieved from  http://www.scholarpedia.org/article/Local_Binary_Patterns, dated September 15, 2015

[2] Images obtained from Outex - University of Oulu  from http://www.outex.oulu.fi/index.php?page=browse and retrieved September 15 2015

Moment Invariants as a tool in pattern recognition and understanding images

An image moment can be thought of as a weighted average of intensities in image pixels[1]. Just like moments defined in mathematics, it gives us an idea on the shape of a set of points, so that when image moments are known, the corresponding information about the orientation, centroid, and area is also obtained.

For a given digital image $f(x,y)$, the 2D raw moment is defined as:
$$ m_{pq} = \sum_{x=0}^{M-1} \sum_{y=0}^{N-1} x^p y^q f(x,y) $$
where p and q are real valued integers. Alternatively, we can also get the central moments by using the following relation:
$$\mu_{pq} = \sum_{x=0}^{M-1}\sum_{y=0}^{N-1} (x-\overline{x})^p (y-\overline{y})^q f(x,y) $$
where
$$\overline{x} = \frac{m_{10}}{m_{00}} $$ $$\overline{y} = \frac{m_{01}}{m_{00}}$$
From here, we can obtain the normalized central moments by solving for $\eta_{pq}$ defined as
$$\eta_{pq} = \frac{\mu_{pq}}{\mu_{00}^\gamma}$$
for $$\gamma = \frac{p+q}{2} +1 $$ where $$p+q = 2,3...$$

Using Hu's Uniqueness theorem which states that if $f(x,y)$ is a continuous piecewise function and has nonzero values only on the finite part of xy-plane, all moments of different orders exists. It also follows that the moments $m_{pq}$ are uniquely determined from $f(x,y)$, and conversely, we can obtain $f(x,y)$ from the information in the moments $m_{pq}$[2].

In Hu's paper published back in 1962, he enumerated a set of seven moments that are invariant to translation, scale change, mirroring (within a minus sign) and rotation. In this activity, the goal is to calculate for these moments using the following set of images:
1. Grayscale Image
2. Synthetic Edge Image (synthetically generated)
3. Edge Image of a real object
We generate each of these images, and their corresponding rescaled and rotated versions. In total, we should have 9 images. The moments of each of these images are to be computed and are then eventually compared to one another.

We start by showing the following set of images that I generated:

Figure 1. Synthetic images (a) original (b) rotated and (c) scaled down, which are generated for analysis of Hu's invariant moments.

Figure 2. Edges of a real image for (a) original (b) rotated and (scaled down). The image of Bingbong from the movie Inside Out is obtained from google images [3]

Figure 3. Gray images of a cup (a) original (b) rotated and (c) scaled. Image is taken from google images

After generating the images, I applied the program which computes for the corresponding normalized central moments (all seven of Hu's invariant moments, $\phi_1$, $\phi_2$, $\phi_3$, $\phi_4$, $\phi_5$, $\phi_6$, $\phi_7$)

I got the following results:

Table 1. Hu’s invariant moments for real image edge

Moments	Original	Rotated	Rescaled (0.5x)
ϕ1	2.8469	2.8465	2.8469
ϕ2	4.0568	4.0559	4.0568
ϕ3	4.9024	4.8953	4.9024
ϕ4	4.6394	4.6449	4.6394
ϕ5	9.3788	9.3821	9.3788
ϕ6	5.9112	5.929	5.9112
ϕ7	8.3275	8.0973	8.3275

Table 2. Hu’s invariant moments for grayscale image

Moments	Original	Rotated	Rescaled (0.5x)
ϕ1	-6.8124	-6.8124	-6.8125
ϕ2	-21.6559	-21.6559	-21.6559
ϕ3	-26.54	-26.54	-26.54
ϕ4	-26.021	-26.021	-26.0213
ϕ5	-54.504	-54.504	-54.5046
ϕ6	-37.0249	-37.0249	-37.0252
ϕ7	-27.5753	-25.2761	-27.5758

Table 3. Hu’s invariant moments for synthetic image

Moments	Original	Rotated	Rescaled (0.5x)
ϕ1	2.1266	2.1196	2.1266
ϕ2	-11.698	-11.7406	-11.698
ϕ3	-7.8382	-7.8438	-7.8382
ϕ4	-7.7756	-7.6766	-7.7756
ϕ5	-17.3893	-16.7467	-17.3892
ϕ6	-16.1036	-14.7318	-16.1036
ϕ7	-8.8509	-6.5823	-8.8509

The tables above show the corresponding Hu's moments that are seen to be invariant to scaled and rotated images. Notice that for real image edge, the moments are positive, while the rest are negative. Although almost all of the computed  Hu's moments in each of the transformations are observed to be invariant with minimal difference in their values, the 6th and 7th moments differ a little for the rotated transformation.

Another related independent basis set is presented by Flusser, Suk and Zitová [3]. In this work, they identified only 6 invariants and identified which of the 7 Hu's moments is actually dependent.

The following table shows the Flusser-Suk moment invariants for the synthetic edge image.

Table 4. Flusser -Suk moment invariants for synthetic edge image.

[1] Image moment. Wikipedia. Retrieved September 1 2015.

[2] M.-K. Hu. Visual pattern recognition by moment invariants, IRE Trans. on Information Theory, 8(2):179-187, 1962.

[3] J. Flusser, T. Suk, B. Zitová. Moment and Moment Invariants in Pattern Recognition. John Wiley and Sons, 2009.