Color Feature Extraction as a Method to Track Skin Color

When dealing with image retrieval, indexing, and classification, perhaps the easiest feature to use is the color due to it being visually easy to inspect. However, the large variety in its wide spectrum as well as the unavoidable changes in the illumination during experimentation give rise to certain difficulties that need to be addressed.

In this activity, the goal is to be able to track our face in a real-time video by using our skin as a feature. The additional challenge is that it has to be successful even if the illumination varies from time to time. Like I have previously discussed in the segmentation activity in our Applied Physics 186 course, the success of the segmentation is highly dependent on the region of interest (ROI) that is used in constructing the 2D-histogram below:

Figure 1. Normalized Chromaticity Index (NCC) color space (left) and the corresponding 2D histogram of my skin (right)

If we use a region of interest that only corresponds to a single illumination, most likely the program won't be able to successfully track the skin. To demonstrate this, I tried using two types of ROI. Let's try first using a single illumination as ROI:

(a) Single Illumination in ROI

Figure 2. One region of my face at a single illumination

Using Figure 2 as ROI, I arrived at the following result:

Video 1. Demonstration of face tracking using a single representative ROI under a single illumination

As one may easily observe, the bounding box switches from small to big, suggesting that only portion of my face was detected. At the end of the demonstration, it was able to track the whole face since the illumination is now similar to that of the ROI used.

(b) Varying illumination in ROI

Figure 3. Several regions of my skin at various illumination.

Using different representative ROI from different frames of the vid, I was able to fully track the whole of my face, unlike the result of the previous where only a single cheek is bounded by the bounding box.

As much as I want to post here a real-time face tracker, I'm still in the process of figuring out how to post a user-defined media in blogger. So in the mean time, please bear with the demos first. Ciao! :)