HP's personal beauty advisor

A skilled beauty consultant is capable of determining the right shade of make-up for any skin tone, regardless of lighting effects. One of the main challenges for the research team, led by Nina Bhatti, was the problem of making a precise color judgment as to the correct shade of skin shown on a digital picture, and cleaning up all illumination and camera effects present on the photograph that might alter the true skin color. The solution was to include a small reference chart in every photo, calculated from a scan of the picture, which contains blocks of color with known color values. "We know what colors the chart is supposed to be, and so we can mathematically transform the image to correct it. When we do that, we also correct the skin tone." – explained Bhatti.

After the correct skin tone has been determined, the system chooses the matching foundation. A correct foundation would not only match the skin color, but also reduce the appearance of blemishes, like reddish skin and pimples. Therefore, the process is not a straightforward one-to-one choice of color. The researchers asked makeup artists to recommend the correct foundation colors for a large group of women, picked as reference models. Their recommendations serve as the basis for a statistical processing program, which compares the submitted face color to that of the models’. "We find the reference model whose color best matches yours and then with very high confidence we can say you wear the same foundation that she does," says Bhatti.

In this mobile color-matching method the researchers used both old and new HP technology. The system uses existing HP face-detection software and draws on experience in color and imaging science. Within this project, the team developed a statistical training and processing system that determines what skin pixels constitute what foundation color. Bhatti especially noted the contributions of researchers from HP’s Digital Printing and Imaging Lab, and its Mobile and Media Systems Lab.

The new technology may have applications for retailers, consumer goods companies, healthcare providers and others who are aiming for a high level of color or image coordination.