Artificial intelligence: five minutes to understand Google’s new skin color detection system

The tech giant wants to commit to diversity. During its annual conference, Google announced last Wednesday the launch of a new classification of shades, the Monk Skin Tone (MST) Scale, which will allow, according to the company, a better representation of skin colors through its platforms. Explanations.

How does this ranking work?

Before the creation of this index, which is used to classify skin colors into several types, artificial intelligence engineers relied on the classification of Fitzpatrick, originally designed for dermatologists in 1975, which detailed only six shades, and favored shades clear. They used it in particular to classify skin types when it was necessary to program artificial intelligence systems learning to recognize humans on an image. But for the creators of the MST Scale, this index does not reveal dark skin well enough, which could create a bias.

Shaped by engineers and researchers from Google, and sociology professor Ellis Monk, practicing at Harvard, the MST Scale is made up of ten shades, defined by precise color codes, and having the best representation of the diversity of skin colors.

What will it be used for?

This sort of cue can be useful in computer vision, a field of artificial intelligence where computers are able to read and understand an image, much like the human eye. The systems behind Google Images could thus be smarter using this tool, and better recognize images of people with darker skin. This can for example improve the results of specific searches: “when you search for items related to makeup on Google Images, you will see an option to refine the results by skin tone”, develops Tulsee Doshi, product manager at Google, in a communicated.

In more general searches, the image results could be more diverse, which would be more representative of the population. In addition to Google Images, “we also use the MST classification to improve Google Photos”, Google’s online storage and editing platform, adds Tulsee Doshi. “We are launching a new series of Real Tone filters designed to work better on all skin tones. Clearly: the editing software would better highlight darker skin with new parameters.

Could this classification be used elsewhere than at Google?

Yes. The company has chosen to share in open source (in “open source”, in French), which will make it usable by other systems using artificial intelligence. This classification could thus improve a whole bunch of systems that poorly recognize people with darker skin, such as facial recognition programs.

But there are applications in which this classification could prove inadequate, such as “skin cancer risk assessment”, where artificial intelligence can perform diagnoses, confide the creators of the MST Scale on their dedicated site. In dermatology or in clinical contexts, “the MST would have to be validated” by studies, which has not yet been done.

Leave a Comment