Facebook's Rosetta AI detects offensive memes

A figurine is seen in front of the Facebook logo in this illustration taken, March 20, 2018. REUTERS/Dado Ruvic - RC155C02C7D0

This week, Facebook announced the deployment of a large-scale machine learning system named Rosetta, which it's using to automatically and proactively identify "inappropriate or harmful content" in images on the social network. In other words, Facebook developed an AI that can tell if a meme is offensive.

The social media giant isn't intending to just use this information for memes, of course. In a blog post, Facebook describes how the algorithm could be used to detect text in a photo of a storefront, street sign, or restaurant menu.

With recent appearances on Capitol Hill, the news that 26% of Americans deleted the app off their phones, and the concerns about fake news that have been nipping at Facebook's heels since the election, using it to curb offensive content will surely be a priority.

Rosetta's intelligence starts with a two step process: detecting images that might potentially contain text and then recognising what the text in the image actually is. This model is not just for the English language, as Facebook says that it supports different languages and encodings including Arabic and Hindi, meaning the system will be able to read right to left as well.

Rosetta is already being used by teams at Facebook and Instagram to improve the quality of photo search, improve the accuracy of photos in the News Feed, and to identify hate speech.

Facebook has struggled in the past to adequately identify hate speech or misleading information and in documents seen by Motherboard, Facebook's own training material incorrectly said that an image of the 2010 earthquake in Jeiegu, China was a picture of a Myanmar genocide.

Using artificial intelligence to rate the severity of speech has already run into difficulties. It was recently discovered that Google's Perspective AI, which is used to detect harmful comments, can be tricked easily with typos, spaces between words, and adding unrelated words to the original sentence.

Suffice to say, with approximately 350 million photos being uploaded to the social network each day, Rosetta and Facebook are fighting an uphill battle.

This article originally appeared on PCMag.com.