Facebook's 'Rosetta' AI Can Extract Text From A Billion Images Daily

Doris Richards
September 14, 2018

Facebook's approach to training data is a mixture of human-annotated public images with words and their locations, as well as synthetic generation of text on public languages.

Facebook split up the task of "extracting" text from an image into two separate matters, that of first detecting whether there is text at all in an image, and then to parsing what that word of phrase might be.

Facebook is setting itself up to use AI to read your memes.something like a billion memes a day on its platforms.

In a bid to curtail offensive memes that promote hate speech, Facebook is building a novel artificial intelligence (AI) system that uses machine learning to identify text in images and videos as well as transcribe it.

Facebook has already widely deployed an object-recognition system throughout its infrastructure called "Detectron", and having that in place clearly helped in this case. As great as the technology is, Facebook is seeking something better: the ability to pull text from images and understand the text.


The system inputs all this text - in various languages - into a recognition model that has been trained on classifiers to understand the context of the text and the image together. That's where the company's Rosetta machine learning system comes in. The social media platform is also planning to use Rosetta to determine which content should appear in a user's News Feed.

With more than 2 billion users worldwide, the sheer volume of languages that appear on Facebook and Instagram presents another significant challenge.

By its own admission, Facebook has been struggling to suppress the spread of inappropriate content - from hate speech and threats of violence to disinformation and "fake news" across its huge platform.

The company has begun removing accounts to "stop the spread of hate" and continues to do so if it discovers disinformation with the intention to encourage violence.

Regarding Facebook's content moderation technology, Guy Rosen, vice president, product management, recently conceded: "Our technology still doesn't work that well, so it needs to be checked by our review teams". For now, Facebook is employing Rosetta's help to make its photo searches more relevant, as well as automatically detect content that violates its hate-speech policy.

Other reports by Iphone Fresh

Discuss This Article

FOLLOW OUR NEWSPAPER