Facebook's 'Rosetta' AI can extract text from a billion images daily

People online tend to communicate not just with words, but also with images. For a platform like Facebook with over 2 billion monthly active users, that means a plethora of images gets posted every day, including memes. In order to include images with text in relevant photo search results, to give screen readers a way to read what's written on them and to make sure they don't contain hate speech and other words that violate the website's content policy, Facebook has created and deployed a large-scale machine learning system called "Rosetta."

Facebook needed an optical character recognition system that can regularly process huge volumes of content, so it had to conjure up its own technology. According to the social network, Rosetta extracts text from over a billion images and video frames in a wide variety of languages every day in real time.

In a new blog post, the company explained how Rosetta works: it starts by detecting rectangular regions in images that potentially contain text. It then uses a convolutional neural network to recognize and transcribe what's written in that region, even non-English words or non-Latin alphabets, such as Arabic and Hindi. To train the system, Facebook used a mixture of human- and machine-annotated public images.

Various teams within Facebook and Instagram are already using Rosetta to surface more content and to police their platforms. The company plans to keep on growing the number of languages it can understand and to make it better at extracting text from video frames.

Speaking of languages, Facebook has also added 24 new languages to its automatic translation services, including Serbian, Belarusian, Marathi, Sinhalese, Telugu, Nepali, Kannada, Urdu, Punjabi, Cambodian, Pashto, Mongolian, Zulu, Xhosa and Somali. Facebook admits that translations for those languages are at an early stage, so they will still have a lot of errors. It plans to keep on improving them, though, and to introduce more languages in the future.

Source: Facebook (1), (2)



via Engadget RSS Feed https://ift.tt/2MlV8kZ
RSS Feed

If New feed item from http://www.engadget.com/rss-full.xml, then send me


Unsubscribe from these notifications or sign in to manage your Email Applets.

IFTTT

Comments

Post a Comment

Popular posts from this blog

Evernote cuts staff as user growth stalls

Image
Note-taking app Evernote has fallen on hard times of late, culminating in its latest spate of job cuts impacting 15 percent of its workforce (54 employees). CEO Chris O' Neil -- an ex-Googler who took the reins in 2015 -- announced the firings at an all-hands meeting earlier today, reports TechCrunch . In a message on the Evernote blog , O' Neill admitted he'd set "incredibly aggressive goals" for the company in 2018. He continued: "Going forward, we are streamlining certain functions, like sales, so we can continue to speed up and scale others, like product development and engineering." The layoffs follow an exec exodus just weeks ago and the company's recent brand refresh (complete with a refined logo and wordmark). But critics are more concerned about its product, especially the free tier, which they claim lacks the perks to
Read more

The best air conditioner

Image
By Liam McCabe This post was done in partnership with Wirecutter . When readers choose to buy Wirecutter's independently chosen editorial picks, it may earn affiliate commissions that support its work. Read the full article here . After six summers of researching, testing, and recommending window air conditioners, we've learned that quiet and affordable ACs make most people the happiest—and we think the LG LW8016ER will fit the bill in most rooms. This 8,000 Btu unit cools as efficiently and effectively as any model with an equal Btu rating, and runs at a lower volume and deeper pitch than others at this price. Little extra features like a fresh-air vent, two-axis fan blades, and a removable drain plug help set it apart, too. The LG LW8016ER is a top choice for an office or den, and some people will find it quiet enough for a bedroom, too. If our main pic
Read more

We won't see a 'universal' vape oil cartridge anytime soon

Image
Pre-loaded cartridges of cannabis concentrate are currently among the most popular means of consumption, and for good reason. They're discreet to use and easy to handle, a far cry from the dark days of 2016 when we had to dribble hash oil or load wax into narrow-mouthed vape pens by hand. But, frustratingly, an ever increasing number of oil cartridge manufacturers employ one-off design standards so that their products won't work with those of their competitors, thereby locking customers into proprietary ecosystems. We've already seen this with nicotine vaporizers -- which has a seen a massive rise in "pod systems" in the last few years, each outfitted with a unique canister and battery built to be incompatible with those of their competition. Is it too late for the burgeoning cannabis industry to set a universal standard for their product designs?
Read more