Skip to main content

Salesforce created an algorithm that automatically summarizes text using machine learning

Salesforce created an algorithm that automatically summarizes text using machine learning


AI-powered tl;dr

Share this story

Image: Salesforce Research

This year, people are expected to spend more than half their day reading email, articles, or posts on social media, and it’s only going to get worse. To help solve this problem, researchers at Salesforce have developed an algorithm that uses machine learning to produce “surprisingly coherent and accurate” summaries according to MIT Technology Review.

Automatic summarization would be a particularly useful technology for Salesforce, which produces a variety of customer-service focused products. The company notes that the resulting summaries could be used by sales or customer service representatives to quickly digest emails and information, which would allow them to spend more time focused on their customers.

To that end, Salesforce is turning to machine learning to find ways to summarize longer blocks of texts, which it could eventually incorporate into its products. The company announced that it made two breakthroughs in natural language processing, introducing a new, “contextual word generation model,” and a “new way of training summarization models.” Together, the two advances allow researchers to automatically create summaries of longer texts that are accurate and readable. The company acquired a deep learning outfit MetaMind last year, which was behind the research.

Automatic text summarization works in two ways: extraction or abstraction

The researchers explain that automatic text summarization works in two ways: extraction or abstraction. With extraction, computer can draw from preexisting wording in a text, but it’s not very flexible. Abstraction allows the computer to introduce new words, but the system has to understand the original article enough to be able to introduce the right words.

This is where deep learning neural networks come into play. They process numerous examples of sentences and words to spit out new representations of each phrase, which allows the system to interpret texts and introduce its own words. The researchers let their model to look back at the text it’s working off of for additional context. It also looks back at earlier generated examples, to ensure that it’s not repeating itself.

Salesforce’s approach uses two teaching methods: teacher forced and reinforcement learning

The other breakthrough concerns how the researchers train the system to learn and improve itself. They used two approaches: teacher forcing and reinforcement learning. Reinforcement learning is a method that draws inspiration from how animals learn, and been used to teach Google’s DeepMind how to play video games. In this instance, the model is allowed to generate a sequence of words, and the result is then scored with an automated evaluation metric known as ROUGE (Recall-Oriented Understudy for Gisting Evaluation). The algorithm updates itself with higher scores, leading to better outcomes with future summaries. Teacher forcing is when the results are scored word by word off of an established reference, which provide “very decent results,” but which doesn’t allow for much flexibility.

Researchers found that “ROUGE-optimized RL helps improve recall...and word level learning supervision ensures good language flow, making the summary more coherent and readable.” Scored against this system, they found that their joint model scored higher than other approaches, and Richard Socher, Salesforce’s chief scientist, noted that he didn’t think that he’d ever seen “such a large improvement in any [natural-language-processing] task.”

The results are pretty astonishing: the researchers provided several examples, showing the original article, a human-generated summary, and a summary generated by their own model, and in each case, the summaries are considerably shorter than the original text, but contain the essentials in a readable form. Despite their advances, there’s still considerable work to be done in this field: MIT Technology Review spoke with Kristian Hammond, a professor at Northwestern University, who noted that the advance “shows the limits of relying purely on statistical machine learning,” but that it’s a step in the right direction.