Neural Summarization

Notes:

Neural summarization is a method for automatically generating summaries of text documents using artificial neural networks. Neural networks are computational models that are inspired by the structure and function of the human brain, and they are commonly used for machine learning and pattern recognition tasks.

In the context of neural summarization, a neural network is trained to analyze a text document and generate a summary of the document that is shorter than the original text, but that contains the most important information and ideas from the document. The neural network typically uses a sequence-to-sequence model, where the input sequence is the original text document, and the output sequence is the summary of the document.

There are several key advantages to using neural summarization, compared to other summarization methods:

• Accuracy and quality: Neural summarization can produce high-quality summaries that accurately reflect the content and meaning of the original text. This is because neural networks are able to learn and adapt to the structure and patterns of the text, and they can generate summaries that are coherent, fluent, and relevant.
• Efficiency and speed: Neural summarization is typically faster and more efficient than other summarization methods, because the neural network can process the text in parallel and generate the summary in a single pass. This can be useful in applications where summaries need to be generated quickly, such as in news aggregation or content curation.
• Flexibility and adaptability: Neural summarization is flexible and adaptable, because the neural network can be trained and fine-tuned to different types of text, styles, and domains. This means that the same neural summarization system can be used to generate summaries for different languages, genres, or topics, without the need to design and implement separate summarization algorithms for each case.

In summary, neural summarization is a method for automatically generating summaries of text documents using artificial neural networks. Neural summarization can produce high-quality summaries that accurately reflect the content and meaning of the original text, and it is fast, efficient, and flexible.

• Document summarization
• Long text summarization
• Multi-document summarization
• Neural abstractive summarization
• Neural document model
• Neural extractive summarization
• Neural sentence summarization
• Neural summarisation
• Sentence summarization
• Short text summarization
• Speech summarization
• Text summarization
• Video summarization

Resources:

• pytorch.org .. tensors and dynamic neural networks in python

Wikipedia:

• Document modelling
• Neural gas
• Random indexing
• Text simplification

References:

• Attention and Memory in Deep Learning and NLP (2016)
• SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents (2016)

See also:

Automatic Summarization

---

Neural summarization by extracting sentences and words
J Cheng, M Lapata – arXiv preprint arXiv:1603.07252, 2016 – arxiv.org
Abstract: Traditional approaches to extractive summarization rely heavily on human-engineered features. In this work we propose a data-driven approach based on neural networks and continuous sentence features. We develop a general framework for single-

A neural attention model for abstractive sentence summarization
AM Rush, S Chopra, J Weston – arXiv preprint arXiv:1509.00685, 2015 – arxiv.org
Abstract: Summarization based on text extraction is inherently limited, but generation-style abstractive methods have proven challenging to build. In this work, we propose a fully data-driven approach to abstractive sentence summarization. Our method utilizes a local attention-

Abstractive sentence summarization with attentive recurrent neural networks
S Chopra, M Auli, AM Rush – Proceedings of the 2016 Conference of the …, 2016 – aclweb.org
Abstract Abstractive Sentence Summarization generates a shorter version of a given sentence while attempting to preserve its meaning. We introduce a conditional recurrent neural network (RNN) which generates a summary of an input sentence. The conditioning is

Ranking with Recursive Neural Networks and Its Application to Multi-Document Summarization.
Z Cao, F Wei, L Dong, S Li, M Zhou – AAAI, 2015 – aaai.org
Abstract We develop a Ranking framework upon Recursive Neural Networks (R2N2) to rank sentences for multi-document summarization. It formulates the sentence ranking task as a hierarchical regression process, which simultaneously measures the salience of a sentence

Text summarization techniques: SVM versus neural networks
K Kianmehr, S Gao, J Attari, MM Rahman… – Proceedings of the 11th …, 2009 – dl.acm.org
Abstract Automated text summarization is important to for humans to better manage the massive information explosion. Several machine learning approaches could be successfully used to handle the problem. This paper reports the results of our study to compare the

Extractive broadcast news summarization leveraging recurrent neural network language modeling techniques
KY Chen, SH Liu, B Chen, HM Wang, EE Jan… – IEEE/ACM Transactions …, 2015 – dl.acm.org
Abstract Extractive text or speech summarization manages to select a set of salient sentences from an original document and concatenate them to form a summary, enabling users to better browse through and understand the content of the document. A recent stream

Attsum: Joint learning of focusing and summarization with neural attention
Z Cao, W Li, S Li, F Wei, Y Li – arXiv preprint arXiv:1604.00125, 2016 – arxiv.org
Abstract: Query relevance ranking and sentence saliency ranking are the two main tasks in extractive query-focused summarization. Previous supervised summarization systems often perform the two tasks in isolation. However, since reference summaries are the trade-off

Video summarization using a self-growing and self-organized neural gas network
DP Papadopoulos, SA Chatzichristofis… – … on Computer Vision …, 2011 – Springer
Abstract In this paper, a novel method to generate video summaries is proposed, which is allocated mainly for being applied to on-line videos. The novelty of this approach lies in the fact that the video summarization problem is considered as a single query image retrieval

Summarization-based video caption via deep neural networks
G Li, S Ma, Y Han – Proceedings of the 23rd ACM international …, 2015 – dl.acm.org
Abstract Generating appropriate descriptions for visual content draws increasing attention recently, where the promising progresses were obtained owing to the breakthroughs in deep neural networks. Different from the traditional SVO (subject, verb, object) based methods, in

A recurrent neural network language modeling framework for extractive speech summarization
KY Chen, SH Liu, B Chen, HM Wang… – Multimedia and Expo …, 2014 – ieeexplore.ieee.org
Abstract: Extractive speech summarization, with the purpose of automatically selecting a set of representative sentences from a spoken document so as to concisely express the most important theme of the document, has been an active area of research and development. A

Probabilistic neural network based text summarization
MA Fattah, F Ren – … and Knowledge Engineering, 2008. NLP-KE …, 2008 – ieeexplore.ieee.org
Abstract: This work proposes an approach to address the problem of improving content selection in automatic text summarization by using probabilistic neural network (PNN). This approach is a trainable summarizer, which takes into account several features, including

Document segmentation using textural features summarization and feedforward neural network
OK Oyedotun, A Khashman – Applied Intelligence, 2016 – Springer
Abstract Document Segmentation is a process that aims to filter documents while identifying certain regions of interest. Generally, the regions of interest include texts, graphics (image occupied regions) and the background. This paper presents a novel top-bottom approach to

Multiview convolutional neural networks for multidocument extractive summarization
Y Zhang, MJ Er, R Zhao… – IEEE transactions on …, 2017 – ieeexplore.ieee.org
Multidocument summarization has gained popularity in many real world applications because vital information can be extracted within a short time. Extractive summarization aims to generate a summary of a document or a set of documents by ranking sentences and

SummaRuNNer: A recurrent neural network based sequence model for extractive summarization of documents
R Nallapati, F Zhai, B Zhou – hiP (yi= 1| hi, si, d), 2017 – aaai.org
Abstract We present SummaRuNNer, a Recurrent Neural Network (RNN) based sequence model for extractive summarization of documents and show that it achieves performance better than or comparable to state-of-the-art. Our model has the additional advantage of

Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization
J Suzuki, M Nagata – Proceedings of the 15th Conference of the …, 2017 – aclweb.org
Abstract This paper tackles the reduction of redundant repeating generation that is often observed in RNN-based encoder-decoder models. Our basic idea is to jointly estimate the upper-bound frequency of each target vocabulary in the encoder and control the output

Graph-based Neural Multi-Document Summarization
M Yasunaga, R Zhang, K Meelu, A Pareek… – arXiv preprint arXiv …, 2017 – arxiv.org
Abstract: We propose a neural multi-document summarization (MDS) system that incorporates sentence relation graphs. We employ a Graph Convolutional Network (GCN) on the relation graphs, with sentence embeddings obtained from Recurrent Neural Networks

A semantic relevance based neural network for text summarization and text simplification
S Ma, X Sun – arXiv preprint arXiv:1710.02318, 2017 – arxiv.org
Abstract: Text summarization and text simplification are two major ways to simplify the text for poor readers, including children, non-native speakers, and the functionally illiterate. Text summarization is to produce a brief summary of the main ideas of the text, while text

Classify or select: Neural architectures for extractive document summarization
R Nallapati, B Zhou, M Ma – arXiv preprint arXiv:1611.04244, 2016 – arxiv.org
Abstract: We present two novel and contrasting Recurrent Neural Network (RNN) based architectures for extractive summarization of documents. The Classifier based architecture sequentially accepts or rejects each sentence in the original document order for its

Multi-document and multi-lingual summarization using neural networks
M Keyan, K Srinivasagan – International Conference on …, 2012 – pdfs.semanticscholar.org
ABSTRACT This system proposes Multi-lingual (Tamil and English) Multi-document summarization by neural networks. The system involves three steps. In first step, the sentences of the documents are converted into vector form. In the second step weight values

Abstractive sentence summarization with attentive deep recurrent neural networks
A Alifimoff – Journal of https://cs224d. stanford. edu/report …, 2015 – cs224d.stanford.edu
Summarization is the task of taking an input text and creating a supplementary text which contains the same meaning as the original text, but in less words. In Natural Language Processing, there are three primary approachs to summarization. The first is reducing the

Abstractive document summarization with a graph-based attentional neural model
J Tan, X Wan, J Xiao – Proceedings of the 55th Annual Meeting of the …, 2017 – aclweb.org
Abstract Abstractive summarization is the ultimate goal of document summarization research, but previously it is less investigated due to the immaturity of text generation techniques. Recently impressive progress has been made to abstractive sentence

Aspect-based Opinion Summarization with Convolutional Neural Networks
H Wu, Y Gu, S Sun, X Gu – Neural Networks (IJCNN), 2016 …, 2016 – ieeexplore.ieee.org
Abstract: This paper studies Aspect-based Opinion Summarization (AOS) of reviews on particular products. In practice, an AOS system needs to address two core subtasks, aspect extraction and sentiment classification. Most existing approaches to aspect extraction, using

Neural Abstractive Text Summarization.
G Rossiello – DC@ AI* IA, 2016 – aixia2016.unige.it
Abstract. Abstractive text summarization is a complex task whose goal is to generate a concise version of a text without necessarily reusing the sentences from the original source, but still preserving the meaning and the key contents. We address this issue by modeling the

Frequent itemsets summarization based on neural network
Z Zhao, J Qian, J Cheng, N Lu – Computer Science and …, 2009 – ieeexplore.ieee.org
Abstract: In this paper, we propose a Neural Network and cluster based method K-ANN-FP to summarize the frequent itemsets to solve the interpretability obstacle of the large number of frequent itemsets. This method assume that the item exit in each frequent itemsets or not

Experiments in Automatic Text Summarization Using Deep Neural Networks
B King, R Jha, T Johnson… – Machine …, 2011 – pdfs.semanticscholar.org
Abstract This project aims at applying neural network-based deep learning to the problem of extractive text summarization. Our work is inspired by the work of Collobert and Weston [Collobert et al., 2011], who created a unified deep learning architecture to learn several

Single Document Text Summarization using Random Indexing and Neural Networks.
N Chatterjee, A Bhardwaj – KEOD, 2010 – researchgate.net
Abstract: This paper presents a new extraction-based summarization technique developed using neural networks and Random Indexing. The technique exploits the advantages that a neural network provides in terms of compatibility and adaptability of a system as per the

Extractive speech summarization leveraging convolutional neural network techniques
CI Tsai, HT Hung, KY Chen… – … Workshop (SLT), 2016 …, 2016 – ieeexplore.ieee.org
Abstract: Extractive text or speech summarization endeavors to select representative sentences from a source document and assemble them into a concise summary, so as to help people to browse and assimilate the main theme of the document efficiently. The recent

An Unsupervised Multi-Document Summarization Framework Based on Neural Document Model.
S Ma, ZH Deng, Y Yang – COLING, 2016 – aclweb.org
Abstract In the age of information exploding, multi-document summarization is attracting particular attention for the ability to help people get the main ideas in a short time. Traditional extractive methods simply treat the document set as a group of sentences while ignoring the

An Examination of the CNN/DailyMail Neural Summarization Task
V Chen, ET Montaño, L Puzon – stanford.edu
Abstract Abstractive word summarization has proven to be difficult as the task involves a good understanding of a language model to reproduce both structurally correct and meaningful sentences. Given these difficult tasks we focus on improving the decoder, in the

A Pilot Study of Domain Adaptation Effect for Neural Abstractive Summarization
X Hua, L Wang – arXiv preprint arXiv:1707.07062, 2017 – arxiv.org
Abstract: We study the problem of domain adaptation for neural abstractive summarization. We make initial efforts in investigating what information can be transferred to a new domain. Experimental results on news stories and opinion articles indicate that neural summarization

Extractive document summarization based on convolutional neural networks
Y Zhang, MJ Er, M Pratama – Industrial Electronics Society …, 2016 – ieeexplore.ieee.org
Abstract: Extractive summarization aims to generate a summary by ranking sentences, whose performance relies heavily on the quality of sentence features. In this paper, a document summarization framework based on convolutional neural networks is successfully

OnSeS: A Novel Online Short Text Summarization Based on BM25 and Neural Network
J Niu, Q Zhao, L Wang, H Chen… – Global …, 2016 – ieeexplore.ieee.org
Abstract: The last decade has witnessed a dramatic growth of social networks, such as Twitter, Sina Microblog, etc. Messages/short texts on these platforms are generally of limited length, causing difficulties for machines to understand. Moreover, it is rarely possible for

From Neural Sentence Summarization to Headline Generation: A Coarse-to-Fine Approach
J Tan, X Wan, J Xiao – static.ijcai.org
Abstract Headline generation is a task of abstractive text summarization, and previously suffers from the immaturity of natural language generation techniques. Recent success of neural sentence summarization models shows the capacity of generating informative, fluent

Neural Text Summarization
U Khandelwal – cs224d.stanford.edu
Abstract Generation based text summarization is a hard task and recent deep learning attempts show that sequence to sequence models hold promise. In this paper, we approach this problem with a similar class of models, in a relatively smaller data setting and attempt to

Hierarchical Recurrent Neural Network for Video Summarization
B Zhao, X Li, X Lu – Proceedings of the 2017 ACM on Multimedia …, 2017 – dl.acm.org
Abstract Exploiting the temporal dependency among video frames or subshots is very important for the task of video summarization. Practically, RNN is good at temporal dependency modeling, and has achieved overwhelming performance in many video-based

Distraction-Based Neural Networks for Document Summarization
Q Chen, X Zhu, Z Ling, S Wei, H Jiang – arXiv preprint arXiv:1610.08462, 2016 – arxiv.org
Abstract: Distributed representation learned with neural networks has recently shown to be effective in modeling natural languages at fine granularities such as words, phrases, and even sentences. Whether and how such an approach can be extended to help model larger

The Summarization of the Research on Neural Network Control in the Thermal Electric Power Group
X YANG, Z YAN – Journal of Electric Power, 2012 – en.cnki.com.cn
This paper makes an introduction to the research and development of neutral network control technology, putting emphasis on the research of neutral network control system in the main control systems of thermal electric power, and outlines the researches in its other

Suitability of Artificial Neural Network to Text Document Summarization in the Indian Language-Kannada
R Jayashree, S Murthy, BS Anami – pdfs.semanticscholar.org
Abstract-The work explores the suitability of artificial neural network based summarizer to text document summarization in the Kannada language. A feed forward neural network, which is also called as back propagation network is trained on a corpus of text documents.

An artificial neural network approach to text document summarization in the Kannada language
R Jayashree, KS Murthy… – Hybrid Intelligent Systems …, 2013 – ieeexplore.ieee.org
Abstract: The paper discusses a machine learning approach that uses artificial neural networks to produce summaries of arbitrary length text documents. A feed forward neural network, which is also called as back propagation network is trained on a corpus of text

Design and Development of an Automatic Text Summarization Using Pragmatic-Enabled Features with LMS based Neural network
CS Kameswari, JA Chandulal – 2015 – irjet.net
ABSTRACT The main intend of the text summarization is to generate a condensed version of one or more texts using computer techniques. This will help reader to decide if a document contains needed information with minimum effort and time loss. In past decades, a number

Improving Neural Abstractive Text Summarization with Prior Knowledge
G Rossiello, P Basile, G Semeraro, M Di Ciano… – 2016 – lia.disi.unibo.it
Recurrent neural network (RNN) is a neural network model proposed in the 80’s for modelling time series. The structure of the network is similar to feedforward neural network, with the distinction that it allows a recurrent hidden state whose activation at each time is

CS585 Project Report Long Text Summarization using Neural Networks and Rule-Based Approach
S Liu – 2017 – shujianliu.com
1 Abstract Automatic text summarization is the task for computers to produce a concise and fluent summary conveying the key information in the input. There are generally two types of automatic texts summarization: extraction and abstraction. Many papers has been published

Faithful to the Original: Fact Aware Neural Abstractive Summarization
Z Cao, F Wei, W Li, S Li – arXiv preprint arXiv:1711.04434, 2017 – arxiv.org
Abstract: Unlike extractive summarization, abstractive summarization has to fuse different parts of the source text, which inclines to create fake facts. Our preliminary study reveals nearly 30% of the outputs from a state-of-the-art neural summarization system suffer from

Augmenting Neural Sentence Summarization through Extractive Summarization
J Zhu, L Zhou, H Li, J Zhang, Y Zhou… – National CCF Conference …, 2017 – Springer
Abstract Neural sequence-to-sequence model has achieved great success in abstractive summarization task. However, due to the limit of input length, most of previous works can only utilize lead sentences as the input to generate the abstractive summarization, which

Abstractive Summarization using a Feed-Forward Neural Attention Model
A Alifimoff, J Lee – pdfs.semanticscholar.org
Abstract We implement a model from Rush et al. which performs abstractive sentence summarization. We train the model over a series of text, summary pairs scraped from Wikipedia. We then try to combine this model with distributed vector representations of

Neural Extractive Summarization with Side Information
S Narayan, N Papasarantopoulos, M Lapata… – arXiv preprint arXiv …, 2017 – arxiv.org
Abstract: Most extractive summarization methods focus on the main body of the document from which sentences need to be extracted. The gist of the document often lies in the side information of the document, such as title and image captions. These types of side

Abstractive Document Summarization via Neural Model with Joint Attention
L Hou, P Hu, C Bei – National CCF Conference on Natural Language …, 2017 – Springer
Abstract Due to the difficulty of abstractive summarization, the great majority of past work on document summarization has been extractive, while the recent success of sequence-to-sequence framework has made abstractive summarization viable, in which a set of recurrent

News Article Summarization with Attention-based Deep Recurrent Neural Networks
H Yu, C Yue, C Wang – stanford.edu
Abstract This paper focuses on approaches to building a text automatic summarization model for news articles, generating a one-sentence summarization that mimics the style of a news title given some paragraphs. We managed to build and train two relatively complex

Automatic Document Summarization via Deep Neural Networks
C Yao, J Shen, G Chen – Computational Intelligence and …, 2015 – ieeexplore.ieee.org
Abstract: Automatic document summarization aim to extracting sentences which might cover the main content of a document or documents. To achieve this, many algorithms have been tried to rank the sentences by using task-specific features in a shallow architecture. The

Wireless capsule endoscopy video summarization: A learning approach based on Siamese neural network and support vector machine
J Chen, Y Zou, Y Wang – Pattern Recognition (ICPR), 2016 23rd …, 2016 – ieeexplore.ieee.org
Abstract: Wireless capsule endoscopy video summarization (WCE-VS) is highly demanded for eliminating redundant frames with high similarity. Conventional WCE-VS methods extract various hand-crafted features as image representations. Researches show that such

Sequence-guided siamese neural network for video summarization of unmanned aerial vehicles
J Chen, Y Wang, Z Chen, Y Zou – Digital Signal Processing …, 2017 – ieeexplore.ieee.org
Video summarization (VS) is one of key video signal processing techniques for unmanned aerial vehicles (UAVs). Essentially VS aims at eliminating redundant frames in aerial videos (AVs) with high similarity, which is helpful for quick browsing, retrieving and efficient storage

A Neural Model for Joint Event Detection and Summarization
Z Wang, Y Zhang – people.sutd.edu.sg
Abstract Twitter new event detection aims to identify first stories in a tweet stream. Typical approaches consider two sub tasks. First, it is necessary to filter out mundane or irrelevant tweets. Second, tweets are grouped automatically into event clusters. Traditionally, these

Leveraging Contextual Sentence Relations for Extractive Summarization Using a Neural A ention Model
P Ren, Z Chen, Z Ren, F Wei, J Ma, M de Rijke – 2017 – ir.sdu.edu.cn
ABSTRACT As a framework for extractive summarization, sentence regression has achieved state-of-the-art performance in several widely-used practical systems. e most challenging task within the sentence regression framework is to identify discriminative features to encode

A Neural Attention Model for Abstractive Sentence Summarization
ARSCJ Weston – nlp.seas.harvard.edu
Page 1. A Neural Attention Model for Abstractive Sentence Summarization Alexander Rush Sumit Chopra Jason Weston Facebook AI Research Harvard SEAS Rush, Chopra, Weston (Facebook AI) Neural Abstractive Summarization 1 / 42 Page 2. Sentence Summarization Source Russian

A Comparative Study Of Hindi Text Summarization Techniques: Genetic Algorithm And Neural Network
PG Student, DM COE – 2015 – academicscience.co.in
Abstract Automatic text summarization is a process which filters out the most essential part of the original source text/s. It eliminates the redundant, less important content and provides you with the vital information in a shorter version usually half a length of original text. As it

Video Summarization Using a Self-Growing and Self-Organized Neural Gas Network
N Papamarkos, D Papadopoulos – 2011 – hephaestus.nup.ac.cy
In this paper, a novel method to generate video summaries is proposed, which is allocated mainly for being applied to on-line videos. The novelty of this approach lies in the fact that the video summarization problem is considered as a single query image retrieval problem.

Query-Based Abstractive Summarization Using Neural Networks
J Hasselqvist, N Helmertz, M Kågebäck – arXiv preprint arXiv:1712.06100, 2017 – arxiv.org
Abstract: In this paper, we present a model for generating summaries of text documents with respect to a query. This is known as query-based summarization. We adapt an existing dataset of news article summaries for the task and train a pointer-generator model using this

Ambient Intelligent Monitoring of Dementia Suffers Using Unsupervised Neural Networks and Weighted Rule Based Summarisation
F Doctor, C Jayne, R Iqbal – … on Engineering Applications of Neural …, 2012 – Springer
Abstract This paper investigates the development of a system for monitoring of dementia suffers living in their own homes. The system uses unobtrusive pervasive sensor and actuator devices that can be deployed within a patient’s home grouped and accessed via

Category driven deep recurrent neural network for video summarization
X Song, K Chen, J Lei, L Sun, Z Wang… – Multimedia & Expo …, 2016 – ieeexplore.ieee.org
Abstract: A large number of videos are generated and uploaded to video websites (like youku, youtube) every day and video websites play more and more important roles in human life. While bringing convenience, the big video data raise the difficulty of video

Abstractive Text Summarization with Quasi-Recurrent Neural Networks
P Adelson, S Arora, J Hara – stanford.edu
Abstract We investigate a recent neural net model-Quasi-Recurrent Neural Networksand their application to abstractive text summarization, specifically generating a headline from the text of a news article. We use an encoder-decoder with attention model. We compare a

Neural network based approach to study the effect of feature selection on document summarization
DY Sakhare, D Rajkumar – enggjournals.com
Abstract As the amount of textual Information increases, we experience a need for Automatic Text Summarizers. In Automatic summarization a text document or a larger corpus of multiple documents are reduced to a short set of words or paragraph that conveys the main

Blind Equalization Algorithms Summarization Based on RBF Neural Network
J Ya-tian – Electronics Quality, 2009 – en.cnki.com.cn
RBF is a part imminent multilayer forward neural network. It has simple arithmetic, rapid convergent speed, good imminent result, better extensive ability. RBF can achieve nonlinear conversion from input space to output space by linear combination of nonlinear basis

Text Summarization using Neural Networks and Rhetorical Structure Theory
MK AR – pdfs.semanticscholar.org
Abstract: A new technique for summarization is presented here for summarizing articles known as text summarization using neural network and rhetorical structure theory. A neural network is trained to learn the relevant characteristics of sentences by using back