Neural Network & Dialog Systems 2018


Neural Dialog Systems (Neural Dialogue Systems)

Notes:

A statistical language model is a probability distribution over sequences of words.

  • Neural conversational agent
  • Neural conversation model
  • Neural dialog generation
  • Neural dialog generator
  • Neural dialog model
  • Neural dialog modelling
  • Neural dialog state tracker
  • Neural dialog state tracking
  • Neural dialog system
  • Neural end-of-turn detection
  • Neural language generation
  • Neural multivoice model
  • Neural named-entity recognition
  • Neural natural language understanding
  • Neural network language model
  • Neural user simulation
  • Reasoning neural network

Resources:

  • ijcnn.org .. international joint conference on neural networks
  • opennn.cimne.com .. multilayer perceptron neural network in  C++

Wikipedia:

References:

See also:

Classification Algorithms In Dialog Systems | Classifiers In Dialog Systems | Holographic Neural Technology | Ontology In Dialog SystemsParsing Algorithms & Dialog Systems


This table represents the relative importance of neural network topics to dialog systems, based on the number of academic publications returned for the period 2012 to 2017. Topics with zero significance have been removed.

Perceptron 472
Recurrent neural network 461
Long short-term memory 365
Activation function 247
Word embedding 236
Backpropagation 213
Radial basis function 200
Sigmoid function 196
Multilayer perceptron 195
Word2vec 191
Convolutional neural network 180
Dropout (neural networks) 179
Feedforward (control) 124
Softmax function 123
Autoencoder 105
Early stopping 91
Multimodal learning 63
Vanishing gradient problem 55
Deep belief network 54
Feedforward neural network 53
Backpropagation through time 49
Recursive neural network 45
Boltzmann machine 42
Self-organizing map 41
Probabilistic neural network 40
Restricted Boltzmann machine 37
Learning rule 33
Learning vector quantization 31
Bidirectional recurrent neural networks 29
Artificial Intelligence System 25
Time delay neural network 20
Competitive learning 19
Rprop 19
Artificial neuron 17
Rectifier (neural networks) 17
Adaptive neuro fuzzy inference system 15
Radial basis function network 15
Caffe (software) 14
Neural Networks (journal) 13
Adaptive resonance theory 12
Neural gas 12
Spiking neural network 11
Winner-take-all (computing) 11
Reservoir computing 9
CoDi 8
Computational cybernetics 8
Convolutional Deep Belief Networks 8
Modular neural network 8
Delta rule 7
Echo state network 7
The Emotion Machine 7
Committee machine 6
Hierarchical temporal memory 6
Infomax 6
Quickprop 6
Deeplearning4j 5
Hybrid neural network 5
Liquid state machine 5
Synaptic weight 5
European Neural Network Society 4
Neocognitron 4
Catastrophic interference 3
Generalized Hebbian Algorithm 3
Hopfield network 3
U-matrix 3
ADALINE 2
Bidirectional associative memory 2
Counter propagation network 2
Leabra 2
Linde–Buzo–Gray algorithm 2
NETtalk (artificial neural network) 2
Randomneural network 2
Universal approximation theorem 2
Cellular neural network 1
Compositional pattern-producing network 1
Helmholtz machine 1
MoneyBee 1
Neuroevolution of augmenting topologies 1
Physical neural network 1

Unsupervised discrete sentence representation learning for interpretable neural dialog generation
T Zhao, K Lee, M Eskenazi – arXiv preprint arXiv:1804.08069, 2018 – arxiv.org
The encoder-decoder dialog model is one of the most prominent methods used to build dialog systems in complex domains. Yet it is limited because it cannot output interpretable actions as in traditional systems, which hinders humans from understanding its generation …

Knowledge diffusion for neural dialogue generation
S Liu, H Chen, Z Ren, Y Feng, Q Liu, D Yin – Proceedings of the 56th …, 2018 – aclweb.org
End-to-end neural dialogue generation has shown promising results recently, but it does not employ knowledge to guide the generation and hence tends to generate short, general, and meaningless responses. In this paper, we propose a neural knowledge diffusion (NKD) …

On the effects of using word2vec representations in neural networks for dialogue act recognition
C Cerisara, P Kral, L Lenc – Computer Speech & Language, 2018 – Elsevier
Dialogue act recognition is an important component of a large number of natural language processing pipelines. Many research works have been carried out in this area, but relatively few investigate deep neural networks and word embeddings. This is surprising, given that …

Bootstrapping a neural conversational agent with dialogue self-play, crowdsourcing and on-line reinforcement learning
P Shah, D Hakkani-Tur, B Liu, G Tur – … of the 2018 Conference of the …, 2018 – aclweb.org
End-to-end neural models show great promise towards building conversational agents that are trained from data and on-line experience using supervised and reinforcement learning. However, these models require a large corpus of dialogues to learn effectively. For goal …

Neural user simulation for corpus-based policy optimisation for spoken dialogue systems
F Kreyssig, I Casanueva, P Budzianowski… – arXiv preprint arXiv …, 2018 – arxiv.org
User Simulators are one of the major tools that enable offline training of task-oriented dialogue systems. For this task the Agenda-Based User Simulator (ABUS) is often used. The ABUS is based on hand-crafted rules and its output is in semantic form. Issues arise from …

Neural multivoice models for expressing novel personalities in dialog
S Oraby, L Reed, S Tandon, M Walker – arXiv preprint arXiv:1809.01331, 2018 – arxiv.org
Natural language generators for task-oriented dialog should be able to vary the style of the output utterance while still effectively realizing the system dialog actions and their associated semantics. While the use of neural generation for training the response …

Visual coreference resolution in visual dialog using neural module networks
S Kottur, JMF Moura, D Parikh… – Proceedings of the …, 2018 – openaccess.thecvf.com
Visual dialog entails answering a series of questions grounded in an image, using dialog history as context. In addition to the challenges found in visual question answering (VQA), which can be seen as one-round dialog, visual dialog encompasses several more. We focus …

Dialog generation using multi-turn reasoning neural networks
X Wu, A Martinez, M Klyen – Proceedings of the 2018 Conference of the …, 2018 – aclweb.org
In this paper, we propose a generalizable dialog generation approach that adapts multiturn reasoning, one recent advancement in the field of document comprehension, to generate responses (“answers”) by taking current conversation session context as a “document” and …

A Context-based Approach for Dialogue Act Recognition using Simple Recurrent Neural Networks
C Bothe, C Weber, S Magg, S Wermter – arXiv preprint arXiv:1805.06280, 2018 – arxiv.org
Dialogue act recognition is an important part of natural language understanding. We investigate the way dialogue act corpora are annotated and the learning approaches used so far. We find that the dialogue act is context-sensitive within the conversation for most of …

Integrated neural network model for identifying speech acts, predicators, and sentiments of dialogue utterances
M Kim, H Kim – Pattern Recognition Letters, 2018 – Elsevier
A dialogue system should capture speakers’ intentions, which can be represented by combinations of speech acts, predicators, and sentiments. To identify these intentions from speakers’ utterances, many studies have independently dealt with speech acts, predicators …

Adversarial learning of task-oriented neural dialog models
B Liu, I Lane – arXiv preprint arXiv:1805.11762, 2018 – arxiv.org
In this work, we propose an adversarial learning method for reward estimation in reinforcement learning (RL) based task-oriented dialog models. Most of the current RL based task-oriented dialog systems require the access to a reward signal from either user …

Importance of a Search Strategy in Neural Dialogue Modelling
I Kulikov, AH Miller, K Cho, J Weston – arXiv preprint arXiv:1811.00907, 2018 – arxiv.org
Search strategies for generating a response from a neural dialogue model have received relatively little attention compared to improving network architectures and learning algorithms in recent years. In this paper, we consider a standard neural dialogue model …

Cross-language neural dialog state tracker for large ontologies using hierarchical attention
Y Jang, J Ham, BJ Lee, KE Kim – IEEE/ACM Transactions on …, 2018 – ieeexplore.ieee.org
Dialog state tracking, which refers to identifying the user intent from utterances, is one of the most important tasks in dialog management. In this paper, we present our dialog state tracker developed for the fifth dialog state tracking challenge, which focused on cross …

Structured dialogue policy with graph neural networks
L Chen, B Tan, S Long, K Yu – … of the 27th International Conference on …, 2018 – aclweb.org
Recently, deep reinforcement learning (DRL) has been used for dialogue policy optimization. However, many DRL-based policies are not sample-efficient. Most recent advances focus on improving DRL optimization algorithms to address this issue. Here, we …

Can Neural Generators for Dialogue Learn Sentence Planning and Discourse Structuring?
L Reed, S Oraby, M Walker – arXiv preprint arXiv:1809.03015, 2018 – arxiv.org
Responses in task-oriented dialogue systems often realize multiple propositions whose ultimate form depends on the use of sentence planning and discourse structuring operations. For example a recommendation may consist of an explicitly evaluative utterance …

Adversarial Domain Adaptation for Variational Neural Language Generation in Dialogue Systems
VK Tran, LM Nguyen – arXiv preprint arXiv:1808.02586, 2018 – arxiv.org
Domain Adaptation arises when we aim at learning from source domain a model that can per-form acceptably well on a different target domain. It is especially crucial for Natural Language Generation (NLG) in Spoken Dialogue Systems when there are sufficient …

Eliciting positive emotion through affect-sensitive dialogue response generation: A neural network approach
N Lubis, S Sakti, K Yoshino, S Nakamura – Thirty-Second AAAI Conference …, 2018 – aaai.org
An emotionally-competent computer agent could be a valuable assistive technology in performing various affective tasks. For example caring for the elderly, low-cost ubiquitous chat therapy, and providing emotional support in general, by promoting a more positive …

Convolutional neural networks for dialogue state tracking without pre-trained word vectors or semantic dictionaries
M Korpusik, J Glass – Proceedings of 2018 IEEE Spoken …, 2018 – groups.csail.mit.edu
ABSTRACT A crucial step in task-oriented dialogue systems is tracking the user’s goal over the course of the conversation. This involves maintaining a probability distribution over possible values for each slot (eg, the food slot might map to the value Turkish), which gets …

Neural Dialogue System with Emotion Embeddings
R Shantala, G Kyselov… – 2018 IEEE First …, 2018 – ieeexplore.ieee.org
Emotional intelligence is a vital human mechanism that allows people to identify and react to different feelings in various environments, especially conversations. That is why it is important to address the emotional aspect of generative dialogue systems. We propose to …

Toward Scalable Neural Dialogue State Tracking Model
E Nouri, E Hosseini-Asl – arXiv preprint arXiv:1812.00899, 2018 – arxiv.org
The latency in the current neural based dialogue state tracking models prohibits them from being used efficiently for deployment in production systems, albeit their highly accurate performance. This paper proposes a new scalable and accurate neural dialogue state …

Neural Dialogue Context Online End-of-Turn Detection
R Masumura, T Tanaka, A Ando, R Ishii… – … and Dialogue, 2018 – aclweb.org
This paper proposes a fully neural network based dialogue-context online end-of-turn detection method that can utilize longrange interactive information extracted from both target speaker’s and interlocutor’s utterances. In the proposed method, we combine multiple time …

Unsupervised Counselor Dialogue Clustering for Positive Emotion Elicitation in Neural Dialogue System
N Lubis, S Sakti, K Yoshino, S Nakamura – … on Discourse and Dialogue, 2018 – aclweb.org
Positive emotion elicitation seeks to improve user’s emotional state through dialogue system interaction, where a chatbased scenario is layered with an implicit goal to address user’s emotional needs. Standard neural dialogue system approaches still fall short in this situation …

A Two-Step Neural Dialog State Tracker for Task-Oriented Dialog Processing
A Kim, HJ Song, SB Park – Computational intelligence and …, 2018 – hindawi.com
Dialog state tracking in a spoken dialog system is the task that tracks the flow of a dialog and identifies accurately what a user wants from the utterance. Since the success of a dialog is influenced by the ability of the system to catch the requirements of the user, accurate state …

Another Diversity-Promoting Objective Function for Neural Dialogue Generation
R Nakamura, K Sudoh, K Yoshino… – arXiv preprint arXiv …, 2018 – arxiv.org
Although generation-based dialogue systems have been widely researched, the response generations by most existing systems have very low diversities. The most likely reason for this problem is Maximum Likelihood Estimation (MLE) with Softmax Cross-Entropy (SCE) …

Explicit State Tracking with Semi-Supervisionfor Neural Dialogue Generation
X Jin, W Lei, Z Ren, H Chen, S Liang, Y Zhao… – Proceedings of the 27th …, 2018 – dl.acm.org
The task of dialogue generation aims to automatically provide responses given previous utterances. Tracking dialogue states is an important ingredient in dialogue generation for estimating users’ intention. However, the expensive nature of state labeling and the weak …

An End-to-End Neural Dialog State Tracking for Task-Oriented Dialogs
AY Kim, TH Kim, HJ Song… – 2018 IEEE International …, 2018 – ieeexplore.ieee.org
Dialog state tracking in spoken dialog system is the task that tracks the flow of a dialog and grasps what a user wants from the utterance precisely. Since the dialog success is related to catching the want of the user, dialog state tracking is a necessary component for spoken …

Improving Robustness of Neural Dialog Systems in a Data-Efficient Way with Turn Dropout
I Shalyminov, S Lee – arXiv preprint arXiv:1811.12148, 2018 – arxiv.org
Neural network-based dialog models often lack robustness to anomalous, out-of-domain (OOD) user input which leads to unexpected dialog behavior and thus considerably limits such models’ usage in mission-critical production environments. The problem is especially …

Recurrent neural network language generation for dialogue systems
TH Wen – 2018 – ethos.bl.uk
Language is the principal medium for ideas, while dialogue is the most natural and effective way for humans to interact with and access information from machines. Natural language generation (NLG) is a critical component of spoken dialogue and it has a significant impact …

Learning Task-Oriented Dialog with Neural Network Methods
B Liu – 2018 – bingliu.me
Dialog system is class of intelligent system that interacts with human via natural language interfaces with a coherent structure. Based on the nature of the conversation, dialog systems are generally divided into two sub-classes, task-oriented dialog systems that are created to …

Cascaded deep neural network models for dialog state tracking
G Yang, X Wang – Multimedia Tools and Applications, 2018 – Springer
Dialog state tracking (DST) maintains and updates dialog states at each time step as the dialog progresses. It is necessary to include dialog historical information in DST. Previous word-based DST models took historical utterances as a word sequence and used n-grams …

Neural-based methods for user simulation in dialog systems
M Schmidt – 2018 – elib.uni-stuttgart.de
Spoken Dialog Systems ermöglichen es Nutzern mittels Sprache oder Text, Aufgaben zu erledigen oder einfachen Zugang zu einer Datenbank zu erhalten. State-of-the-art Ansätze modellieren dieses Problem als Markov Decision Process. Dies ermöglicht den Einsatz von …

Extending Output Attentions In Recurrent Neural Networks For Dialog Generation
C Lee – aircconline.com
In natural language processing, attention mechanism in neural networks are widely utilized. In this paper, the research team explore a new mechanism of extending output attention in recurrent neural networks for dialog systems. The new attention method was compared with …

Lexico-Acoustic Neural-Based Models for Dialog Act Classification
D Ortega, NT Vu – … on Acoustics, Speech and Signal Processing …, 2018 – ieeexplore.ieee.org
Recent works have proposed neural models for dialog act classification in spoken dialogs. However, they have not explored the role and the usefulness of acoustic information. We propose a neural model that processes both lexical and acoustic features for classification …

Long Short-Term Memory Neural Networks for Artificial Dialogue Generation
SA Selouani, MS Yacoub – 2018 IEEE 42nd Annual Computer …, 2018 – ieeexplore.ieee.org
This paper investigates both of user and system modeling to extend an existing corpus of human-machine dialogue data with simulated/artificial dialogues. To simulate and generate such artificial dialogues, a long-short term memory (LSTM) neural network system is …

A Unified Neural Architecture for Joint Dialog Act Segmentation and Recognition in Spoken Dialog System
T Zhao, T Kawahara – … SIGdial Meeting on Discourse and Dialogue, 2018 – aclweb.org
In spoken dialog systems (SDSs), dialog act (DA) segmentation and recognition provide essential information for response generation. A majority of previous works assumed ground-truth segmentation of DA units, which is not available from automatic speech recognition …

A Neural Method for Goal-Oriented Dialog Systems to interact with Named Entities
J Rajendran, J Ganhotra, X Guo, M Yu, S Singh – 2018 – openreview.net
Many goal-oriented dialog tasks, especially ones in which the dialog system has to interact with external knowledge sources such as databases, have to handle a large number of Named Entities (NEs). There are at least two challenges in handling NEs using neural  …

Changing the Level of Directness in Dialogue using Dialogue Vector Models and Recurrent Neural Networks
L Pragst, S Ultes – … Annual SIGdial Meeting on Discourse and Dialogue, 2018 – aclweb.org
In cooperative dialogues, identifying the intent of ones conversation partner and acting accordingly is of great importance. While this endeavour is facilitated by phrasing intentions as directly as possible, we can observe in human-human communication that a number of …

Probabilistic word association for dialogue act classification with recurrent neural networks
N Duran, S Battle – … Conference on Engineering Applications of Neural …, 2018 – Springer
Abstract The identification of Dialogue Act’s (DA) is an important aspect in determining the meaning of an utterance for many applications that require natural language understanding, and recent work using recurrent neural networks (RNN) has shown promising results when …

A Dual-Attention Hierarchical Recurrent Neural Network for Dialogue Act Classification
R Li, C Lin, M Collinson, X Li, G Chen – arXiv preprint arXiv:1810.09154, 2018 – arxiv.org
Recognising dialogue acts (DA) is important for many natural language processing tasks such as dialogue generation and intention recognition. In this paper, we propose a dual-attention hierarchical recurrent neural network for dialogue act classification. Our model is …

Implementation of A Neural Natural Language Understanding Component for Arabic Dialogue Systems
AM Bashir, A Hassan, B Rosman, D Duma… – Procedia computer …, 2018 – Elsevier
Abstract Natural Language Understanding (NLU) is considered a core component in implementing dialogue systems. NLU has been greatly enhanced by deep learning techniques such as word embeddings and deep neural network architectures, but current …

Engagement Recognition in Spoken Dialogue via Neural Network by Aggregating Different Annotators’ Models
K Inoue, D Lala, K Takanashi… – Proc. Interspeech …, 2018 – isca-speech.org
This paper addresses engagement recognition based on four multimodal listener behaviors-backchannels, laughing, eyegaze, and head nodding. Engagement is an indicator of how much a user is interested in the current dialogue. Multiple third-party annotators give ground …

Dialogue Act Classification in Reference Interview Using Convolutional Neural Network with Byte Pair Encoding
S Kawano, K Yoshino, Y Suzuki, S Nakamura – ahcweb01.naist.jp
Dialogue act classification is an important component of dialogue management, which captures the user’s intention and chooses the appropriate response action. In this paper, we focus on the dialogue act classification in reference interviews to model the behaviors of …

Effects of dimensional input on paralinguistic information perceived from synthesized dialogue speech with neural network
M Yokoyama, T Nagata, H Mori – Proc. Interspeech 2018, 2018 – isca-speech.org
A novel method of controlling paralinguistic information in neural network-based dialogue speech synthesis is proposed. Controlling paralinguistic information was achieved by feeding emotion dimensions in continuous values into the input layer of the neural networks …

Dialogue Act Classification Model Based on Deep Neural Networks for a Natural Language Interface to Databases in Korean
M Kim, H Kim – 2018 IEEE International Conference on Big …, 2018 – ieeexplore.ieee.org
Dialogue act classification is an essential task for implementing a natural language interface to databases because speakers’ intentions can be represented by dialogue acts (domain-independent speech act and domain-dependent predicator pairs). To resolve ambiguities in …

Exploring the importance of context and embeddings in neural NER models for task-oriented dialogue systems
P Jayarao, C Jain, A Srivastava – arXiv preprint arXiv:1812.02370, 2018 – arxiv.org
Named Entity Recognition (NER), a classic sequence labelling task, is an essential component of natural language understanding (NLU) systems in task-oriented dialog systems for slot filling. For well over a decade, different methods from lookup using …

On Modelling Uncertainty in Neural Language Generation for Policy Optimisation in Voice-Triggered Dialog Assistants
S Wang, T Gunter, D VanDyke – alborz-geramifard.com
While much effort has gone into user-modelling in the context of simulation for dialog policy training through reinforcement-learning (RL), the majority of this research has focused on matching user behaviour, with relatively little work dedicated to accurately replicating system …

Auto-Dialog Systems: Implementing Automatic Conversational Man-Machine Agents by Using Artificial Intelligence & Neural Networks
A Bala, T Padmaja, GKD Gopisettry – chsd-theresacollege.net
Many companies are hoping to develop bots to have natural conversations indistinguishable from human ones, and many are claiming to be using Neuro-Linguistic Programming and Deep Learning techniques to make this possible. Microsoft is making big bets on chat bots …

(Visited 145 times, 1 visits today)