Notes:
A statistical language model is a type of machine learning model that uses probability to generate text. It is trained on a large corpus of text data, and uses statistical patterns in the data to make predictions about the likelihood of different sequences of words. The model can then be used to generate new text that is similar to the text it was trained on. This can be used in a variety of natural language processing tasks such as text generation, language translation, and text classification. Additionally, the most advanced models are pre-trained on massive data sets and fine-tuned on smaller task-specific data sets to perform specific tasks with high accuracy.
Wikipedia:
See also:
IRSTLM (IRST Language Modeling) Toolkit 2016 | Language Modeling & Dialog Systems 2017 | Neural Conversation Models 2016 | Neural Dialog Systems | Neural Language Models 2016 | Rule-based Language Modeling
Learning discourse-level diversity for neural dialog models using conditional variational autoencoders
T Zhao, R Zhao, M Eskenazi – arXiv preprint arXiv:1703.10960, 2017 – arxiv.org
Abstract: While recent neural encoder-decoder models have shown great promise in modeling open-domain conversations, they often generate dull and generic responses. Unlike past work that has focused on diversifying the output of the decoder at word-level to
Challenging Neural Dialogue Models with Natural Data: Memory Networks Fail on Incremental Phenomena
I Shalyminov, A Eshghi, O Lemon – arXiv preprint arXiv:1709.07840, 2017 – arxiv.org
Abstract: Natural, spontaneous dialogue proceeds incrementally on a word-by-word basis; and it contains many sorts of disfluency such as mid-utterance/sentence hesitations, interruptions, and self-corrections. But training data for machine learning approaches to
A Dual Encoder Sequence to Sequence Model for Open-Domain Dialogue Modeling
S Tandon, R Bauer – arXiv preprint arXiv:1710.10520, 2017 – arxiv.org
… process of generating the current system-response. For neural dialogue models, the idea of incorporat- ing prior context during generation of a response started off with (Sordoni et al., 2015). Genera- tive neural models, in particular …
Iterative policy learning in end-to-end trainable task-oriented neural dialog models
B Liu, I Lane – arXiv preprint arXiv:1709.06136, 2017 – arxiv.org
Abstract: In this paper, we present a deep reinforcement learning (RL) framework for iterative dialog policy optimization in end-to-end task-oriented dialog systems. Popular approaches in learning dialog policy with RL include letting a dialog agent to learn against a user
Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model
J Lu, A Kannan, J Yang, D Parikh… – Advances in Neural …, 2017 – papers.nips.cc
… the human responses. Across a variety of domains, a recurring problem with MLE trained generative neural dialog models (G) is that they tend to produce ‘safe’ and generic responses (‘I don’t know’, ‘I can’t tell’). In contrast, discriminative …
End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning
B Liu, G Tur, D Hakkani-Tur, P Shah, L Heck – arXiv preprint arXiv …, 2017 – arxiv.org
… arXiv preprint arXiv:1703.01008, 2017. [14] Bing Liu and Ian Lane. Iterative policy learning in end-to-end trainable task-oriented neural dialog models. In ASRU, 2017. [15] Jason D Williams, Kavosh Asadi, and Geoffrey Zweig …
A knowledge-grounded neural conversation model
M Ghazvininejad, C Brockett, MW Chang… – arXiv preprint arXiv …, 2017 – arxiv.org
Page 1. A Knowledge-Grounded Neural Conversation Model Marjan Ghazvininejad1* Chris Brockett2 Ming-Wei Chang2 Bill Dolan2 Jianfeng Gao2 Wen-tau Yih2 Michel Galley2 1Information Sciences Institute, USC 2Microsoft Research …
Coherent Dialogue with Attention-Based Language Models.
H Mei, M Bansal, MR Walter – AAAI, 2017 – aaai.org
… (2016a)) to mea- sure the ability of the A-RNN to promote diversity in the generations, because typical neural dialogue models gener- ate generic, safe responses (technically appropriate but not informative, eg, “I dont know”) …
An end-to-end trainable neural network model with belief tracking for task-oriented dialog
B Liu, I Lane – arXiv preprint arXiv:1708.05956, 2017 – arxiv.org
… After collecting new evidence from a user’s input at turn k, the neural dialog model updates the probability distribution P(Sm k ) over candi- date values for each slot type m ? M. For example, in restau- rant search domain, the model maintains a multinomial proba- bility …
FlipDial: A Generative Model for Two-Way Visual Dialogue
D Massiceti, N Siddharth, PK Dokania… – arXiv preprint arXiv …, 2018 – arxiv.org
Page 1. FLIPDIAL: A Generative Model for Two-Way Visual Dialogue Daniela Massiceti University of Oxford, UK daniela@robots.ox.ac.uk N. Siddharth University of Oxford, UK nsid@robots.ox. ac.uk Puneet Kumar Dokania University of Oxford, UK puneet@robots.ox.ac.uk …
Generative encoder-decoder models for task-oriented spoken dialog systems with chatting capability
T Zhao, A Lu, K Lee, M Eskenazi – arXiv preprint arXiv:1706.08476, 2017 – arxiv.org
Page 1. Generative Encoder-Decoder Models for Task-Oriented Spoken Dialog Systems with Chatting Capability Tiancheng Zhao, Allen Lu, Kyusong Lee and Maxine Eskenazi Language Technologies Institute Carnegie Mellon …
Adversarial Advantage Actor-Critic Model for Task-Completion Dialogue Policy Learning
B Peng, X Li, J Gao, J Liu, YN Chen… – arXiv preprint arXiv …, 2017 – arxiv.org
… learning,” in EMNLP, 2017, pp. 2221–2230. [9] Bing Liu and Ian Lane, “Iterative policy learning in end-to-end trainable task-oriented neural dialog models,” arXiv preprint arXiv:1709.06136, 2017. [10] Zachary C Lipton, Jianfeng …
Modeling Situations in Neural Chat Bots
S Sato, N Yoshinaga, M Toyoda… – Proceedings of ACL 2017 …, 2017 – aclweb.org
Page 1. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics- Student Research Workshop, pages 120–127 Vancouver, Canada, July 30 – August 4, 2017. c 2017 Association for Computational …
Deep Learning for Dialogue Systems
YN Chen, A Celikyilmaz, D Hakkani-Tür – Proceedings of ACL 2017 …, 2017 – aclweb.org
… More re- cent work on Neural Dialog Managers that provide conjoint representations between the utterances, slot-value pairs as well as knowledge graph repre- sentations (Wen et al., 2016; Mrkšic et al., 2016) demonstrate that using neural dialog models can overcome current …
Affective Neural Response Generation
N Asghar, P Poupart, J Hoey, X Jiang, L Mou – arXiv preprint arXiv …, 2017 – arxiv.org
… However, emotional aspects are not explicitly captured by existing methods. Our goal is to alleviate this issue in open-domain neural dialogue models by augmenting them with affective intelli- gence. We do this in three ways …
Exploring personalized neural conversational models
S Kottur, X Wang, VR Carvalho – Proceedings of the 26th …, 2017 – xiaoyumu.com
… One of the first attempts of neural dialog models was proposed by Vinyals and Le [Vinyals and Le, 2015], where a dialog response is generated from a dialog question (or previous sentence) in a sequence-to-sequence framework …
Common Sense Knowledge in Large Scale Neural Conversational Models
DS Tarasov, ED Izotova – International Conference on Neuroinformatics, 2017 – Springer
… questions related to the single entity (Table 5). 4 Conclusions. We found that large neural dialog models can learn some common-sense knowledge, although to the limited extent. There is, however, a room for improvement, because …
Reference-aware language models
Z Yang, P Blunsom, C Dyer, W Ling – arXiv preprint arXiv:1611.01628, 2016 – arxiv.org
Page 1. Under review as a conference paper at ICLR 2017 REFERENCE-AWARE LANGUAGE MODELS Zichao Yang1?, Phil Blunsom2,3, Chris Dyer1,2, and Wang Ling2 1Carnegie Mellon University, 2DeepMind, and 3University …
Key-Value Retrieval Networks for Task-Oriented Dialogue
M Eric, CD Manning – arXiv preprint arXiv:1705.05414, 2017 – arxiv.org
… While this work has been somewhat successful, these task-oriented neural dialogue models suffer from a number of problems: 1) They struggle to effectively reason over and incorporate knowledge base information while still preserving their end- to-end trainability and 2) They …
Natural Language Does Not Emerge’Naturally’in Multi-Agent Dialog
S Kottur, JMF Moura, S Lee, D Batra – arXiv preprint arXiv:1706.08502, 2017 – arxiv.org
… While historically such agents have been based on slot filling (Lemon et al., 2006), the domi- nant paradigm today is neural dialog models (Bor- des and Weston, 2016; Weston, 2016; Serban et al., 2016a,b) trained on large quantities of data …
Integrating planning for task-completion dialogue policy learning
B Peng, X Li, J Gao, J Liu, KF Wong – arXiv preprint arXiv:1801.06176, 2018 – arxiv.org
Page 1. Integrating planning for task-completion dialogue policy learning Baolin Peng?? Xiujun Li† Jianfeng Gao† Jingjing Liu† Kam-Fai Wong? †Microsoft Research, Redmond, WA, USA ?The Chinese University of Hong …
Z-Forcing: Training Stochastic Recurrent Networks
AGAP GOYAL, A Sordoni, MA Côté, N Ke… – Advances in Neural …, 2017 – papers.nips.cc
Paper accepted and presented at the Neural Information Processing Systems Conference (http://nips.cc/).
Bootstrapping incremental dialogue systems from minimal data: the generalisation power of dialogue grammars
A Eshghi, I Shalyminov, O Lemon – arXiv preprint arXiv:1709.07858, 2017 – arxiv.org
Page 1. Bootstrapping incremental dialogue systems from minimal data: the generalisation power of dialogue grammars Arash Eshghi Interaction Lab Heriot-Watt University a.eshghi@hw.ac.uk Igor Shalyminov Interaction Lab Heriot-Watt University is33@hw.ac.uk …
Neural Response Generation with Dynamic Vocabularies
Y Wu, W Wu, D Yang, C Xu, Z Li, M Zhou – arXiv preprint arXiv:1711.11191, 2017 – arxiv.org
Page 1. Neural Response Generation with Dynamic Vocabularies Yu Wu†, Wei Wu‡, Dejian Yang†, Can Xu‡,Zhoujun Li†, Ming Zhou‡ †State Key Lab of Software Development Environment, Beihang University, Beijing, China …
Elastic Responding Machine for Dialog Generation with Dynamically Mechanism Selecting
G Zhou, P Luo, Y Xiao, F Lin, B Chen, Q He – 2018 – researchgate.net
Page 1. Elastic Responding Machine for Dialog Generation with Dynamically Mechanism Selecting Ganbin Zhou1,2, Ping Luo1,2, Yijun Xiao3, Fen Lin4, Bo Chen4, Qing He1,2 1Key Lab of Intelligent Information Processing …
Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
R Barzilay, MY Kan – Proceedings of the 55th Annual Meeting of the …, 2017 – aclweb.org
Page 1. ACL 2017 The 55th Annual Meeting of the Association for Computational Linguistics Proceedings of the Conference, Vol. 1 (Long Papers) July 30 – August 4, 2017 Vancouver, Canada Page 2. Platinum Sponsors: Gold Sponsors: ii Page 3. Silver Sponsors …
Sample efficient deep reinforcement learning for dialogue systems with large action spaces
G Weisz, P Budzianowski, PH Su, M Gaši? – arXiv preprint arXiv …, 2018 – arxiv.org
Page 1. 1 Sample efficient deep reinforcement learning for dialogue systems with large action spaces Gellért Weisz, Pawe? Budzianowski, Student Member, IEEE, Pei-Hao Su, Student Member, IEEE, and Milica Gašic, Member, IEEE …
Topic-based Evaluation for Conversational Bots
F Guo, A Metallinou, C Khatri, A Raju… – arXiv preprint arXiv …, 2018 – arxiv.org
… [24] T. Zhao, R. Zhao, and M. Eskenazi. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, page 654–664, 2017. 10
Wake-Sleep Variational Autoencoders for Language Modeling
X Shen, H Su, S Niu, D Klakow – International Conference on Neural …, 2017 – Springer
… In: Association for the Advancement of Artificial Intelligence (2017)Google Scholar. 6. Zhao, T., Zhao, R., Eskenazi, M.: Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In …
MojiTalk: Generating Emotional Responses at Scale
X Zhou, WY Wang – arXiv preprint arXiv:1711.04090, 2017 – arxiv.org
Page 1. MOJITALK: Generating Emotional Responses at Scale Xianda Zhou Dept. of Computer Science and Technology Tsinghua University Beijing, 100084 China zhou-xd13@mails.tsinghua.edu.cn William Yang Wang Department …
Generating Thematic Chinese Poetry with Conditional Variational Autoencoder
X Yang, X Lin, S Suo, M Li – arXiv preprint arXiv:1711.07632, 2017 – arxiv.org
Page 1. Generating Thematic Chinese Poetry with Conditional Variational Autoencoder Xiaopeng Yang, Xiaowen Lin, Shunda Suo, and Ming Li David R. Cheriton School of Computer Science University of Waterloo Waterloo …
Learning Generative End-to-end Dialog Systems with Knowledge
T Zhao – 2017 – cs.cmu.edu
Page 1. November 21, 2017 DRAFT Learning Generative End-to-end Dialog Systems with Knowledge Tiancheng Zhao December 2017 Language Technologies Institute School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 …
Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction
E Schmerling, K Leung, W Vollprecht… – arXiv preprint arXiv …, 2017 – arxiv.org
Page 1. Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction Edward Schmerling1 Karen Leung2 Wolf Vollprecht3 Marco Pavone2 Abstract— This paper presents a method for constructing human-robot …