Word2vec & Dialog Systems 2015


Resources:

Wikipedia:

See also:

100 Best GitHub: N-gram | 100 Best GitHub: Natural Language | SkipGrams 2013


Learning OOV through semantic relatedness in spoken dialog systems M Sun, YN Chen, AI Rudnicky – Proc. The 16th Annual Conference of …, 2015 – cs.cmu.edu … Inevitably, a dialog system with static vocabulary has to face the OOV issue after deployment, eg, with the newly created words such as “selfie”. … 1https://code.google.com/p/word2vec/ Page 3. The learned W? satisfies W? = arg max W ? w?W Mw · fD T , (2) … Cited by 5 Related articles All 4 versions

Evaluating prerequisite qualities for learning end-to-end dialog systems J Dodge, A Gane, X Zhang, A Bordes, S Chopra… – arXiv preprint arXiv: …, 2015 – arxiv.org … The design of our tasks is such that all test one or more key characteristics a dialog system should have but also that an … one of the major uses of word embedding models is to learn unsupervised embeddings over large unlabeled datasets such as in Word2Vec (Mikolov et al … Cited by 10 Related articles All 3 versions

Spoken language understanding in a nutrition dialogue system MB Korpusik – 2015 – dspace.mit.edu … 1.3 A Nutrition Dialogue System . . . . . 1.3.1 Previous Work . . . . . … 34 4-1 20 nearest neighbors of “bowl” and “cheese,” using 300-dimension word2vec embeddings trained on the Google News corpus and reduced to two dimensions through t-SNE. . . . . … Cited by 3 Related articles All 3 versions

Efficient learning for spoken language understanding tasks with word embedding based pre-training Y Luan, S Watanabe, B Harsham – Sixteenth Annual Conference of the …, 2015 – Citeseer … 4. Experiments We built and tested models using data from a Japanese route guidance spoken dialog system. We used a 1.8G Japanese web text corpus from the Internet to train word2vec, and the full Japanese Wikipedia (900k documents) to train LDA. … Cited by 4 Related articles All 8 versions

Hierarchical neural network generative models for movie dialogues IV Serban, A Sordoni, Y Bengio, A Courville… – arXiv preprint arXiv: …, 2015 – arxiv.org … They are close to human spoken language (Forchini, 2009), which makes them suitable for bootstrapping goal-driven dialogue systems. 2 Models … 1Note that the bidirectional RNN is always one utterance behind the decoder RNN. 2http://code.google.com/p/word2vec/ Page 5. … Cited by 16 Related articles All 4 versions

Online adaptative zero-shot learning spoken language understanding using word-embedding E Ferreira, B Jabaian, F Lefèvre – 2015 IEEE International …, 2015 – ieeexplore.ieee.org … In dialogue systems, the Spoken Language Understanding (SLU) module extracts a list of semantic concept hypothe- ses from an input sentence transcription of the user’s query. … To define the semantic space, the word2vec [16] word- embedding model is considered. … Cited by 6 Related articles All 2 versions

Conversational knowledge teaching agent that uses a knowledge base K LEE, PH SEO, J CHOI, S KOO, GG LEE – 16th Annual Meeting of the …, 2015 – aclweb.org … agent for knowledge education more tightly integrated into QA systems and dialog systems. 3 http://isoft. postech. ac. kr/~ kyusonglee/sigdial/p. emb. vec Table 3: Ranked Results of the top 5 entities gen- erated for Bill Gates Rank Human Proposed Word2Vec 1 Microsoft … Cited by 1 Related articles All 9 versions

Humor Utterance Generation for Non-task-oriented Dialogue Systems S Fujikura, Y Ogawa, H Kikuchi – … of the 3rd International Conference on …, 2015 – dl.acm.org … CONCLUSION We proposed a humor generation method to increase user “desire of continuing dialogue”.The proposed method uses linguistic humor to adapt a variety of dialogue systems. In addition, the proposed method uses word similarity using word2vec which constructs … Related articles

Adopting Semantic Similarity for Utterance Candidates Discovery from Human-to-Human Dialogue Corpus RY Shtykh, M Makita – International Workshop on Future and Emergent …, 2015 – Springer … This is rather a conventional approach for corpus-based dialogue systems and it shows decent results. … To increase the quality of utterances at the top of the retrieved candidates’ list, we use word2vec (with skip-gram model) for a semantic similarity measure. … Related articles All 4 versions

Context Sensitive Spoken Language Understanding using Role Dependent LSTM layers H Chiori, T Hori, S Watanabe, JR Hershey – 2015 – pdfs.semanticscholar.org … for LR w/ word2vec, 78.8% for context sensitive LSTMs, and 84.0% for role dependent LSTMs. We confirmed significant improvement by using context sensitive role dependent LSTMs. 1 Introduction Spoken language understanding (SLU) methods are used in dialog systems to … Cited by 1 Related articles All 3 versions

Dialogue State Tracking using Long Short Term Memory Neural Networks K Yoshino, T Hiraoka, G Neubig, S Nakamura – phontron.com … This change expands the variety of expressions of users, because the users will be free from limitations im- posed by dialogue systems. … relieve this spar- sity, we compress the utterance to a 300 dimensional vector by using doc2vec [6]. doc2vec is a variation of word2vec [7], a … Related articles

Unsupervised Learning and Modeling of Knowledge and Intent for Spoken Dialogue Systems YN Chen – target, 2015 – cs.cmu.edu Page 1. Unsupervised Learning and Modeling of Knowledge and Intent for Spoken Dialogue Systems Yun-Nung (Vivian) Chen Ph.D. Thesis Proposal … 69 xiii Page 20. xiv Page 21. 1Introduction 1.1 Spoken Dialogue System … Related articles All 10 versions

Natural Language Dialogue-Future Way of Accessing Information H Li – 2015 – hangli-hl.com … 3 t T WC M = matrix factorization word embedding or word2vec W M Page 34. Recurrent Neural Network (RNN) … Neural Responding Machine (Shang et al., ACL 2015) Page 40. Natural Language Dialogue System – Retrieval based Approach index of messages and responses … Related articles All 2 versions

Evaluation of Question-answering System about Conversational Agent’s Personality H Sugiyama, T Meguro, R Higashinaka – kecl.ntt.co.jp … Since this behavior also appears in con- versational dialogues with a dialogue system, systems must be developed to respond to such questions. … We assume that thesauri like WordNet or word-clustering methods like brown-clustering [17] or word2vec [9] can improve the … Related articles

Driver prediction to improve interaction with in-vehicle HMI B Harsham, S Watanabe, A Esenther, J Hershey… – 2015 – merl.com … 3.3. Experiments We built and tested models using data from a Japanese rou- te guidance spoken dialog system. We used a 1.8G Japane- se web text corpus from the Internet to train word2vec, and the full Japanese Wikipedia (900k documents) to train LDA. … Cited by 1 Related articles All 6 versions

Convolutional Neural Networks for Multi-topic Dialog State Tracking H Shi, T Ushio, M Endo, K Yamagami, N Horii – colips.org … The details of these features are: • fw1 : a 50-dimensional word vector representation trained on text8 corpus (the first 100MB of the cleaned English Wiki corpus), using word2vec [8]. • fw2 : similar to fw1 but trained … Data-Driven Methods for Adaptive Spoken Dialogue Systems. …

Dialog Management with Deep Neural Networks L Zilka – pdfs.semanticscholar.org … Our model fits in the big picture of a spoken dialog system described in subsection 2.1 as a spoken language understanding and … 5 Informal experiments with different types of initialisaion of word embedings, such as using word2vec (Mikolov et al., 2013) embedings estimated … Related articles All 3 versions

Is it time to switch to Word Embedding and Recurrent Neural Networks for Spoken Language Understanding? V Vukotic, C Raymond, G Gravier – InterSpeech, 2015 – hal.inria.fr … MEDIA The research project MEDIA [8] evaluates different SLU mod- els of spoken dialogue systems dedicated to provide tourist in- formation. … To build the numeric representations, we used the word2vec model [10] trained on the training corpus where words belong- ing to an … Cited by 13 Related articles All 11 versions

A Model of Zero-Shot Learning of Spoken Language Understanding M Yazdani, J Henderson – aclweb.org … Dialogue systems often use hand-crafted gram- mars for SLU, such as Phoenix (Ward, 1994), which are expensive to develop, and expensive to extend or adapt to new attributes and values. … In this work we use the word2vec software of Mikolov et al. … Related articles All 10 versions

A robust spoken Q&A system with scarce in-domain resources LF D’Haro, S Kim, RE Banchs – 2015 Asia-Pacific Signal and …, 2015 – ieeexplore.ieee.org … Semantic Similarity: Weighted sum of semantic similarities computed based on word embeddings generated by Word2Vec. … “IRIS: a chat-oriented dialogue system based on the vector space model,” in Proceedings of the ACL 2012 System Demonstrations. … Related articles All 2 versions

Discourse Relation Recognition by Comparing Various Units of Sentence Expression with Recursive Neural Network A Otsuka, T Hirano, C Miyazaki, R Masumura… – 2015 – aclweb.org … Here, word vectors are given by the word2vec model created using Japanese Wikipedia data. … Our future work is to enable more feature selec- tion using intermediate expression vectors and to consider applications for dialogue systems. … Related articles All 8 versions

Keyword question answering system with report generation for linked data S Han, H Shim, B Kim, S Park, S Ryu… – … Conference on Big …, 2015 – ieeexplore.ieee.org … interpretation of the keyword input query based on human judgment 3 https://code.google.com/ p/word2vec/ 25 … Research Foundation of Korean (NRF) [NRF-2014R1A2A1A01003041, Development of multi-party anticipatory knowledge-intensive natural language dialog system]. … Cited by 1 Related articles All 2 versions

Computing Semantic Textual Similarity Based on Partial Textual Entailment M V?ta – researchgate.net … art: it describes the keystones of our approach – recognizing textual entailment, recognizing partial textual entailment and word2vec model. … Textual Entail- ment Challenge at SemEval-2013 Task 7. They were inspired by developments of tutorial dialogue systems (Dzikovska et … Related articles

Compilation and evaluation of paraphrase representation list of compound verbs: Toward development of “Control language for action” T Shirai, K Kanzaki, H Yabumoto… – … : Concepts, Theory and …, 2015 – ieeexplore.ieee.org … Keywords—Word2Vec; Compound Verb; Semantic Similarity; Control Language … Therefore, we started developing a practical Japanese-based controlled language for man-machine communication, aiming to verify its usefulness through experiments with the dialogue system. … Related articles

Word embeddings combination and neural networks for robustness in asr error detection S Ghannay, Y Esteve, N Camelin – … (EUSIPCO), 2015 23rd …, 2015 – ieeexplore.ieee.org … word2vec [7]: we used the model based on the analysis of continuous bag-of-words (CBOW). … De Mori, “Integration of word and semantic features for theme identification in telephone conversations,” in 6th International Work- shop on Spoken Dialog Systems (IWSDS 2015 … Cited by 5 Related articles All 3 versions

Automatic ranking of swear words using word embeddings and pseudo-relevance feedback LF D’Haro, RE Banchs – 2015 Asia-Pacific Signal and …, 2015 – ieeexplore.ieee.org … Since the combination of both kind of 8 Available at https://code.google.com/p/word2vec/ … [9] Banchs, Rafael E., and Haizhou Li. “IRIS: a chat-oriented dialogue system based on the vector space model.” Proceedings of the ACL 2012 System Demonstrations. … Related articles All 2 versions

Box: Natural Language Processing Research Using Amazon Web Services A Axelrod – The Prague Bulletin of Mathematical Linguistics, 2015 – degruyter.com … Mikolov et al., 2010), and word2vec (Mikolov et al., 2013), with the intent to expand the list shortly. There are many open- source tools that each perform one NLP-related task well, such as speech recognition, optical character recognition, text-to-speech, dialog systems, and so … Cited by 1 Related articles All 5 versions

A critical review of recurrent neural networks for sequence learning ZC Lipton, J Berkowitz, C Elkan – arXiv preprint arXiv:1506.00019, 2015 – arxiv.org … convincingly engage in dialogue [Turing, 1950]. Besides dialogue systems, modern interactive systems of economic importance include self-driving cars and robotic surgery, among others. Without an explicit model of sequentiality … Cited by 28 Related articles All 10 versions

Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretability E Agirrea, C Baneab, C Cardiec, D Cerd… – Proceedings of the 9th …, 2015 – aclweb.org … pairs of sen- tences from news headlines (HDL), image descrip- tions (Images), answer pairs from a tutorial dialogue system (Answers-student … 0.5822 3 SimCompass prefix 0.8360 0.5834 0.7474 0.5338 8 0.8361 0.4708 0.7269 0.4157 12 SimCompass word2vec 0.8716 0.5806 … Cited by 69 Related articles All 12 versions

Machine comprehension with syntax, frames, and semantics H Wang, MBKGD McAllester – Volume 2: Short Papers, 2015 – anthology.aclweb.org … com/p/word2vec/ 6All accuracies are computed with tie-breaking partial credit (similar to previous work), ie, if we have the same to the best feature set combination … Unsupervised induction and fill- ing of semantic slots for spoken dialogue systems using frame-semantic parsing. … Cited by 8 Related articles All 11 versions

Negative Emotion Recognition in Spoken Dialogs X Zhang, H Wang, L Li, M Zhao, Q Li – Chinese Computational Linguistics …, 2015 – Springer … First, word embeddings were trained by word2vec 3 on the combination of three corpus, namely Chinese Gigaword 4 , Chinese Wikipedia 5 , and SougouCA 6 … Liscombe, J., Riccardi, G., Hakkani-Tür, D.: Using context to improve emotion detection in spoken dialog systems. … Related articles All 2 versions

New transfer learning techniques for disparate label sets YB Kim, K Stratos, R Sarikaya, M Jeong – ACL. Association for …, 2015 – aclweb.org … label. How- ever, CCA’s low-dimensional projection is com- putationally more convenient and arguably more generalizable. One can also consider training a predictive model similar to word2vec (Mikolov 477 Page 6. Figure … Cited by 9 Related articles All 9 versions

Negative Emotion Recognition in Spoken Dialogs Q Li – … Linguistics and Natural Language Processing Based …, 2015 – books.google.com … org/ 3 https://code. google. com/p/word2vec/ 4https://catalog. ldc. upenn. … IEEE Trans. Speech Audio Process. 13 (2), 293–303 (2005) 12. Liscombe, J., Riccardi, G., Hakkani-Tür, D.: Using context to improve emotion detection in spoken dialog systems. In: Eurospeech (2005) 13. … Related articles

Learning to understand phrases by embedding the dictionary F Hill, K Cho, A Korhonen, Y Bengio – arXiv preprint arXiv:1504.00548, 2015 – arxiv.org … Prior to training these NLMs, we learn target lexi- cal representations by training the Word2Vec soft- ware (Mikolov et al., 2013) on billions of words of raw text. … 2The Word2Vec embedding models are well known; further details can be found … Cited by 13 Related articles All 10 versions

Which ASR errors are hard to detect S Ghannay, N Camelin, Y Esteve – Errors by Humans and …, 2015 – errare2015.racai.ro … of three different continuous word embeddings: a variant of the Collobert and Wetson word embeddings [5], word2vec [8] and … and semantic features for theme identification in telephone conversa- tions,” in 6th International Workshop on Spoken Dialog Systems (IWSDS 2015 … Cited by 2 Related articles All 2 versions

Deep Reinforcement Learning with an Action Space Defined by Natural Language J He, J Chen, X He, J Gao, L Li… – arXiv preprint arXiv …, 2015 – pdfs.semanticscholar.org … term rewards. network architecture that is trained jointly with multitask learning. Mikolov et al. (2013) introduced word2vec, which is an efficient estimation of continuous vector representations of words. They further explored … Cited by 4 Related articles

Topics, Trends, and Resources in Natural Language Processing (NLP) M Bansal – Citeseer Page 1. Topics, Trends, and Resources in Natural Language Processing (NLP) Mohit Bansal TTI-Chicago (CSC2523, ‘Visual Recognition with Text’, UToronto, Winter 2015 – 01/21/2015) (various slides adapted/borrowed from Dan Klein’s and Chris Manning’s course slides) … Related articles All 2 versions

Structured Vectors for Chinese Word Representations C Li, B Xu, X Wang, G Wu, G Tian… – International Journal of …, 2015 – search.proquest.com … Since we had. 2http://baike.baidu.com /. 3http://www.ictclas.org/. 4http://word2vec. googlecode.com/svn/trunk/. … His current research interests include spoken dialogue systems, dialogue management, reinforcement learning and deep learning. … Related articles All 3 versions

Applying deep learning to answer selection: A study and an open task M Feng, B Xiang, MR Glass, L Wang… – 2015 IEEE Workshop …, 2015 – ieeexplore.ieee.org … 1. INTRODUCTION Natural language understanding based spoken dialog system has been a popular topic in the past years of artificial intelli- gence renaissance. … The word embed- ding (100 dimensions) is trained by word2vec [2] and used for initialization. … Cited by 14 Related articles All 4 versions

Weakly Supervised Natural Language Processing Framework for Abstractive Multi-Document Summarization: Weakly Supervised Abstractive Multi-Document … P Li, W Cai, H Huang – Proceedings of the 24th ACM International on …, 2015 – dl.acm.org … 4.1 Construct Pattern Feature Vector We use word2vec 7 as the word embedding resources. It is gen- erated by learning a recurrent neural network [19]. … Word2vec contains pre-trained vectors trained on part of 7https://code.google.com/p/word2vec … Related articles

Sarcasm Detection in Social Media A Signhaniya, G Shenoy, R Kondekar – rohitkondekar.github.io … Detection of sarcasm if important as it in turn will help in building better sentiment analyzers for review summarization, dialogue systems and review … We use the popular open Word2Vec[1] Library to estimate the distributed word vector represen- tations for all words in the tweets. … Related articles

Solving verbal comprehension questions in IQ test by Knowledge-Powered word embedding H Wang, F Tian, B Gao, J Bian, TY Liu – arXiv preprint arXiv:1505.07909, 2015 – arxiv.org … between pair (A, B) and pair (C, D). Such questions test the abil- ity of identifying an implicit relation from word pair (A, B) and apply it to compose word pair (C, D). Note that the Analogy-I questions are also used as a major evaluation task in the word2vec models [Mikolov et al … Cited by 7 Related articles All 6 versions

Recurrent Neural Networks in Speech Disfluency Detection and Punctuation Prediction M Reisser – 2015 – isl.anthropomatik.kit.edu … in- creasingly relevant. These applications, such as automated machine transla- tion systems, dialogue systems or information extraction systems, usually are trained on large amount of text corpora. Since acquiring, manually …

Measuring Semantic Relatedness using Mined Semantic Analysis W Shalaby, W Zadrozny – arXiv preprint arXiv:1512.03465, 2015 – arxiv.org … linguistics Lemmatisation Natural language processing Indigenous Tweets Internet linguistics Statistical semantics Grammar induction Treebank Dialog systems Light verb … 7] and LDA [10], or more recently through neural embeddings like CW vectors [11], Word2Vec [12], and … Cited by 1 Related articles All 3 versions

Spoken Term Detection and Spoken Word Sense Induction on Noisy Data J Chiu – 2015 – cs.cmu.edu … 11 2.1.1 Word Recurrence in Dialogue Systems . . . . . … 2.1.1 Word Recurrence in Dialogue Systems (Barnett, 1973) propose the “Thematic Memory” as the content-word equivalent of the user-state syntax model. … Related articles