## Skipgrams, Deep Learning & Question Answering 2017

Notes:

The continuous Skip-gram algorithm is an efficient deep learning method for learning high-quality distributed vector representations that capture a large number of precise semantic word relationships.

Between 2014 and 2017, the number of academic papers using skip-grams for deep learning has sky-rocketed.

Resources:

• bioasq .. organizes challenges on biomedical semantic indexing and question answering
• computational stylistics group .. suite of stylometric tools
• ijcai.org .. international joint conference on artificial intelligence (melbourne, august 2017)
• movieqa .. dataset which aims to evaluate automatic story comprehension

Wikipedia:

Recent trends in deep learning based natural language processing
T Young, D Hazarika, S Poria, E Cambria – arXiv preprint arXiv …, 2017 – arxiv.org
… it across a window of size k. On the other hand, the skip-gram model does … Thus, works employing deep learning applica- tions on such languages tend to prefer character … NLP tasks such as sentiment analysis, summarization, machine translation, and question answering (QA) …

Learning to rank question answer pairs with holographic dual LSTM architecture
Y Tay, MC Phan, LA Tuan, SC Hui – arXiv preprint arXiv:1707.06372, 2017 – arxiv.org
… tiveness of our model on both factoid question answering and community-based question answering … erefore, we initializeW with pretrained SkipGram [11] embeddings which is aligned with … In this section, we introduce our novel compositional deep learning model for modeling …

Report on the sigir 2016 workshop on neural information retrieval (neu-ir)
N Craswell, WB Croft, J Guo, B Mitra, M de Rijke – ACM Sigir forum, 2017 – dl.acm.org
… The accepted papers also covered a broad range of tasks, including question/answering [34], proactive IR … Some attendees posited that progress in the area of deep learning may push forward less … An empirical study on large scale text classification with skip-gram embeddings …

Semantically enhanced software traceability using deep learning techniques
J Guo, J Cheng, J Cleland-Huang – Proceedings of the 39th International …, 2017 – dl.acm.org
… Language Processing (NLP) tasks including parsing [63], sentiment analysis [66], question answering [35], and … one of the primary reasons for the success of recent deep learning models for … Skip-gram with negative sampling [48], [50] and GloVe [53] are the most popular word …

Unsupervised and scalable algorithm for learning node representations
T Pimentel, A Veloso, N Ziviani – 2017 – openreview.net
… Representation learning is one of the foundations of Deep Learning and allowed big … on several Machine Learning fields, such as Neural Machine Translation, Question Answering and Speech … This method is based on SkipGram and uses nodes’ neighborhoods as contexts to …

Deep Learning approach for sentiment analysis of short texts
A Hassan, A Mahmood – Control, Automation and Robotics …, 2017 – ieeexplore.ieee.org
… level embedding using the word2vec [2 ], which implemented the continuous bag-of-words and SkIP gram architectures for … that unsupervised pre-trained of word vectors is a significant feature in deep learning for NLP … Semantic Parsing for Single-Relation Question Answering …

Modeling and learning distributed word representation with metadata for question retrieval
G Zhou, JX Huang – IEEE Transactions on Knowledge and Data …, 2017 – ieeexplore.ieee.org
… recursive neural network (RNN) based on dependency tree for factoid question answering, by mapping … Recently, a series of works applied deep learning techniques to learn high-quality … models for learning word representations, includ- ing the continuous skip-gram model and …

Biometrics and forensics integration using deep multi-modal semantic alignment and joint embedding
AS Toor, H Wechsler – Pattern Recognition Letters, 2017 – Elsevier
… This paper proposes collaborative and context-aware visual question answering (C2VQA) for multi-modal information … Test (VTT), engages deep semantic alignment and joint embedding using deep learning (DL) for image analysis, vector space as skip-grams and long-term …

D Bonadiman, A Uva, A Moschitti – … of the 15th Conference of the …, 2017 – aclweb.org
… Neural Networks: we mapped words to embed- dings of size 50, pre-initializing them with stan- dard skipgram embeddings of dimensionality 50 … 2014. Deep learning for answer sentence selection … Question retrieval with high quality answers in community question answering …

Delving deep into personal photo and video search
L Jiang, Y Kalantidis, L Cao, S Farfade, J Tang… – Proceedings of the …, 2017 – dl.acm.org
… In recent years, deep learning based image classifi- cation [30] has been employed for media auto-tagging; pho- tos can automatically … personal media search, but also benefit a variety of related tasks such as personal photo browsing, recommendation, and question answering …

SwissAlps at SemEval-2017 task 3: Attention-based convolutional neural network for community question answering
JM Deriu, M Cieliebak – Proceedings of the 11th International Workshop …, 2017 – aclweb.org
… York’. The word embeddings are gener- ated using the skip-gram model setting the context window to 5 and the dimensionality to d = 200 … set. 4 Conclusion We described a deep learning approach to question-answering. The …

Query Answering to IQ Test Questions Using Word Embedding
M Fr?ckowiak, J Dutkiewicz, C J?drzejek… – Multimedia and Network …, 2017 – Springer
… The results obtained in [26] created a lot of publicity under a controversial title “Deep learning machine beats … The starting point of the [26] analysis is the skip-gram method in word2vec [11]. Skip grams are word windows from which one word is excluded, an n-gram with gap …

MappSent: a Textual Mapping Approach for Question-to-Question Similarity
A Hazem, BEA Boussaha, N Hernandez – Proceedings of the …, 2017 – acl-bg.org
… of neural networks, it has been noticed a jump in 2017 on the number of deep learning methods (Nakov … 1. We train a Skip-Gram 5 model using Gen- sim ( ?Rehurek and Sojka, 2010)6 on a … experienced but it turned out to give lower results while compared to the SkipGram model …

Y Xie, M Wang, J Ma, J Jiang, Z Lu – Proceedings of the 11th …, 2017 – aclweb.org
… forums with vector size of 100, window size 10, minimum words frequency of 5, and skip-gram 1. LDA … 2016. Ecnu at semeval- 2016 task 3: Exploring traditional method and deep learning method for question retrieval and answer ranking in community question answering …

Product function need recognition via semi-supervised attention network
H Xu, S Xie, L Shu, PS Yu – arXiv preprint arXiv:1712.02186, 2017 – arxiv.org
… Keywords-product function need recognition; semi- supervised learning; deep learning; attention … product owners or sellers regarding product functions via Product Community Question Answering (PCQA for … We pre-train the word embedding via skip-gram model [5]. Then we …

Unsupervised learning of sentence embeddings using compositional n-gram features
M Pagliardini, P Gupta, M Jaggi – arXiv preprint arXiv:1703.02507, 2017 – arxiv.org
… one hand, a trend in deep-learning for NLP moves towards increasingly powerful and complex models, such as recur- rent neural networks (RNNs), LSTMs, attention models and even Neural Turing Machine architectures … Skipgram 69.3 / 77.2 73.6 77.3 89.2 85.0 82.2 …

A Deep Learning Approach for Expert Identification in Question Answering Communities
C Zheng, S Zhai, Z Zhang – arXiv preprint arXiv:1711.05350, 2017 – arxiv.org
… Word2Vec provides an efficient implementation of the CBOW architecture and skip-gram architectures for comput … by a local window to predict a (known) center word, and skipgram works exactly … address a good way to solve expert identification problem by deep learning method …

A Study of Word Embeddings for Biomedical Question Answering
S Kamath, B Grau, Y Ma – 4e édition du Symposium sur l’ …, 2017 – hal.archives-ouvertes.fr
… Biomedical Question Answering … know, BIOASQ dataset is the only dataset for biomedical QA, whose size is certainly small to train deep learning based models … which is trained on open domain data (Wikipedia) performs with the best scores and also the Skipgram model with …

MappSent at IJCNLP-2017 Task 5: A Textual Similarity Approach Applied to Multi-choice Question Answering in Examinations
A Hazem – Proceedings of the IJCNLP 2017, Shared Tasks, 2017 – aclweb.org
… However, with the boom of neural networks, various deep learning approaches ranging from a word level embedding … 1. We train a Skip-Gram 3 model using Gensim ( ?Rehurek and Sojka, 2010)4 on a … but it turned out to give lower results while compared to the SkipGram model …

Automatic Question-Answering Using A Deep Similarity Neural Network
S Minaee, Z Liu – arXiv preprint arXiv:1708.01713, 2017 – arxiv.org
… Fig. 2. The block-diagram of the CBOW and Skip-gram models [15] … V. CONCLUSION In this work, we propose a deep learning framework for question-answering, with application to customer-case service automation …

Investigating Stroke-Level Information for Learning Chinese Word Embeddings
S Cao, W Lu, J Zhou, X Li – ceur-ws.org
… consistently outperforms several state-of-the-art methods, in- cluding skipgram, cbow, GloVe and … With the extensive development of deep learning, word (or concept) embed- dings are more and more widely used in entity linking [12], question-answering [11], knowledge …

Learning Word Vectors in Deep Walk Using Convolution
I Chaturvedi, S Cavallari, E Cambria, V Zheng – 2017 – pdfs.semanticscholar.org
… We learn low-dimensional la- tent embeddings ?ik from random walk sequences using skip-gram model … Community-based question answering via hetero- geneous social network learning … Deep learning based document modeling for person- ality detection from text …

An Enhanced Convolutional Neural Network Model for Answer Selection
J Guo, B Yue, G Xu, Z Yang, JM Wei – Proceedings of the 26th …, 2017 – dl.acm.org
… Take the sentence the cat sat on the mat for exam- ple, the skip-grams, ie, “cat sat on mat” and “cat on the 789 Page 2 … Wikiqa: A challenge dataset for open-domain question answering. In EMNLP, pages 2013–2018, 2015 … Deep learning for answer sentence selection …

Question Retrieval in Community Question Answering Enhanced by Tags Information in a Deep Neural Network Framework
C ZAVOU – 2017 – esc.fnwi.uva.nl
… The research for answer selection is even wider and longer; one recent deep learning approach is represented in [25]. Traditional question retrieval, and in general, question answering research, concentrates mostly on factoid questions …

Question answering system based on sentence similarity
M Kashif, C Arora – 2017 – repository.iiitd.edu.in
… Such systems can be used in a wide variety of applications ranging from Open domain question answering or closed domain question answering which involves searching for … Next, we explore Siamese Deep Learning Network to perform sentence matching … 4.2 Skip-gram Model …

Ranking Kernels for Structures and Embeddings: A Hybrid Preference and Classification Model
K Tymoshenko, D Bonadiman, A Moschitti – Proceedings of the 2017 …, 2017 – aclweb.org
… Recent work on learning to rank (L2R) has shown that deep learning and kernel methods are … In factoid question answering, Q typically asks for an entity or a fact, eg, time location … Tymoshenko et al., 2016a): we pre-initialize the word embeddings with skipgram embedding of …

Improved Answer Selection with Pre-Trained Word Embeddings
R Chakravarti, J Navratil, CN Santos – arXiv preprint arXiv:1708.04326, 2017 – arxiv.org
… KEYWORDS word embeddings, rank, re-rank, question answering, answer se- lection … with comparisons to traditional IR as well as a state-of-the-art deep learning system … 6] trained on the following term coverage based fea- tures: Unigram, Bigram, Skipgram, SloppyBigram, Solr …

On Usage Of Machine Learning For Natural Language Processing Tasks As Illustrated By Educational Content Mining
AV Melnikov, DS Botov, JD Klenin – 2017 – cyberleninka.ru
… In natural language question-answering task a hidden variable perceptron was used to filter incorrect relations, extracted from the knowledge … in Paragraph2Vec is called Distributed Bag-of-Words (PV-DBOW) and is similar to the skip-gram model in … 4 Deep learning in NLP tasks …

Distributed Representation for Traditional Chinese Medicine Herb via Deep Learning Models
W Li, Z Yang – arXiv preprint arXiv:1711.01701, 2017 – arxiv.org
… 2014; Sunetal., 2017), text summarization (MaandSun, 2017; Maetal., 2017) question answering (Rajpurkar et al … the TCM diagnosis procedure, Wang (2013) proposed to use deep learning and CRF … yet effective models called continuous bag of words (CBOW) and skip gram …

Question Recommendation in Medical Community-Based Question Answering
H Cai, C Yan, A Yin, X Zhao – International Conference on Neural …, 2017 – Springer
… We use Word2vec and skip-gram algorithm [15] with negative sampling to training word vector … W., Wu, H., et al.: Question retrieval with high quality answers in community question answering … Bian, J., Gao, B., Liu, T.-Y.: Knowledge-powered deep learning for word embedding …

Attention-based Natural Language Person Retrieval
T Zhou, M Chen, J Yu… – arXiv preprint arXiv …, 2017 – openaccess.thecvf.com
… In contrast to previous works, we develop a deep learning framework to learn a scoring … [5] analyzed the consis- tency between human and deep network attention in visual question answering … As a requisite of natural language processing, the Skip- gram model [22] is used to …

Multi-Modal Information Extraction in a Question-Answer Framework
V Chitters, F Frank, L Jebe – 2017 – cs229.stanford.edu
… 13] is one of the most com- monly used datasets for natural language question answering … As for the image mode, literature review includes both non-deep and deep learning models (which … works as follows: 1. Train a Word2vec [11] word embedding (skip-gram archi- tecture …

End-to-end Learning for Short Text Expansion
J Tang, Y Wang, K Zheng, Q Mei – Proceedings of the 23rd ACM SIGKDD …, 2017 – dl.acm.org
… vised text embedding model showing comparable performance to more sophisticated deep learning models [12 … Speci cally, we use Skip-gram, LINE, and PTE to learn word embed- dings, and … with only short text features, the unsupervised approaches BOW, Skipgram,and LINE …

Chinese Question Classification Based on Semantic Joint
X Li, HF Liu, SY Jiang – tcci.ccf.org.cn
… The structure of skip-gram model includes three layers: input layer, projection layer and output … features extracted by our method is interpreta- tive than that of deep learning methods … HY, Hu CL: On question classification in an ontology-based Chinese question-answering System …

D Teney, Q Wu… – IEEE Signal Processing …, 2017 – ieeexplore.ieee.org
Page 1. 63 IEEE SIgnal ProcESSIng MagazInE | November 2017 | DEEP LEARNING FOR VISUAL UNDERSTANDING 1053-5888/17©2017IEEE Damien Teney, Qi Wu, and Anton van den Hengel Visual Question Answering A tutorial …

Convolutional neural networks for expert recommendation in community question answering
J Wang, J Sun, H Lin, H Dong, S Zhang – Science China Information …, 2017 – Springer
… They trained words distributed representation through the skip-gram model. LR … 3111–3119 13 LeCun Y, Bengio Y, Hinton G. Deep learning … 1746–1751 16 Pal A, Chang S, Konstan J A. Evolution of experts in question answering communities …

A Jain, F Wasim – pdfs.semanticscholar.org
… The purpose of our work is to explore different state of art deep learning architectures, which … Models like Skip gram[9] and CBOW[10] tries to capture co-occurrence of words … faster training GloVe are scalable and captures semantic relationships just like Skip Grams and CBOW …

Chinese Question Classification Based on Semantic Joint Features
X Li, HF Liu, SY Jiang – National CCF Conference on Natural Language …, 2017 – Springer
… Automatic question-answering system includes question analysis, information retrieval and answer extraction [1 … Although there are some deep learning based methods like convolutional neural network models … In this paper, we use the skip-gram model proposed by Mikolov [15 …

RA Islam – 2017 – uleth.ca
… 10 2.6 Skip-gram model (Mikolov et al., 2013a) … In chapter 2, we describe some research works related to the tasks in the field of “Community Question Answering”. We briefly explain the deep learning techniques. We mainly used these techniques for solving our tasks …

CS224n: Natural Language Processing with Deep Learning1
C Manning, R Socher, GG Fang, R Mundra – 2017 – stanford.edu
… Singu- lar Value Decomposition. Skip-gram. Continuous Bag of Words (CBOW) … Page 2. cs224n: natural language processing with deep learning 2 … Coreference (eg What does “he” or “it” refer to given a docu- ment?) • Question Answering (eg Answering Jeopardy questions) …

Learning Contextual Embeddings for Structural Semantic Similarity using Categorical Information
M Nicosia, A Moschitti – Proceedings of the 21st Conference on …, 2017 – aclweb.org
… effectiveness on a plethora of NLP tasks such as Question Answering (QA) (Severyn … Deep learning models (DLMs) usually do not include traditional features; they extract relevant … extensive comparison between different methods, ie, word2vec, using CBOW and SkipGram, and …

Helping Users Understand Privacy Notices with Automated Query Answering Functionality: An Exploratory Study
KM Sathyendra, A Ravichander, PG Story… – 2017 – reports-archive.adm.cs.cmu.edu
… This work uses deep learning algorithms with 21 different classifiers to build a question answering … In this section, we describe two models to perform open question-answering., We also … We trained the word vectors using the Word2Vec skip-gram model [27] using Gensim[36] …

Soft Seeded SSL Graphs for Unsupervised Semantic Similarity-based Retrieval
A Srivastava, M Datt – Proceedings of the 2017 ACM on Conference on …, 2017 – dl.acm.org
… feature vectors, and pro- duces state-of-the-art performance on question-answering tasks … significantly improve semantic retrieval when used in conjunction with latent deep learning based representations … we use word overlap features, common n-grams and skip-grams (up to …

Deep learning for Arabic NLP: A survey
M Al-Ayyoub, A Nuseir, K Alsmearat, Y Jararweh… – Journal of …, 2017 – Elsevier
… Cover image Cover image. Deep learning for Arabic NLP: A survey … Abstract. The recent advances in deep learning (DL) have caused breakthroughs in many fields such as computer vision, natural language processing (NLP) and speech processing …

Learning Chinese Word Representations From Glyphs Of Characters
TR Su, HY Lee – arXiv preprint arXiv:1708.04755, 2017 – arxiv.org
… processing tasks, for example, sentence classi- fication (Kim, 2014), question answering (Zhou et … CBOW predicts the word given its context embeddings, while Skip- gram predicts contexts given … Predicting the occurrence of word/context in CBOW and Skipgram models could be …

Bilinear Joint Learning of Word and Entity Embeddings for Entity Linking
H Chen, B Wei, Y Liu, Y Li, J Yu, W Zhu – Neurocomputing, 2017 – Elsevier
… a crucial technique to discover knowledge in texts and would facilitate different applications such as information retrieval, question answering, and knowledge … BJLM extends the skip-gram [2] framework by injecting a bilinear model to simulate the interaction between words and …

Tree LSTMs for Learning Sentence Representations
H Cerezo-Costas, G AlantTic, M Mart?n-Vicente – pdfs.semanticscholar.org
… We compare the performance of our recursive Tree-LSTMs against other deep learning models: a recurrent version … This concept is similar to the learning of word embeddings with the skipgram model (Mikolov … A Neural Network for Factoid Question Answering over Paragraphs …

If You Can’t Beat Them Join Them: Handcrafted Features Complement Neural Nets for Non-Factoid Answer Reranking
D Bogdanova, J Foster, D Dzendzik, Q Liu – Proceedings of the 15th …, 2017 – aclweb.org
… ment distributed lexical semantic information ob- tained with a skip-gram model (Mikolov … et al., 2015) for answer sentence selection for factoid question answering and models … Recently, the Wide and Deep learning model for recommendation systems was proposed (Cheng et al …

Improving relational aggregated search from big data sources using deep learning
S Achsas, EH Nfaoui – 2017 Intelligent Systems and Computer …, 2017 – ieeexplore.ieee.org
… in several IR tasks like query reformulation, text matching or question answering, it has … characteristics automatically using stacked autoencoders as a model of unsupervised deep learning algorithms … Paragraph Vector) ? This model works in the same way as skip-gram model …

Deep Memory Networks for Natural Conversations
??? – 2017 – s-space.snu.ac.kr
… Keywords : Attention Model, Memory Network, Deep Learning, Natural Language Understanding, Machine Comprehension … 22 [Figure 3.2] Skip-Gram and TWE models … natural language problem. Most tasks in natural language processing can be cast into question answering …

Sequence Mining and Pattern Analysis in Drilling Reports with Deep Natural Language Processing
J Hoffimann, Y Mao, A Wesley, A Taylor – arXiv preprint arXiv:1712.01476, 2017 – arxiv.org
… the paper does not benefit from the recent advances in NLP and deep learning … for more challenging tasks, such as information retrieval, summarization and question answering [3], [9 … the conditional probabilities: the continuous bag-of-words (CBOW) and the skip-gram models …

Empirical Evaluation of Word Representations on Arabic Sentiment Analysis
M Gridach, H Haddad, H Mulki – International Conference on Arabic …, 2017 – Springer
… applications such as dependency parsing [11], language modeling [12], question answering [13, 14 … In this paper, we present a deep learning model based on convolutional neural … We investigate three main word representations: Stanford Glove vectors, Skip-gram (SG) model …

A Survey on Paraphrase Detection Techniques for Indian Regional Languages
S Srivastava, S Govilkar – International Journal of Computer …, 2017 – search.proquest.com
… classifiers. The features they used skip grams, longest … Identifying paraphrases is an important task that is used in. information retrieval, question answering, text summarization … provides a better result. 5.7 Bi-CNN-MI. It is a deep learning architecture [5] used for English language …

A Case for Deep Learning in Mining Software Repositories
R Nijessen – 2017 – pdfs.semanticscholar.org
… An immensely large amount of research has been done on using deep learning in natural … such as sentiment analysis [33, 34], machine translation [35, 36] and question answering [37, 38] … Word2Vec consists of two models: Continous Bag Of Words (CBOW) and Skip-gram …

Combing Context and Commonsense Knowledge Through Neural Networks for Solving Winograd Schema Problems
Q Liu, H Jiang, ZH Ling, X Zhul, S Wei, Y Hu – 2017 – researchgate.net
… bined, which is equal to the skip-gram models) from Wiki- Small and Wiki-Large … Probabilistic rea- soning via deep learning: Neural association models. arXiv preprint arXiv:1603.07704 … Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv prep

Developments in The Field of Natural Language Processing
B Goel – International Journal of Advanced Research in …, 2017 – search.proquest.com
… (with/without negative sampling). Skip-gram model primarily … lying or not. • Generate keyword tags. • Automated question answering bots such as Siri and. Google Assistant … With advancement in related technologies. such as Cognitive Computing and Deep Learning, NLP will …

Morphological Embeddings for Named Entity Recognition in Morphologically Rich Languages
O Gungor, E Yildiz, S Uskudarli, T Gungor – arXiv preprint arXiv …, 2017 – arxiv.org
… tasks such as re- lation extraction, knowledge base generation, and question answering (Liu and … Recently, deep learning models have been in- strumental in deciding how the parts of the in- put … 1 of Turkish words as vectors of length 100 using the skipgram algorithm (Mikolov …

Towards enhanced hierarchical attention networks in ICD-9 tagging of clinical notes
MJC Samonte, BD Gerardo, RP Medina – Proceedings of the 3rd …, 2017 – dl.acm.org
… includes document classification, single label and multi-label, sentiment analysis, question answering and topic … is a most common framework for associating word vectors Skip-Gram’s goal is … Deep Learning was applied in a research entitled, ?Multi-Task Deep Neural Network …

Challenges and opportunities: from big data to knowledge in AI 2.0
Y Zhuang, F Wu, C Chen, Y Pan – Frontiers of Information Technology & …, 2017 – Springer
… According to the authors’ note, CBOW is faster while skip-gram is slower, but the latter … The authors investigated their approach in the context of question answering (QA), where the long … In recent years, we have witnessed the unprecedented advance in deep learning in various …

A joint deep model of entities and documents for cumulative citation recommendation
L Ma, D Song, L Liao, Y Ni – Cluster Computing, 2017 – Springer
… 2, have been successfully leveraged in various applications including entity linking [1], query expansion [2], knowledge graph [3], question answering [4], entity … This approach is inspired by deep learning successfully applying to various natural language processing applications …

Relation Classification via Target-Concentrated Attention CNNs
J Zhu, J Qiao, X Dai, X Cheng – International Conference on Neural …, 2017 – Springer
… an important role in various scenarios, eg, information extraction [1], question answering [2], knowledge … Additionally, some other deep learning models have been proposed such as [7, 16, 18] … our experiments, we use the publicly available word2vec 1 skip-gram architecture [12 …

Knowledge Acquisition From Text Based On Sentence Rewriting
X Ma, ZW Muran Su, L Zhang – sist.shanghaitech.edu.cn
… Index Terms— Knowledge Acquisition, Sentence Rewrit- ing, Deep Learning, Data Mining … There are still some skip grams, such as “how many people live in” and “what do * do for a … [4] Anthony Fader, Luke Zettlemoyer, and Oren Etzioni, “Open question answering over curated …

Reading between the Lines: Content Aware Deep Multi-Relation Learning
A Banerjee – researchgate.net
… One of Deep Learning based solution would be to use word embedding (which … [19] proposed the skip-thoughts model, which extends the skip-gram approach of … application areas include machine translation (MT), summarization, generation, question answering (QA), short …

Word and Relation Embedding for Sentence Representation
T Rath – 2017 – search.proquest.com
… than the dependency based embedding Levy and Goldberg (2014) and skipgram based embeddings … paragraph vector, which was based on the above mentioned skip-gram model to … to create an internal representation of a sentence directly using deep learning methods like …

SemEval-2017 Task 1: Semantic Textual Similarity-Multilingual and Cross-lingual Focused Evaluation
D Cer, M Diab, E Agirre, I Lopez-Gazpio… – arXiv preprint arXiv …, 2017 – arxiv.org
… understanding problem relevant to numerous applications including: machine trans- lation (MT), summarization, generation, question answering (QA), short … More recently, deep learning became competitive with top performing feature engineered systems (He et al., 2016) …

Towards Accurate Duplicate Bug Retrieval Using Deep Learning Techniques
J Deshmukh, S Podder, S Sengupta… – … and Evolution (ICSME …, 2017 – ieeexplore.ieee.org
… approaches in many tasks like Part of Speech Tagging, Question Answering, Sentiment Analysis … A. Deep Learning Prerequisites 1) Word Embeddings: The first challenge encountered in applying … new models – continuous bag of words (CBOW) and skip-gram model, popularly …

The Importance of Automatic Syntactic Features in Vietnamese Named Entity Recognition
TH Pham, P Le-Hong – arXiv preprint arXiv:1705.10610, 2017 – arxiv.org
… NER is a crucial pre- processing step used in some NLP applications such as information extraction, question answering, auto- matic translation, speech processing, and biomedi- cal science … Page 5. Figure 3: Our deep learning model Entity Skip-Gram Random One-hot Pre …

A comparative study of word embeddings for reading comprehension
B Dhingra, H Liu, R Salakhutdinov… – arXiv preprint arXiv …, 2017 – arxiv.org
… Several RC models based on deep learning have been proposed (Cui et al., 2016; Munkhdalai and Yu, 2016 … Dynamic coattention networks for question answering … For word2vec, we used skip-gram architecture with hierarchical softmax, and sub- sampled frequent words with a …

Classification of radiology reports using neural attention models
B Shin, FH Chokshi, T Lee… – Neural Networks (IJCNN) …, 2017 – ieeexplore.ieee.org
… deep learning techniques to be effective … for image processing, attention mechanism has successfully been adopted in various NLP domains including question answering, sentiment analysis … can be learned by either continuous bag-of-words (CBOW) or skip-gram (SKIP) models …

Deep Collective Inference.
J Moore, J Neville – AAAI, 2017 – aaai.org
… However, despite recent advances in deep learning and the successes of recurrent neural net- works (RNNs), researchers have only just recently begun to study how to apply RNNs to heterogeneous graph and net- work datasets …

Programming bots by synthesizing natural language expressions into API invocations
S Zamanirad, B Benatallah, M Chai Barukh… – Proceedings of the …, 2017 – dl.acm.org
… This back and forth question-answering between user and bot (and vice versa) is referred … We use a deep-learning model called Word Embedding[11] to generate semantically similar keywords … network uses 150 hidden layers; sliding window size of five; skip gram with negative …

Peer-review Form pdf preview for 24hrs, from 0: 00am May 10, 2017 to11: 59pm May 10th
CP Classrooms – pdfs.semanticscholar.org
… Open-domain Question Answering (QA) systems typically leverage an answer selection component to rank … This work contributes to the fields of deep learning, pharmacology and organic chemistry by … the training corpus into semantic tokens and use the Skip-gram model to train …

Enhancing Document-Based Question Answering via Interaction Between Question Words and POS Tags
Z Xie – National CCF Conference on Natural Language …, 2017 – Springer
… Recently, with the upsurge of deep learning, a variety of neural approaches have been proposed to solve … The document-based question answering task is described as follows … obtain the word embedding matrix $$\mathbf {M}_{word}$$ by training the skip-gram neural language …

Review Rating with Joint Classification and Regression Model
J Xu, H Yin, L Zhang, S Li, G Zhou – National CCF Conference on Natural …, 2017 – Springer
… language processing (NLP) tasks, such as machine translation [7], question answering [8] and … However, it is worthy to note that although these deep learning approaches have … For word representation, we employ skip-gram algorithm (gensim 2 implementation) by word2vec to …

Dual Long Short-Term Memory Networks for Sub-Character Representation Learning
H Han, Y Xiaokun, W Lei, Y Hua, G Zhimin, F Yi… – arXiv preprint arXiv …, 2017 – arxiv.org
… Take a brief review of the continuous skip-gram model introduced by Mikolov et al … c. Let c1, …, cT be a large-scale corpus represented as a sequence of characters, the objective function of the skipgram model is to … [11] X. Zheng, H. Chen, and T. Xu, “Deep Learning for Chinese …

Supervised Domain Adaptation for Sentiment Regression
J Xu, H Yin, S Li, G Zhou – Chinese National Conference on Social Media …, 2017 – Springer
… language processing (NLP) tasks, such as machine translation [7] and question answering [8]. It … In this paper, we employ a popular deep learning method, named Long Short … For word representation, we employ skip-gram algorithm (gensim 3 implementation) by word2vec to pre …

mAnI: Movie Amalgamation using Neural Imitation
N Panwar, S Khare, N Gantayat, R Aralikatte… – arXiv preprint arXiv …, 2017 – arxiv.org
… KEYWORDS creative AI, multi-modal learning, deep learning … [11]. A memory network based question answering system is built on a movie corpus using multiple sources of information such as movie plot, movie video … extension of skip-gram model for word embedding learning …

Living a discrete life in a continuous world: Reference with distributed representations
G Boleda, S Padó, NT Pham, M Baroni – arXiv preprint arXiv:1702.01815, 2017 – arxiv.org
… As recently dis- cussed by Boleda and Herbelot (2016), data-driven methods such as distributional semantics and deep learning deal with continuous aspects of meaning very … VQA: Visual Question Answering … Combining language and vision with a multimodal skip-gram model …

Living a discrete life in a continuous world: reference with distributed representations
G Boleda Torrent, S Padó, M Baroni – 2017 – repositori.upf.edu
… As recently dis- cussed by Boleda and Herbelot (2016), data-driven methods such as distributional semantics and deep learning deal with continuous aspects of meaning very … VQA: Visual Question Answering … Combining language and vision with a multimodal skip-gram model …

Rating Prediction Based On Sentimental Analysis Of Reviews From Online Trending Forums
J Abhishek, A Ezhilan, JE Mouli, S Nalini – pdfs.semanticscholar.org
… 1][8]. We show an example of SEWE in Figure 3.The traditional continuous skip-gram predicts surrounding … L. Claudino, R. Socher, and H. DaumeIII, “A neural network for factoid question answering over paragraphs … [10] Y. Bengio, IJ Goodfellow, and A. Courville, Deep Learning …

Synapse at CAp 2017 NER challenge: Fasttext CRF
D Sileo, C Pradel, P Muller, T Van de Cruys – arXiv preprint arXiv …, 2017 – arxiv.org
… uses negative sampling with the pa- rameters described in table 2. A different skipgram model was … Towards deep learning in hindi NER: an approach to tackle the labelled data sparsity … Named Entity Recognition and Question Answering Using Word Vectors and Clustering …

Extracting visual knowledge from the web with multimodal learning
D Gong, DZ Wang – International?? Joint?? Conference?? on?? …, 2017 – static.ijcai.org
… Experimental results based on 46 object categories show that the extraction precision is improved significantly from 73% (with state-of- the-art deep learning programs) to 81%, which is equivalent to a 31% reduction … Kiros et al., 2014] proposed a mul- timodal skip-gram model to …

Improving Word Embeddings with Convolutional Feature Learning and Subword Information.
S Cao, W Lu – AAAI, 2017 – aaai.org
… We showed improvements over existing state- of-the-art methods for learning word embeddings, including skipgram, GloVe, char n-gram and DSSM … 2016) is a variant based on Skipgram model, which enriches word embeddings with character n-gram information …

Semi-supervised vision-language mapping via variational learning
Y Shen, L Zhang, L Shao – Robotics and Automation (ICRA) …, 2017 – ieeexplore.ieee.org
… in image-text based learning such as image-tag or image- sentence retrieval [1]–[12], visual question answering [13]– [15 … Recent advances in deep learning are as well adopted into the proposed model … DeViSE: The original DeViSE [26] is based on the skip- gram text feature …

Modelling semantic relations with distributitional semantics and deep learning: question answering, entailment recognition and paraphrase detection
V Maraev – 2017 – repositorio.ul.pt
… FACULDADE DE MEDICINA FACULDADE DE PSICOLOGIA Modelling Semantic Relations with Distributional Semantics and Deep Learning: Question Answering, Entailment Recognition and Paraphrase Detection Vladislav Maraev Thesis supervised by Prof …

Disambiguating Spatial Prepositions Using Deep Convolutional Networks.
K Hassani, WS Lee – AAAI, 2017 – aaai.org
… machine vision, text- to-scene conversion systems, geographical information systems, question answering systems, search … is con- cerned, this is the first work that applies deep learning models to … models such as continuous Bag-of-Words (CBOW) and skip-gram with negative …

Using Word Embeddings for Search in Linked Data with Ontodia
G Wohlgenannt, N Klimov, D Mouromtsev… – iswc2017.semanticweb.org
… Word embeddings and deep learning techniques are prominently listed as promising techniques for future … from fastText using word morphology together with the word2vec skipgram model, which is … AJ, Endris, KM, Singh, K., Thakkar, H., Lange, C.: Question answering on linked …

The struggle with academic plagiarism: Approaches based on semantic similarity
T Vrbanec, A Meštrovi? – Information and Communication …, 2017 – ieeexplore.ieee.org
… disambiguation, machine translation, question answering, plagiarism detection, etc … Similarly with the previous methods, in deep learning documents or texts can be represented as vectors … method such as LSA; another is context base methods such as skip-grams, a continuous …

Context Aware Document Embedding
Z Zhu, J Hu – arXiv preprint arXiv:1707.01521, 2017 – arxiv.org
… Page 5. Domain skip-gram dbow CA(CNN) CA(GRU) w-dbow(IDF) WIKI STS WIKI STS (WIKI init) STS STS … 2014. Empirical evaluation of gated recurrent neural networks on sequence model- ing. NIPS Deep Learning Workshop …

TextFlow: A Text Similarity Measure based on Continuous Sequences
Y Mrabet, H Kilicoglu, D Demner-Fushman – Proceedings of the 55th …, 2017 – aclweb.org
… in deep learning highlighted further the relevance of sequential mod- els in natural language generation, existing similarity measures do not fully exploit the sequential nature of language. Exam- ples of such similarity measures include n- grams and skip-grams overlap which …

Query Similarity for Community Question Answering System Based on Recurrent Encoder Decoder
B Ye – 2017 – uwspace.uwaterloo.ca
… Database Figure 1.1: Example of a community question answering system … Kiros et al. [28] train their contextual skip-gram word vectors over … They tested their word vector quality on the MSRP task. Other work embed sentence vectors into self-defined deep learning structures …

Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
R Barzilay, MY Kan – Proceedings of the 55th Annual Meeting of the …, 2017 – aclweb.org
… Hopefully, some of it is still relevant now that deep learning has solved language … Many of these have become familiar to mainstream computer users in the from of web search, question answering, sentiment analysis, and notably machine translation …

Neural semantic encoders
T Munkhdalai, H Yu – Proceedings of the conference. Association …, 2017 – ncbi.nlm.nih.gov
… Answer sentence selection is an integral part of the open-domain question answering … for example, to train them to produce encoding memory and representation vector of entire sentences or documents using either new or existing models such as the skip-gram model (Mikolov …

Chinese Named Entity Recognition Based on B-LSTM Neural Network with Additional Features
L Ouyang, Y Tian, H Tang, B Zhang – International Conference on Security …, 2017 – Springer
… from texts and facilitates downstream tasks, such as machine translation and question answering … in entity recognition is meaningful, but relevant classifications are rarely performed in deep learning … Skip-gram is often insufficient in generating special word vectors only in the …

On the Impact of Linguistic Information in Kernel-Based Deep Architectures
D Croce, S Filice, R Basili – Conference of the Italian Association for …, 2017 – Springer
… based Deep Architecture discussed in [10], that efficiently combines kernel methods and Deep Learning by using … a question into a closed set of answer types in a Question Answering system … the SPTK and CSPTK are generated using the Word2vec tool with a Skip-gram model …

Living a discrete life in a continuous world: Reference in cross-modal entity tracking
G Boleda, S Padó, M Baroni – IWCS 2017—12th International …, 2017 – aclweb.org
… DIRE uses the concept of external memory from deep learning (Joulin and Mikolov, 2015; Graves et al., 2016) to build an entity library for an exposure sequence that … VQA: Visual Question Answering … Combining language and vision with a multimodal skip-gram model …

JA Rodrigues, C Saedi, V Maraev, J Silva… – Proceedings of the 6th …, 2017 – aclweb.org
… with contribu- tions from any other user (which are piled up in a thread); and (ii) from Quora, an online moderated question answering site where each … (2015) specify only the skip-gram neural network … We resorted to the Keras deep learning library to replicate this architecture …

Learning Domain-Specific Word Embeddings from Sparse Cybersecurity Texts
A Roy, Y Park, SH Pan – arXiv preprint arXiv:1709.07470, 2017 – arxiv.org
… all the other models with the highest MRR of 7%, which represents a 22.22% MRR improvement over the next best systems, the retrofitting model and the Skip- Gram with hi- erarchical softmax model … Knowledge-powered deep learning for word embed- ding …

Learning word representations for sentiment analysis
Y Li, Q Pan, T Yang, S Wang, J Tang, E Cambria – Cognitive Computation, 2017 – Springer
… some public sentiment lexicons such as the multi-perspective question answering (MPQA) corpus … WLJC outperform other word embedding algorithms such as GloVe, Skip-gram and CBOW … Majumder N, Poria S, Gelbukh A, Cambria E. Deep learning-based document modeling …

Improvised theatre alongside artificial intelligences
K Mathewson, P Mirowski – AAAI Conference on Artificial …, 2017 – pdfs.semanticscholar.org
… processing and speech recognition to reinforce- ment and deep learning, these chatbots … domains such as appointment setting, bank information requests, question-answering, and movie … Extrapolat- ing human judgments from skip-gram vector representations of word meaning …

Building CMU Magnus from User Feedback
S Prabhumoye, F Botros, K Chandu… – Alexa Prize …, 2017 – nzini.com
… The skip-thoughts vectors model abstracts the skip-gram model to the sentence level … Lstm-based deep learning models for non-factoid answer selection … A long short-term memory model for answer sentence selection in question answering. In ACL (2), pages 707–712, 2015 …

Document-Level Multi-Aspect Sentiment Classification as Machine Comprehension
Y Yin, Y Song, M Zhang – Proceedings of the 2017 Conference on …, 2017 – aclweb.org
… 2016). The model parameters are tuned based on the de- velopment sets. We learn 200-dimensional word embeddings with Skip-gram model (Mikolov et al., 2013) on in-domain corpus, which fol- lows (Tang et al., 2015a). The …

A Comprehensive Survey of Graph Embedding: Problems, Techniques and Applications
H Cai, VW Zheng, KCC Chang – arXiv preprint arXiv:1709.07604, 2017 – arxiv.org
… Graph Embedding Techniques Matrix Factorization Deep Learning Graph Laplacian Eigenmaps Node Proximity Matrix Factorization … 3), which mainly exist in the below three scenarios. Community-based question answering (cQA) sites …

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
M Palmer, R Hwa, S Riedel – Proceedings of the 2017 Conference on …, 2017 – aclweb.org
… Area Chairs Information Extraction, Information Retrieval, and Question Answering Mihai Surdeanu, University of Arizona Jing Jiang, Singapore Management University Hinrich Schütze … Incremental Skip-gram Model with Negative Sampling Nobuhiro Kaji and Hayato Kobayashi …

Word Sense Determination from Wikipedia Data Using Neural Networks
Q Liu – 2017 – scholarworks.sjsu.edu
… or a factory. Being able to determine the sense of such words is very useful in natural language processing tasks, such as speech synthesis, question answering, and machine translation. For the … 18 3.2 Skip-gram Model ….. 18 …

Rozpoznávání pojmenovaných entit s pomocí rekurentních neuronových sítí
HL Nguyen – 2017 – dspace.cvut.cz
… The results are often used in a pipeline, such as question answering or entity linking … Introduction Named entity recognition (NER) is commonly used in information retrieval tasks such as question answering, where we want to detect entities that contribute to the question …

Detecting Positive Medical History Mentions
B Bai, PF Laquerre, R Jackson, R Stewart – Proceedings of the 40th …, 2017 – dl.acm.org
… e di erence from n-gram is that skip-grams allow intervals between words: a 2-skip-gram for the sequence “ e cat sat on … We list most positive and most negative n-gram/skip-gram features in Table 5. For each set of features, on the le are the … A CLoser Look at Skipgram Modelling …

Automatic synonym extraction using Word2Vec and spectral clustering
L Zhang, J Li, C Wang – Control Conference (CCC), 2017 36th …, 2017 – ieeexplore.ieee.org
… 1], which is useful for many NLP-related tasks, such as question answering (QA), text … been subsequently analyzed and applied by other researchers[14][15], using deep learning ideology to … Word2Vec can use either of continuous skip-gram or continuous bag-of-words (CBOW …

Anaphoricity Determination of Anaphora Resolution in Uygur Pronoun Based on CNN-LSTM Model
T Shengwei, L Dongbai, Y Long, F Guanjun… – International Journal …, 2017 – World Scientific
… Fig. 1. The structure of skip-gram … Recently, deep learning models such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM) have been successfully employed in image processing, speech recognition, etc …

Deep memory networks for attitude identification
C Li, X Guo, Q Mei – Proceedings of the Tenth ACM International …, 2017 – dl.acm.org
… 3.1 Background: Memory Networks As one of the recent developments of deep learning, memory networks [35] have been successfully applied to language model- ing, question answering, and aspect-level sentiment analysis [36], which generates superior performance over …

Deep neural architectures for automatic representation learning from multimedia multimodal data
V Vukotic – 2017 – theses.fr
… With the recent resurgence of neural networks, the rapid development of deep learning methods and the proliferation of massive amounts of unlabeled data, unsupervised learning algorithms — which can automatically discover interesting and useful patterns in such data …

A Hierarchical Book Representation of Word Embeddings for Effective Semantic Clustering and Search.
A Bleiweiss – ICAART (2), 2017 – scitepress.org
… et al., 2013), named entity recognition (Guo et al., 2014), and question answering (Iyyer et al … In recent years, neural network architectures have inspired the deep learning of word embeddings from … Both the skip- gram and continuous bag-of-words (CBOW) models offer a simple …

How to evaluate word embeddings? On importance of data efficiency and simple supervised tasks
S Jastrzebski, D Le?niak, WM Czarnecki – arXiv preprint arXiv:1702.02170, 2017 – arxiv.org
… is the growing evidence that pretrained word embeddings provide little benefit under various settings, especially deep learning mod- els … In some experiments we additionally include publicly available pretrained skip-gram embeddings on Google News corpora and skip- n-gram …

Improving Event Causality Recognition with Multiple Background Knowledge Sources Using Multi-Column Convolutional Neural Networks.
C Kruengkrai, K Torisawa, C Hashimoto, J Kloetzer… – AAAI, 2017 – aaai.org
… In NLP, Dong et al. (2015) used MCNNs to capture the multi- ple aspects of candidate answers for question-answering. Zeng et al … 2012). We pre-trained 300-dimensional word embedding vectors using the skip-gram model (Mikolov et al …

Incorporating message embedding into co-factor matrix factorization for retweeting prediction
C Wang, Q Li, L Wang, DD Zeng – Neural Networks (IJCNN) …, 2017 – ieeexplore.ieee.org
… Word2vec is a deep-learning-inspired method that attempts to understand meaning and semantic relationships among words … Skip-gram … We adopt SkipGram [25] to construct the latent semantic vector for word embedding, then learn latent vector of message, finally, semantic …

Cross Temporal Recurrent Networks for Ranking Question Answer Pairs
Y Tay, LA Tuan, SC Hui – arXiv preprint arXiv:1711.07656, 2017 – arxiv.org
… In the context of question answering, we learn which information to remember or discard in … Our Proposed Approach In this section, we describe our novel deep learning model layer-by … These embeddings are trained with the Skip- gram model using the Wikipedia and AQUAINT …

A Convolutional Neural Network for Search Term Detection
H Salehinejad, J Barfett, P Aarabi, S Valaee… – arXiv preprint arXiv …, 2017 – arxiv.org
… The continuous skip-gram considers heavier weights for the surrounding context words of the … [14] N. Majumder, S. Poria, A. Gelbukh, and E. Cambria, “Deep learning- based document … long short-term memory model for answer sentence selection in question answering,” in ACL …

Breakingnews: Article annotation by image and text processing
A Ramisa, F Yan, F Moreno-Noguer… – IEEE transactions on …, 2017 – ieeexplore.ieee.org
… Relevant work has also been done in the area of Visual Question Answering VQA [2]. DAQUAR … are based on the most recent state-or-the-art approaches in deep learning, and thus … the objective is predicting a word given its context, and the Continuous skip-gram model, where …

Multi-level Attention-Based Neural Networks for Distant Supervised Relation Extraction
L Yang, TLJ Ng, C Mooney, R Dong – aics2017.dit.ie
… a key role in many natural language processing (NLP) tasks, including question answering, web search … this problem, many scholars [17], [22], [12] attempt to apply deep learning techniques instead … Similar to previous work [12], we use the Skip-gram neural network architecture …

What is the Role of Recurrent Neural Networks (RNNs) in an Image Caption Generator?
M Tanti, A Gatt, KP Camilleri – arXiv preprint arXiv:1708.02043, 2017 – arxiv.org
… age, although recently, there has been growing inter- est in moving beyond this, with research on visual question-answering (Antol et … Inject architectures are the most widely attested among deep learning approaches to image captioning (Chen and Zitnick, 2015; Donahue et al …

“Show me the cup”: Reference with Continuous Representations
G Boleda Torrent, M Baroni, S Padó – 2017 – repositori.upf.edu
… Our task is more distantly related to visual question answering [29–31], in the sense … of the COLING, Geneva, Switzerland (2004) 1240–1246 6. Nielsen, M.: Neural Networks and Deep Learning … N., Baroni, M.: Combining language and vision with a multimodal skip-gram model …

Learning multiple layers of knowledge representation for aspect based sentiment analysis
DH Pham, AC Le – Data & Knowledge Engineering, 2017 – Elsevier
… Some other studies [32–34] utilized deep learning techniques such as convolutional neural … tasks like text classification, document classification, information retrieval, question answering, name entity … The skip-gram and continuous bag-of-words (CBOW) models proposed by …

Effective Use of Bidirectional Language Modeling for Medical Named Entity Recognition
DS Sachan, P Xie, EP Xing – arXiv preprint arXiv:1711.07908, 2017 – arxiv.org
… pre-trained word embeddings on several medical NER datasets based on a deep learning model proposed … extraction from images such as object recognition19 and visual question answering20 … for language modeling task, we learn the word vectors using skip-gram28 approach …

Road2Vec: Measuring Traffic Interactions in Urban Road System from Massive Travel Routes
K Liu, S Gao, P Qiu, X Liu, B Yan, F Lu – ISPRS International Journal of …, 2017 – mdpi.com
Good characterization of traffic interactions among urban roads can facilitate traffic-related applications, such as traffic control and short-term forecasting. Most studies measure the traffic interaction between two roads by their topological distance or the correlation between their traffic …

Deep learning in semantic kernel spaces
D Croce, S Filice, G Castellucci, R Basili – … of the 55th Annual Meeting of …, 2017 – aclweb.org
… 3 Deep Learning in Kernel Spaces … QC) is the task of map- ping a question into a closed set of answer types in a Question Answering system … https://www.tensorflow.org/ 5The lexical vectors used in the CSPTK are generated again using the Word2vec tool with a Skip-gram model …

Application of a Hybrid Bi-LSTM-CRF model to the task of Russian Named Entity Recognition
LT Anh, MY Arkhipov, MS Burtsev – arXiv preprint arXiv:1709.09686, 2017 – arxiv.org
… 1 Neural Networks and Deep Learning Lab, Moscow Institute of Physics and Technology, Russia … to be useful in many NLP tasks such as machine translation, question answering, and especially for … To train embeddings on this corpus, we use skip-gram algorithm enriched with …

Semi-Supervised Structured Prediction with Neural CRF Autoencoder
X Zhang, Y Jiang, H Peng, K Tu… – Proceedings of the 2017 …, 2017 – aclweb.org
… Processing (NLP), charac- terized by highly structured tasks, promising re- sults were obtained by models that combine deep learning methods with … et al., 2011), machine trans- lation (Bahdanau et al., 2015), sentiment anal- ysis (Kim, 2014) and question answering (An- dreas …

Application of a Hybrid Bi-LSTM-CRF Model to the Task of Russian Named Entity Recognition
VN Haiphong – Artificial Intelligence and Natural Language: 6th …, 2017 – Springer
… Y. Arkhipov1 (B), and Mikhail S. Burtsev1 1 Neural Networks and Deep Learning Lab, Moscow … be useful in many NLP tasks such as machine translation, question answering, and especially … To train embeddings on this corpus, we use skip-gram algorithm enriched with subword …

Multi-Task Neural Network for Non-discrete Attribute Prediction in Knowledge Graphs
Y Tay, LA Tuan, MC Phan, SC Hui – Proceedings of the 2017 ACM on …, 2017 – dl.acm.org
… Neural networks are connectionist models that form the heart of the deep learning revolution … model was rst incepted in the elds of NLP for sentiment analysis and subsequently applied in multiple domains such as knowledge base completion [32] and question answering [31, 35 …

Sequence to Sequence Networks for Roman-Urdu to Urdu Transliteration
M Alam – arXiv preprint arXiv:1712.02959, 2017 – arxiv.org
… In this paper, neural machine translation based on deep learning has been applied to … a variety of problems: conversational modeling [7], handwriting generation, question answering, speech recognition … important are: the choice between a variant from skip- gram or continuous …

Revisit Word Embeddings with Semantic Lexicons for Modeling Lexical Contrast
J Liu, Z Liu, H Chen – Big Knowledge (ICBK), 2017 IEEE …, 2017 – ieeexplore.ieee.org
… SENNA) [1], the hierarchical log-bilinear model [2], Con- tinuous Bag-of-Word Model (CBOW), Skip-gram [3] and Global Vectors [4]. Effectiveness of these models has been demonstrated by various applications, such as information retrieval [5], question answering [6] and text …

Content Tree Word Embedding for document representation
M Kamkarhaghighi, M Makrehchi – Expert Systems with Applications, 2017 – Elsevier
… base completion task which is useful in query expansion, question answering, and information … In this study, three document representation models based on deep learning are compared … to create word vector representation: Continuous Bag-of-Words (CBOW) and Skip-gram …

Comparison and Fine-grained Analysis of Sequence Encoders for Natural Language Processing
TA Keller – 2017 – search.proquest.com
… 8. Figure 2.5: Skip-gram model for learning word representations. Where wt1;wt ;wt+1 … Additionally, we examine how sentence representations are employed. in question answering models, specifically Memory Networks, and study the impact … deep-learning architectures …

Neural Models for Information Retrieval
B Mitra, N Craswell – arXiv preprint arXiv:1705.01509, 2017 – arxiv.org
… Therefore, IR systems that focus on the question-answering task need to model the patterns expected in the answer passage based on … Deep learning models containing millions (or even billions) of parameters, on the other hand, are known to be more sensitive to distributional …

A structural representation learning for multi-relational networks
L Liu, X Li, WK Cheung, C Xu – Proceedings of the 26th International Joint …, 2017 – ijcai.org
… become important resources to support more advanced information retrieval, question- answering systems, etc … to model network community structure via random walk and skip-gram together to … SDNE [Wang et al., 2016] offers a semi- supervised deep learning framework to …

Prediction of Frame-to-Frame Relations in the FrameNet Hierarchy with Frame Embeddings
T Botschen, HM Sergieh, I Gurevych – Proceedings of the 2nd Workshop …, 2017 – aclweb.org
… Flekova (2016) use the skip- gram model on texts with mapped WordNet super- senses (Miller, 1990; Fellbaum, 1990) … Answer selection model Link Prediction is methodologically related to the key-task of An- swer Selection from Question Answering (QA) …

Deep embodiment: grounding semantics in perceptual modalities
D Kiela – 2017 – cl.cam.ac.uk
… In that case, data is considerably more sparse, making a deep learning approach less feasible … re- trieval (Salton et al., 1975), text classification (see Sebastiani, 2002), question answering (Tellex et … The skip-gram (SG) model (Mikolov et al., 2013b) learns to predict the words that …

Semantic Query Processing: Estimating Relational Purity
JC Kalo, C Lofi, RP Maseli… – 15th Lernen Wissen …, 2017 – pdfs.semanticscholar.org
… diverging user and knowledge base vocabulary for query processing in question answering [10] (ie … embeddings relying on neural networks have become popular, with the skip-gram negative sampling … In: ICML Workshop on Knowledge-Powered Deep Learning for Text Mining …

Semantic specialisation of distributional word vector spaces using monolingual and cross-lingual constraints
N Mrkši?, I Vuli?, DÓ Séaghdha, I Leviant… – arXiv preprint arXiv …, 2017 – arxiv.org
… training word vectors is to improve the lexical cover- age of supervised models for language understanding tasks, eg question answering (Iyyer et … Other ones modify the skip-gram (Mikolov et al., 2013b) objective function by introducing semantic constraints (Yih et al., 2012; Liu …

Multimodal question answering over structured data with ambiguous entities
H Li, Y Wang, G de Melo, C Tu, B Chen – Proceedings of the 26th …, 2017 – dl.acm.org
… Recently, the idea of visual question answering has been proposed [36], which involves answering … similarity of their corresponding word vectors, using Google’s word2vec skip-gram with negative … multitude of visual features, including GIST, HOG, SIFT and deep learning models …

Audio and Text based Multimodal Sentiment Analysis using Features Extracted from Selective Regions and Deep Neural Networks
H Abburi – 2017 – web2py.iiit.ac.in
… In this work also different deep learning architectures like Deep Neural Network (DNN) and Deep Neural Network Attention Mechanism (DNNAM) are explored … By overcoming this problem many researches reported by employing deep learning in sentiment analysis …

Application of a Hybrid Bi-LSTM-CRF Model to the Task of Russian Named Entity Recognition
MY Arkhipov, MS Burtsev – Conference on Artificial Intelligence and …, 2017 – Springer
… to be useful in many NLP tasks such as machine translation, question answering, and especially for … To train embeddings on this corpus, we use skip-gram algorithm enriched with subword … At the moment deep learning methods are seen as the most promising choice for NER …

Visually grounded meaning representations
C Silberer, V Ferrari, M Lapata – IEEE transactions on pattern …, 2017 – ieeexplore.ieee.org
… Examples include infor- mation retrieval [4], search query expansions [5], document classification [6], and question answering [7]. Vector spaces have been also popular in cognitive science figuring promi … [40] modify skip-gram’s learning objective so … Multimodal Deep Learning …

Hitachi at SemEval-2017 Task 12: System for temporal information extraction from clinical notes
PR Sarath, R Manikandan, Y Niwa – Proceedings of the 11th …, 2017 – aclweb.org
… It is essential for improving the performance of ap- plications such as question answering, search, text … by a two-layer perceptron NER using H2O toolkit and skip-gram based word2vec … data of target domain through data augmentation techniques using deep learning methods …

Improving Language Modelling with Noise-contrastive estimation
FF Liza, M Grzes – arXiv preprint arXiv:1709.07758, 2017 – arxiv.org
… its context, play an im- portant role in many downstream applications such as machine translation, question answering and text … This seems to be a common pattern in deep learning research … 2013) trained using the standard continuous skip-gram model with negative sampling …

User Personalized Satisfaction Prediction via Multiple Instance Deep Learning
Z Chen, B Gao, H Zhang, Z Zhao, H Liu… – Proceedings of the 26th …, 2017 – dl.acm.org
… Pre-train the word-embedding of Q and A by skip-gram 2: Initialize … empirically evaluate and validate our proposed frame- work multiple instance deep learning(MIDL), we … Preparation The dataset downloaded from the famous community-based question answering portal Stack …

Entity Extraction in Biomedical Corpora: An Approach to Evaluate Word Embedding Features with PSO based Feature Selection
S Yadav, A Ekbal, S Saha… – Proceedings of the 15th …, 2017 – aclweb.org
… several pipelined applications such as information extraction, automatic summarization, question- answering, word sense … In re- cent times deep learning based approaches such as Recurrent Neural Network … as Continuous Bag-of-Words Model (CBOW) and Skip-gram model …

A Customized Attention-Based Long Short-Term Memory Network for Distant Supervised Relation Extraction
D He, H Zhang, W Hao, R Zhang, K Cheng – Neural computation, 2017 – MIT Press
… It can play a significant role in various natural language processing (NLP) tasks, for example, information extraction (Banko et al., 2007; Wu & Weld, 2010), question answering (Iyyer, Boyd-Graber, Claudino, Socher, & Daumé, 2014), knowledge-based construction (Suchanek …

Deep neural networks for identification of sentential relations
W Yin – 2017 – edoc.ub.uni-muenchen.de
… In den letzten Jahren haben Deep-Learning-Ansätze vielversprechende Ergeb- nisse für eine große Anzahl verschiedener NLP-Tasks erzielt … Für open-domain question answering, zum Beispiel, kann ein System, das erkennt, dass eine neue Frage eine Paraphrase einer …

Do We Need Cross Validation for Discourse Relation Classification?
W Shi, V Demberg – Proceedings of the 15th Conference of the …, 2017 – aclweb.org
… variety of NLP tasks such as summarization (Yoshida et al., 2014), question answering (Jansen et … Many deep learning methods have been proved to be helpful in discourse relation parsing … Word Embeddings are trained with the Skip-gram archi- tecture in Word2Vec (Mikolov et …

Opinion Recommendation Using A Neural Model
Z Wang, Y Zhang – Proceedings of the 2017 Conference on Empirical …, 2017 – aclweb.org
… Deep learning is a relative- ly more feasible choice, offering viabilities of in- formation fusion by fully connected hidden layer- s (Collobert et al., 2011; Henderson et al., 2013; Zhang and Weiss … Word embeddings are trained using the Skip-gram algorithm (Mikolov et al., 2013)1 …

Improved neural relation detection for knowledge base question answering
M Yu, W Yin, KS Hasan, C Santos, B Xiang… – arXiv preprint arXiv …, 2017 – arxiv.org
… Recent re- search benefits a lot from the advancement of deep learning: from word embeddings (Nguyen … Fader et al., 2013)), in order to fit the goal of open-domain question answering … relation-level focus more on global informa- tion (long phrases and skip-grams) but suffer …

An CNN-LSTM Attention Approach to Understanding User Query Intent from Online Health Communities
R Cai, B Zhu, L Ji, T Hao, J Yan… – Data Mining Workshops …, 2017 – ieeexplore.ieee.org
… inestimable medical knowledge.This QA data can benefit many medical related tasks, such as knowledge extraction [1], Question-Answering system [2 … We train the word2vec model with the Skip-gram architecture and setting the windows size as 7. The dimension of word vector …

Commonsense Knowledge for 3D Modeling: A Machine Learning Approach
K Hassani – 2017 – ruor.uottawa.ca
… [5], and visual question answering [6]. Creating 3D contents and animating them in virtual worlds is a time–consuming and … We approach these tasks by using deep learning and probabilistic … Skip?Grams: an n?gram model that allows tokens to be skipped to overcome data …

Opinion Recommendation using Neural Memory Model
Z Wang, Y Zhang – arXiv preprint arXiv:1702.01517, 2017 – arxiv.org
… data. Deep learning is a relatively more feasible choice, offering viabilities of information fusion by fully connected hidden layers (Collobert et al., 2011; Henderson et al., 2013). We leverage this advantage in building our model …

An introduction to neural information retrieval
B Mitra, N Craswell – … and Trends® in Information Retrieval (to appear …, 2017 – microsoft.com
… Therefore, IR systems that focus on the question-answering task need to model the patterns expected in the answer passage based on … Deep learning models containing millions (or even billions) of parameters, on the other hand, are known to be more sensitive to distributional …

Predicting Visual Features from Text for Image and Video Caption Retrieval
J Dong, X Li, CGM Snoek – arXiv preprint arXiv:1709.01362, 2017 – arxiv.org
… feature to a much shorter h-dimensional vector, with the transformation optimized in an end-to-end fashion within a deep learning framework [9 … We therefore train a 500-dimensional word2vec model on English tags of 30 million Flickr images, using the skip-gram algorithm [36] …

On lexical level matching
S Ling – 2017 – ideals.illinois.edu
… vii Page 9. LIST OF FIGURES 3.1 Skip-gram Model … 1.2 Challenges and Motivation With the trend of deep learning, the NLP community shows a strong interest … Other than that, named entity recognition is also widely used in Question Answering [13] and Machine Translation …

Cascade word embedding to sentence embedding: A class label enhanced approach to phenotype extraction
W Xing, X Yuan, L Li, J Peng – Bioinformatics and Biomedicine …, 2017 – ieeexplore.ieee.org
… numbers. One of the most important model is skip-gram model … documents. It has been widely and effec- tively used in various fields such as text classification [30], information retrieval [31], question answering [32], and query rewriting [33] …

Neural ranking models with weak supervision
M Dehghani, H Zamani, A Severyn, J Kamps… – Proceedings of the 40th …, 2017 – dl.acm.org
… is is a key requirement to maximize the bene ts from deep learning models with weak supervision as it enables them to generalize be er … paper belongs to, consists of the approaches that design and train a (deep) neural network for a speci c task, eg, question answering [6, 41 …

Relation Extraction and Its Application to Question Answering
Y Xu – 2017 – era.library.ualberta.ca
… 97 6.11 The results of the open question answering system on the orig- inal development set and the one with expanded answers. . . . 98 ix Page 10 … 36 4.1 A bayesian network that defines the quality of clustering. . . . 44 4.2 The skip-gram word2vec model …

Knowledge-Aware Natural Language Understanding
P Dasigi – 2017 – pdfs.semanticscholar.org
… For example, Question Answering (QA) refers to answering questions about a span of text, or other structured knowledge representations like … et al., 2016; Bowman et al., 2016; Yang et al., 2016, among many others) use representation learning or deep learning techniques to …

Unsupervised Sentence Representations as Word Information Series: Revisiting TF–IDF
I Arroyo-Fernández, CF Méndez-Cruz, G Sierra… – arXiv preprint arXiv …, 2017 – arxiv.org
… Some of these concerns include removing redundancies in question-answering forums or … Although sentence representation methods based purely on deep learning have shown competitive … neural architectures producing word embeddings with W2V: the Skip-gram and the …

CONE: Community Oriented Network Embedding
C Yang, H Lu, KCC Chang – arXiv preprint arXiv:1709.01554, 2017 – arxiv.org
… The deep learning framework of RNN also provides us with the flexibility to modify the neural network architectures in order to leverage other … They usually model network structures by sampling a set of paths from the networks and applying a Skipgram-based model [23] to …

Supervised and unsupervised methods for learning representations of linguistic units
S Rothe – 2017 – edoc.ub.uni-muenchen.de
… 20 1.1.2 Skip-Gram … We will cover the three most frequently used word embedding methods in this section, namely Continuous Bag of Word (CBOW), Skip-gram and Global Vectors (GloVe). 1.1.1 Continuous Bag of Words …

Enhanced word embedding similarity measures using fuzzy rules for query expansion
Q Liu, H Huang, J Lut, Y Gao… – Fuzzy Systems (FUZZ …, 2017 – ieeexplore.ieee.org
… word embed- ding, including a series of works that apply deep learning techniques to … Query expansion based on word embedding methods (QE-CBOW and QE-Skip-gram) outperformed the … QE-Skipgram-Fuzzy rules freeh 0.953 cia 0.613 pitts 0.561 undercover 0.426 mislock …

Improving the Compositionality of Word Embeddings
MJ Scheepers – 2017 – thijs.ai
… retrieval [43], automatic summarization [139], machine translation [123], sentiment analysis [35] and question answering [122] … Compositional models in Deep Learning 11 … which attempts a rigorous theoretical understanding for the performance of averaging skip-gram vectors …

On Sampling Strategies for Neural Network-based Collaborative Filtering
T Chen, Y Sun, Y Shi, L Hong – Proceedings of the 23rd ACM SIGKDD …, 2017 – dl.acm.org
… Hence, it is natural to combine deep learning with traditional collaborative filtering for recommendation tasks, as seen in recent studies [1, 4, 32, 37] … And skip-gram (SG) loss has been successfully utilized to learn robust word embedding [22] …

Word Sense Determination from Wikipedia Data Using Neural Networks Qiao Liu
Q Liu – 2017 – pdfs.semanticscholar.org
… or a factory. Being able to determine the sense of such words is very useful in natural language processing tasks, such as speech synthesis, question answering, and machine translation. For the … 18 3.2 Skip-gram Model ….. 18 …

Neural information retrieval: At the end of the early years
KD Onal, Y Zhang, IS Altingovde, MM Rahman… – Information Retrieval …, 2017 – Springer
… First is question answering (QA) and community question answering (CQA) tasks, which perhaps … A variety of resources for deep learning, including links to popular open-source software … became widespread after the introduction of the shallow models Skip-gram and Continuous …

Hybrid BiLSTM-Siamese network for FAQ Assistance
P Khurana, P Agarwal, G Shroff, L Vig… – Proceedings of the 2017 …, 2017 – dl.acm.org
… 5 RELATED WORK 5.1 NLP using Deep Learning Recently deep learning algorithms have gained … Automatic question answering via paragraph comprehen- sion is a very important problem, and it has … preprocessing the text we learn the word2vec [17] using skip gram algorithm …

Learning Logic Rules From Text Using Statistical Methods For Natural Language Processing
M KAZMI – 2017 – peterschueller.com
… artificial intelligence. Some popular NLP applications are information extraction, machine translation, text summarization, and question answering. This dissertation proposes a new methodology using Answer Set programming (ASP) as …

Improving Arabic sentiment analysis with sentiment-specific embeddings
AA Altowayan, A Elnagar – Big Data (Big Data), 2017 IEEE …, 2017 – ieeexplore.ieee.org
… 1Multi-Perspective Question Answering corpus … B. Word embedding using fastText In the last few years, the idea of using deep learning to represent words by vectors is gaining … FastText, as word2vec, provides two models for computing word embedding: Skip-gram and CBOW …

Event extraction from bio-medical documents
K Nayak – 2017 – library.isical.ac.in
… Analysis, Handwrit- ten Digit Recognition, Machine Translation, Language Modeling, Question Answering, Chat-Bots … 1http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip-gram-model … basics of Recurrent Neural Networks (RNNs) which are deep learning models that …

Sentence?Chain Based Seq2seq Model for Corpus Expansion
E Chung, JG Park – ETRI Journal, 2017 – Wiley Online Library
… Neural conversational models have achieved impressive results in deep learning studies … A question answering study, such as that in [23], converts documents into context-query-answer triples in order … This allows us to use the skip-gram model [5]. The skip-gram model uses an …

A Deep Architecture for Content-based Recommendations Exploiting Recurrent Neural Networks
A Suglia, C Greco, C Musto, M de Gemmis… – Proceedings of the 25th …, 2017 – dl.acm.org
… 3.1 Basics of RNNs and LSTMs In the past years, deep learning has o ered to … is partially inspired by the model based on LSTMs recently proposed in a Question Answering (QA) scenario … D2V, we learn embeddings of dimension 300 and 500 by using the Skip-Gram (SG) and …

Entity-Centric Discourse Analysis and Its Applications
X Wang – 2017 – repository.kulib.kyoto-u.ac.jp
… 54 6.1 An Example of Question Answering … The learnt word vectors thus can carry more information. As high quality word representations are the basis of many deep learning NLP models, the learnt word embeddings from our model provide a solid foundation …

Leveraging bilingually-constrained synthetic data via multi-task neural networks for implicit discourse relation recognition
C Wu, X Shi, Y Chen, Y Huang, J Su – Neurocomputing, 2017 – Elsevier
… Discovering the discourse relation between two sentences/clauses is crucial to understanding the meaning of a coherent text, and also beneficial to many downstream natural language processing (NLP) applications, such as question answering and machine translation …

Whodunnit? Crime Drama as a Case for Natural Language Understanding
L Frermann, SB Cohen, M Lapata – arXiv preprint arXiv:1710.11601, 2017 – arxiv.org
… Our inference task can be viewed as a form of question answering over multi-modal data, focus- ing on one type of question … that our model generalizes well to unseen episodes, de- spite being trained on a relatively small data sample compared to standards in deep learning …

Multiview convolutional neural networks for multidocument extractive summarization
Y Zhang, MJ Er, R Zhao… – IEEE transactions on …, 2017 – ieeexplore.ieee.org
… B. Deep Learning in Multidocument Summarization Many recent works have already been done using deep learning methods to summarize documents … The model was developed based on the architecture of CBOW and skip-gram, but proved to perform subopti- mally on some …

Multi-column convolutional neural networks with causality-attention for why-question answering
JH Oh, K Torisawa, C Kruengkrai, R Iida… – Proceedings of the …, 2017 – dl.acm.org
Multi-Column Convolutional Neural Networks with Causality-Attention for Why-Question Answering … ABSTRACT Why-question answering (why-QA) is a task to retrieve an- swers (or answer passages) to why-questions (eg, “why are tsunamis generated?”) from a text archive …

On the use of vector representation for improved accuracy and currency of Twitter POS Tagging
D Samuel – 2017 – dalspace.library.dal.ca
… in keeping with current trends (especially among the Deep Learning research commu … learning word representations, the skip-gram and continuous bag-of-words (CBOW) models of Mikolov et al … Analysis, Question Answering, Information retrieval and others …

Distant supervised relation extraction via long short term memory networks with sentence embedding
D He, H Zhang, W Hao, R Zhang, G Chen… – Intelligent Data …, 2017 – content.iospress.com
… It can play a significant role in various scenarios, eg information extraction [32, 10], question answering [20], knowledge base construction [9 … Recurrent neural network (RNN) models are promising deep learning models which can represent phrases of arbitrary length in a vector …

Evaluating word-vector generation techniques for use in semantic role labeling with recurrent LSTM neural networks
D TOOM – publications.lib.chalmers.se
… Keywords: word-vector generation, SRL, NCE, CBOW, skip-grams, GloVe, CoNll- 2005, LSTM RNN. v Page 6 … 34 5.4 Results using vectors from CBOW and Skip-gram … Recently, Zhou and Xu [10] showed that a deep-learning method can be used to solve the SRL task …

Encoding syntactic knowledge in neural networks for sentiment classification
M Huang, Q Qian, X Zhu – ACM Transactions on Information Systems …, 2017 – dl.acm.org
… Neural networks; Lexical semantics; Additional Key Words and Phrases: Neural networks, recursive neural network, long short-term memory, deep learning, representation learning, sentiment … 2011b], question answering [Dong et al … 2013b] or Skip-gram models [Mikolov et al …

Artificial neural network methods applied to sentiment analysis
S Ebert – 2017 – edoc.ub.uni-muenchen.de
… for hard sentiment words 23 3.1 Introduction 23 3.2 Linguistic Analysis of Sentiment Contexts of “hard” 24 3.3 Deep Learning Features 26 … An intriguing application are question answering systems that can be enabled to react according to a user’s mood and might soothe them …

De-identification of patient notes with recurrent neural networks
F Dernoncourt, JY Lee, O Uzuner… – Journal of the American …, 2017 – academic.oup.com
We use cookies to enhance your experience on our website. By continuing to use our website, you are agreeing to our use of cookies. You can change your cookie settings at any time. Find out more …

Analysis of sentence embedding models using prediction tasks in natural language processing
Y Adi, E Kermany, Y Belinkov, O Lavi… – IBM Journal of …, 2017 – ieeexplore.ieee.org
… These vectors can then be used for further prediction tasks or as pre-training for non-convex models such as deep-learning models … For the CBOW model, we train Skip-gram word vectors [1], with hierarchical-softmax and a window size of 5 words, using the Gensim …

Joint Probabilistic Reasoning About Coreference And Relations Of Universal Schema
A McCallum, N Monath – 2017 – dtic.mil
… Multi-Sense Skip-Gram Model SCWS Results…..49 Table 35 … Finally, we extended the Neural Programmer model [15], which induces latent programs, to perform question answering on knowledge base fact tables [16]. Page 10 …

Broad Discourse Context for Language Modeling
M Torres Garcia – 2017 – research-collection.ethz.ch
… This problem becomes very important in applications that rely on knowledge-extraction, such as question-answering or reading comprehension, where rare words are of primary interest. 1 Page 11. 1. Introduction 1.2 Thesis Contributions … Continuous Skip-Gram model: given a …

Transfer learning for low-resource natural language analysis
Y Zhang – 2017 – dspace.mit.edu
… should be capable to learn to extract information that is relevant to a particular aspect while ignoring the rest. Recently, the employment of deep learning methods brings new challenges in do- main adaptation problem. While complex neural models have achieved top perfor …

LSTM Language Modeling for Intrusion Detection Systems
??? – 2017 – s-space.snu.ac.kr
… [5, 6], speech recognition [7], question answering [8], and summarization [9] … 12-15]. For more recent deep learning-based techniques, there exists an example that … words (CBOW) and the skip-gram can be adapted [44]. Negative sampling method …

Deep Learning for text data mining: Solving spreadsheet data classification.
A Kimashev – 2017 – brage.bibsys.no
Page 1. FACULTY OF SCIENCE AND TECHNOLOGY MASTER’S THESIS Deep Learning for text data mining: Solving spreadsheet data classification. Formal Supervisor: Prof … This project’s challenge was to develop a new classifier based on Deep Learning …

Social Media Sentiment Analysis: Multilingual Methodology and Monolingual Application
?????????????? – 2017 – ynu.repo.nii.ac.jp
… 1 Illustration of Gender Relations Represented by Word Embeddings 11 2 CBOW and Skip-gram Architectures … 101 32 Results of Deep Learning Models Before and After Space Transfor- mation … ment analysis is not as long as that of machine learning, question answering, or …

An empirical analysis of nmt-derived interlingual embeddings and their use in parallel sentence identification
C España-Bonet, ÁC Varga… – IEEE Journal of …, 2017 – ieeexplore.ieee.org
… We highlight three: being a deep learning architecture, NMT does not require manually predefined features; it allows for the simultaneous training … In information retrieval, particularly in question answering [17], a high simi- larity between a document and an information request is …

Implicit discourse relation recognition for english and Chinese with multiview modeling and effective representation learning
H Li, J Zhang, C Zong – ACM Transactions on Asian and Low-Resource …, 2017 – dl.acm.org
… po- larity ambiguities, and contingency relation detection can improve question answering systems and … have shown superiority over randomly initialized ones in many deep learning frameworks … Based on Word2vec (skip- gram), the sentence embedding is regarded as an …

Information extraction with neural networks
JY Lee – 2017 – dspace.mit.edu
… Extracted relations can also be used for a variety of tasks such as the development of medical ontologies, question-answering sys- tems, and clinical trials [43]. 1.2 Contributions … eling [79], text classification [106, 59, 15, 67], question answering [127, 125], machine …

I Chaturvedi, S Poria, E Cambria – arXiv preprint arXiv:1710.06536, 2017 – arxiv.org
… 1.1 Synonyms Sentiment Analysis, Subjectivity Detection, Deep Learning Aspect Extraction, Po- larity Distribution, Convolutional Neural Network … In this chapter, we offer some insights about each task and apply an ensemble of deep learning and linguistics to tackle both …

Detection Of Causality Relations In Plain Text With The Use Of Word Embeddings
?? ??????? – 2017 – artemis-new.cslab.ece.ntua.gr
… Relevant research finds great applications in question answering, in- formation retrieval, event prediction, generating future scenarios and decision processing. Typical relations that have raised the interest of NLP researchers are part-whole, if-then, cause-effect …

Bag-of-concepts: Comprehending document representation through clustering words in distributed representation
HK Kim, H Kim, S Cho – Neurocomputing, 2017 – Elsevier
… Through training the weights of the network, skip-gram model of word2vec predicts the input word’s neighboring words within certain predefined window size. As shown in Fig … This general description for training skip-gram model of word2vec is depicted in Fig. 3 …

Where to put the image in an image caption generator
M Tanti, A Gatt, KP Camilleri – arXiv preprint arXiv:1703.09137, 2017 – arxiv.org
Page 1. Where to put the Image in an Image Caption Generator ? Marc Tanti marc.tanti.06@um.edu.mt Institute of Linguistics University of Malta Albert Gatt albert.gatt@um.edu.mt Institute of Linguistics University of Malta Kenneth …

Multimodal Learning for Vision and Language
J Mao – 2017 – search.proquest.com
… image question answering, one-shot novel concept captioning, multimodal word embedding, and multi-label classification. Many of these tasks have not been successfully addressed or even been investigated before. Our work are among the first deep learning effort for these …

Methods and Techniques for Clinical Text Modeling and Analytics
Y Ling – 2017 – search.proquest.com
… [97] proposed to apply metadata of category information from community question answering to enhance learning word embedding representation. Experimental results showed that extra knowledge can improve Skip-gram model on question retrieval task …

Using Adversarial Examples in Natural Language Processing
P B?lohlávek – 2017 – dspace.cuni.cz
… for image classification task1. In recent years, deep learning has outperformed many of other machine learning models in various tasks of natural language pro- cessing, such as automatic speech recognition (ASR) [4], speech synthesis [5], question answering [6] and standard …

Remedies against the vocabulary gap in information retrieval
C Van Gysel – arXiv preprint arXiv:1711.06004, 2017 – arxiv.org
Page 1. C h ristop h e V a n G yse l R e m e d ie sa g a in st th e V oc a b u la ry G a p in In form a tion R e trie va l CHRISTOPHE VAN GYSEL IN INFORMATION RETRIEVAL REMEDIES AGAINST THE VOCABULARY GAP arXiv:1711.06004v1 [cs.IR] 16 Nov 2017 Page 2. Page 3 …

Memory mechanism for argument component link prediction
KD Miller – 2017 – csus-dspace.calstate.edu
… achieved state-of-the-art scores on the Facebook’s babI 10k Question Answering (QA) … on each pass, however, complex deep learning models with multiple, wide LSTM layers … bag-of-words (CBOW) and skip-gram (n-gram) implementations for generating word Page 20. 10 …

Pay attention to those sets! Learning quantification from images
I Sorodoc, S Pezzelle, A Herbelot, M Dimiccoli… – arXiv preprint arXiv …, 2017 – arxiv.org
… on content word representations, and nouns in particular (see for example (Anderson et al., 2013; Lazaridou et al., 2015)), whilst another is interested in approximate sentence rep- resentation, as in the Image Captioning (IC) and the Visual Question Answering tasks (VQA) (eg …

Web-scale Multimedia Search for Internet Video Content
L Jiang – PhD diss, 2017 – cs.cmu.edu
… video retrieval. Finally, we apply the proposed video engine to tackle text-and-visual question answering problem called MemexQA. The extensive experiments demonstrate that the proposed methods are able to surpass state-of-the-art accuracy on multiple datasets …

Deep learning in bioinformatics
S Min, B Lee, S Yoon – Briefings in bioinformatics, 2017 – academic.oup.com
Abstract. In the era of big data, transformation of biomedical big data into valuable knowledge has been one of the most important challenges in bioinformatics.

A study on integrating distinct classifiers with bidirectional LSTM for Slot Filling task
KP Do – 2017 – dspace.jaist.ac.jp
… Fortunately, that costly work can be solved with a help of deep learning technique which is able to generate and extract potential features automat- ically during the training period … Following the success of deep learning method, Yao et al. [31] and Mensil et al …

What do we need to build explainable AI systems for the medical domain?
A Holzinger, C Biemann, CS Pattichis… – arXiv preprint arXiv …, 2017 – arxiv.org
… Ontology reasoning, nowadays based on deep learning rather than logic-based formal reasoning (Hohenecker and Lukasiewicz, 2017) is of great interest … In such black-box approaches already the input symbols (words) are replaced by vectors (eg skip-gram model for learning …

End-to-end Learning for Mining Text and Network Data
C Li – 2017 – deepblue.lib.umich.edu
… ineffective in many cases. Therefore, to push research forward, we design end-to-end deep learning algorithms that learn high-level representations directly from data and … To remedy the issue, we developed an end-to-end deep learning architecture, with the two subtasks inter …

Semantic Mapping in Video Retrieval
MHT de Boer – 2017 – repository.ubn.ru.nl
… computing power (ie GPU) and the new techniques within the field of deep learning, the detection … such as co-occurrence statistics (Mensink et al., 2014) and a skip-gram model (Chang et … These models either use skip-grams or continuous bag of words (CBOW) to create neural …

Computational models for semantic textual similarity
A González Aguirre – 2017 – addi.ehu.es
… 22 2.4.3 Continuous Bag-of-Words and Skip-gram models … In a sim- ilar way, STS can help in other tasks such as Plagiarism detection or Question Answering. 3 … Recently, thanks to the rise of Deep Learning, new useful resources (such as word embeddings) have been …

Natural language processing for resource-poor languages
L Duong – 2017 – minerva-access.unimelb.edu.au
… tical character recognition, natural language understanding, parsing, sentiment analysis, speech recognition, speech segmentation, speech synthesis, word seg- mentation, word sense disambiguation, question answering and natural language generation …

R Tian, N Okazaki, K Inui – Machine Learning, 2017 – Springer
… in NLP (Church and Hanks 1990; Dagan et al. 1994; Turney 2001; Turney and Pantel 2010). More recently, the Skip-Gram with Negative Sampling (SGNS) model (Mikolov et al. 2013a) is shown to be a matrix factorization of …

Improving Lexical Inference using Context-sensitive Distributional Models with Rich Context Representations
O Melamud – u.cs.biu.ac.il
… expert, lately applying deep learning methods to both images, speech and text. He … Processing (NLP) applications, including textual entailment, question answering, in … In Chapter 4, I perform an extensive evaluation of the popular skip-gram word …

Analysis of Images, Social Networks and Texts: 6th International Conference, AIST 2017, Moscow, Russia, July 27–29, 2017, Revised Selected Papers
WMP van der Aalst, DI Ignatov, M Khachay… – 2017 – books.google.com
… from the St. Petersburg Department of the Steklov Mathematical Institute presented a tutorial on “Deep Learning for Natural Language Processing.” The business speakers also covered a wide variety of topics3. We list those …

Decoding naturalistic experiences from human brain activity via distributed representations of words
S Nishida, S Nishimoto – NeuroImage, 2017 – Elsevier
… movie scenes. Then, we constructed a high-dimensional feature space of perceptual experiences using skip-gram, a state-of-the-art distributed word embedding model. We built … 2.6. Skip-gram vector space. The skip-gram algorithm …

Handling long-term dependencies and rare words in low-resource language modelling
M Singh – 2017 – publikationen.sulb.uni-saarland.de
… information for tasks like ASR. Though, for tasks like Question Answering using … long-span information in Skip-gram based models. To compare these different … 3.2 Related Work Apart from Skip-grams-based language models as a long-span information cap …

Methods of sentence extraction, abstraction and ordering for automatic text summarization
MT Nayeem – 2017 – uleth.ca
… We designed a novel abstractive sentence generation model which jointly performs sentence fusion and paraphrasing using the skipgram word embedding model. Fur … same deep learning framework for the same task … This can also be called skip-gram co-occurrence …

Identifying lexical relationships and entailments with distributional semantics
SC Roller – 2017 – repositories.lib.utexas.edu
… and synthesis of knowledge. These include tasks like Question Answering (QA) … gan et al., 2006). Modern, deep learning approaches focus on learning word and … munity, with algorithms like Skip-gram Negative Sampling (SGNS) (Mikolov et …

Incorporating visual information into neural machine translation
I Calixto – 2017 – doras.dcu.ie
… many layers between receiving an input and generating an output. Deep neural networks, also popularly referred to as deep learning models, have brought break- throughs to many different learning tasks by structuring computational models in …

Contextual representation of documents, entities, and faces of people using a news corpus
MA Kader – 2017 – search.proquest.com
… unstructured text. Recently, Mikolov et al. [82] proposed two models { skip-gram and … 12. holistic Fourier feature [7, 70] but those methods do not outperform the recently developed. deep learning-based mechanisms for face recognition [118, 88]. 2.3 Context generation for face …

A review of word embedding and document similarity algorithms applied to academic text
JE Alvarez, H Bast – 2017 – pdfs.semanticscholar.org
… There is work on document and image question answering, where the DNN seems to understand the contents in great detail and can retrieve or generate short snippets that answer very specific details about the media … Page 23. 2.5. Deep Learning 11 …

Thoughts don’t have Colour, do they?: Finding Semantic Categories of Nouns and Adjectives in Text Through Automatic Language Processing
P Fallgren – 2017 – diva-portal.org
… Word2vec has two types of algorithms, continuous bag-of-words (CBOW) and skip-gram (SG), both makes use of a neural network with an objective that tries to predict a word based on its context (CBOW) or predict the context of a target word (SG). Co-occurrence Matrices …

Entity Retrieval and Text Mining for Online Reputation Monitoring
PSS da Cruz – 2017 – repositorio-aberto.up.pt
Page 1. Entity Retrieval and Text Mining for Online Reputation Monitoring Pedro dos Santos Saleiro da Cruz Departamento de Engenharia Informática Faculdade de Engenharia da Universidade do Porto In partial fulfillment of requirements for the degree of …

Trustworthy Mining of Noisy Multi-Source Data: From Methodology to Application
Y Li – 2017 – search.proquest.com
… In the second application about medical question answering, we propose methods to tackle the challenge of finding similar questions and the challenge of capturing semantic meanings, and then … thank Shuang Wu for the daily discussion about deep learning techniques when …

Knowledge graph embedding: A survey of approaches and applications
Q Wang, Z Mao, B Wang, L Guo – IEEE Transactions on …, 2017 – ieeexplore.ieee.org
… such as Freebase [1], DBpedia [2], YAGO [3], and NELL [4], have been created and successfully applied to many real-world applications, from semantic parsing [5], [6] and named entity disambiguation [7], [8], to information extraction [9], [10] and question answering [11], [12] …

Visually grounded interaction and language
F Strub, H de Vries, A Das, S Kottur… – Schedule …, 2017 – pdfs.semanticscholar.org
… We will accept papers related to: —language acquisition or learning through interactions —visual captioning, dialog, and question-answering —reasoning in … of Incentive Compatibility in Display Ad Auctions 04: 30 PM Optimal Economic Design through Deep Learning Parkes 04 …

New Frontiers in Supervised Word Sense Disambiguation: Building Multilingual Resources and Neural Models on a large scale
A Raganato, R Navigli – 2017 – iris.uniroma1.it
… been witnessed a real upsurge of machine learning models in the NLP community, specially exploiting neural networks and deep learning. Traditional approaches are … or none). The best parameters was achieved by the Skip-gram model of Word2Vec …

Discourse-Level Language Understanding with Deep Learning
MN Iyyer – 2017 – drum.lib.umd.edu
… 3 Deep Learning for Multi-Sentence Factoid Question Answering 21 …

The challenging task of summary evaluation: an overview
E Lloret, L Plaza, A Aker – Language Resources and Evaluation, 2017 – Springer
… other summaries from the same source. The core of the approach relies on the probability distributions of the vocabulary in these elements using n-grams (unigrams, bigrams, and skip-grams). In their experiments, the set of …

Computational Natural Language Inference: Robust and Interpretable Question Answering
RR Sharp – 2017 – search.proquest.com
Computational Natural Language Inference: Robust and Interpretable Question Answering. Abstract … In particular, here we present four approaches to question answering, each of which shows a significant improvement in performance over baseline methods …

Deep Learning with Keras
A Gulli, S Pal – 2017 – books.google.com
… Embeddings 140 Distributed representations 141 word2vec 142 The skip-gram word2vec model … RNN variants 213 Summary 214 Chapter 7: Additional Deep Learning Models 215 … deep networks 235 Keras example — memory network for question answering 236 Customizing …

Learning Vector Representations for Sentences
P Lê – dare.uva.nl
… dialog processing [Vinyals and Le, 2015, Wen et al., 2015], language modelling [Mikolov et al., 2010, Jozefowicz et al., 2016], and question answering [Weston et al., 2015, Kumar et al., 2015]. However, the road is not always smooth. The impact of deep learning in reducing the …

A Bootstrapped Approach to Multilingual Text Stream Parsing
N Londhe – 2017 – search.proquest.com
… We explore how the said pipeline can be utilized for two sample use cases – question answering and summarization. Full Text. A BOOTSTRAPPED APPROACH TO MULTILINGUAL. TEXT STREAM PARSING. By. Nikhil Londhe … 75. 4 Applications 77. 4.1 Question Answering …

A Comprehensive Method for Automating Test Collection Creation and Evaluation for Retrieval and Summarization Systems
MP Ekstrand-Abueg – 2017 – search.proquest.com
… document retrieval, it has broadened to video retrieval (TRECVID), image retrieval, question answering, summarization, clinical … In addition to n-gram statistics, ROUGE can utilize skip-grams, n-grams with some … In the rst, known as skip-gram, the current word is used to predict …

Schedule Highlights
P Sturm – Machine Learning, 2017 – pdfs.semanticscholar.org
… Predicting the sequence specificities of DNA-and RNA-binding proteins by deep learning … Learning to Run Webpage: https://www.crowdai.org/challenges/nips-2017-learning-to-run Human-Computer Question Answering Competition Webpage: http://sites.google.com/view/hcqa …

On unsupervised algorithms for semantically interpretative and contextually sensitive text-mining
V Gopalakrishnan – 2017 – search.proquest.com
On unsupervised algorithms for semantically interpretative and contextually sensitive text-mining. Abstract. With the growth of world wide web and large-scale digitisation of data, text-mining has assumed a pivotal role in data analytics …

(Visited 458 times, 1 visits today)