Neural Language Model & Avatars


Notes:

CNN is a type of feed-forward artificial neural network, and variation of multilayer perceptron, designed to minmize preprocessing.

  • Bigram neural language model
  • Class-based neural language model
  • Conditional neural language model
  • Context-aware neural language model
  • Factored neural language model
  • Feature-based neural language model
  • Hierarchal neural language model
  • Log-bilinear neural language model
  • Multimodal neural language model
  • Neural language modeLing
  • Neural language modeLLing
  • Neural network language model
  • Probabilistic neural language model
  • Recurrent neural language model
  • Structure-content neural language model

Resources:

  • korymathewson.com .. kory mathewson is a research scientist with deepmind
  • rwthlm .. a toolkit for feedforward and long short-term memory neural network language modeling

Wikipedia:

References:

See also:

100 Best Language Model VideosIRSTLM (IRST Language Modeling) Toolkit 2018Language Modeling & Chatbots 2019Rule-based Language Modeling


Collaborative storytelling with large-scale neural language models
E Nichols, L Gao, R Gomez – Motion, Interaction and Games, 2020 – dl.acm.org
… We present a collaborative storytelling system that is constructed by tuning a large-scale neural language model on a writing prompts story dataset. • We develop a method for ranking …

Semantic Chat: Enabling Greater Believability through Voice Avatars in Multiplayer and Story-Driven Games
S Chen, A Kipnis, K Mathewson, S Ysebert, B Pietrzak… – 2020 – research.google
… In practice, the application of these neural language models is an open problem, with non-trivial … like transfer learning, we discuss the obstacles in realizing believable voice avatars. …

Better word representations with recursive neural networks for morphology
MT Luong, R Socher, CD Manning – Proceedings of the …, 2013 – aclanthology.org
… pheme is a basic unit, with neural language models (NLMs) to consider contextual information in … By training a neural language model (NLM) and integrating RNN structures for complex …

A persona-based neural conversation model
J Li, M Galley, C Brockett, GP Spithourakis… – arXiv preprint arXiv …, 2016 – arxiv.org
We present persona-based models for handling the issue of speaker consistency in neural response generation. A speaker model encodes personas in distributed embeddings that …

Language Modelling
C Room – algorithms, 2022 – devopedia.org
… How can I train or make use of a neural language model? Pre-train a LM, transfer that … A neural language model can be learned in an unsupervised or semi-supervised manner but it …

Realtoxicityprompts: Evaluating neural toxic degeneration in language models
S Gehman, S Gururangan, M Sap, Y Choi… – arXiv preprint arXiv …, 2020 – arxiv.org
… study only on neural language models, and therefore use the term “neural toxic degeneration.” Future work could examine whether non-neural language models exhibit similar behavior…

Zero-Shot Recommendation as Language Modeling
D Sileo, W Vossen, R Raymaekers – European Conference on Information …, 2022 – Springer
… Neural language models are trained over a large corpus of documents: to train a neural network, its parameters \(\varTheta \) are optimized for next word prediction likelihood …

Grounded language learning fast and slow
F Hill, O Tieleman, T Von Glehn, N Wong… – arXiv preprint arXiv …, 2020 – arxiv.org
… Recent work has shown that large text-based neural language models acquire a surprising propensity for one-shot learning. Here, we show that an agent situated in a simulated 3D …

Measuring depression symptom severity from spoken language and 3D facial expressions
A Haque, M Guo, AS Miner, L Fei-Fei – arXiv preprint arXiv:1811.08592, 2018 – arxiv.org
… This corpus is created from semi-structured clinical interviews where a patient speaks to a remote-controlled digital avatar. The clinician, through the digital avatar, asks a series of …

Procedural generation of branching quests for games
ES de Lima, B Feijó, AL Furtado – Entertainment Computing, 2022 – Elsevier
… [14], who present a framework to generate cooking quests for text-adventure games that uses Markov chains and a neural language model to generate recipes. Their system uses a …

Sex and gender bias in natural language processing
D Cirillo, H Gonen, E Santus, A Valencia… – Sex and Gender Bias in …, 2022 – Elsevier
… Statistical language models and neural language models. (A) The n… ) [8] is a neural language model, which can be trained to predict … State-of-the-art neural language models can use …

Designing style matching conversational agents
D Aneja, R Hoegen, D McDuff, M Czerwinski – arXiv preprint arXiv …, 2019 – arxiv.org
… Using a generative neural language model to generate responses. In order to perform … As such, we used a generative neural language model approach to generate the answers. This …

Interactive image manipulation with natural language instruction commands
S Shinagawa, K Yoshino, S Sakti, Y Suzuki… – arXiv preprint arXiv …, 2018 – arxiv.org
We propose an interactive image-manipulation system with natural language instruction, which can generate a target image from a source image and an instruction that describes the …

An end-to-end conversational style matching agent
R Hoegen, D Aneja, D McDuff… – Proceedings of the 19th …, 2019 – dl.acm.org
We present an end-to-end voice-based conversational agent that is able to engage in naturalistic multi-turn dialogue and align with the interlocutor’s conversational style. The system …

Show, Don’t (Just) Tell: Embodiment and Spatial Metaphor in Computational Story-Telling.
P Wicke, T Veale – ICCC, 2020 – computationalcreativity.net
To a human storyteller, a story is more than a textual artifact. Rather, as stories are both generated and generative, each is also a blueprint for performances to come. Tellers must draw …

Neural information retrieval: At the end of the early years
KD Onal, Y Zhang, IS Altingovde, MM Rahman… – Information Retrieval …, 2018 – Springer
… the first to propose a neural language model; they introduce the … We survey the use of neural language models and word … with a foundation on neural language models and with pointers …

Embodied Multimodal Agents to Bridge the Understanding Gap
N Krishnaswamy, N Alalyani – … of the First Workshop on Bridging …, 2021 – aclanthology.org
… As of the 2020s, high-profile NLP successes are being driven by large (and ever-growing) deep neural language models1. These models perform impressively according to common …

Collaborative Storytelling with Human Actors and AI Narrators
B Branch, P Mirowski, KW Mathewson – arXiv preprint arXiv:2109.14728, 2021 – arxiv.org
… Avatar for the AI narrator We designed a virtual avatar that personified the AI narrator. That avatar consisted of a 3D model of a robot, inspired by Aldebaran Robotics’ Nao, built using …

Improvised theatre alongside artificial intelligences
KW Mathewson, P Mirowski – Thirteenth Artificial Intelligence and Interactive …, 2017 – aaai.org
This study presents the first report of Artificial Improvisation, or improvisational theatre performed live, on-stage, alongside an artificial intelligence-based improvisational performer. The …

Collaborative Storytelling with Human Actors and AI Narrators Paper type: Event Report
B Branch, P Mirowski, K Mathewson – academia.edu
… Avatar for the AI narrator We designed a virtual avatar that personified the AI narrator. That avatar consisted of a 3D model of a robot, inspired by Aldebaran Robotics’ Nao, built using …

Hyperparameter selection for offline reinforcement learning
TL Paine, C Paduraru, A Michi, C Gulcehre… – arXiv preprint arXiv …, 2020 – arxiv.org
… humanoid avatar, from visuals provided by an egocentric camera controlled by the policy. … a 56 degrees of freedom humanoid avatar from visuals provided by an egocentric camera. …

Getting started with neural models for semantic matching in web search
KD Onal, IS Altingovde, P Karagoz… – arXiv preprint arXiv …, 2016 – arxiv.org
… We survey the use of neural language models and word … a foundation of on neural language models and with pointers to … background information on neural language models in Section …

Text entry in virtual environments using speech and a midair keyboard
J Adhikary, K Vertanen – IEEE Transactions on Visualization …, 2021 – ieeexplore.ieee.org
… For this we used the bidirectional neural language model BERT [7]. BERT has proven useful in a range of natural language processing tasks where an entire sentence is available at …

Collaborative Storytelling with Social Robots
E Nichols, L Gao, Y Vasylkiv… – 2021 IEEE/RSJ …, 2021 – ieeexplore.ieee.org
… [20] conduct an in-depth analysis of the storytelling capabilities of large-scale neural language models. However, in these works story generation is conducted without human interaction…

Changing the Narrative Perspective: A New Language Processing Task and Machine Learning Approaches
M Chen – 2022 – rave.ohiolink.edu
In this dissertation I introduce the novel text processing task of changing the narrative perspective, where characters in a story are assigned a point of view that is di erent from the one …

On visual coreference chains resolution
S Dobnik, S Loáiciga – Thanks to our sponsors: Gold: Stora …, 2018 – diva-portal.org
… in a 3-d modelling software and two avatars have been placed at the opposite side of this table representing the conversation participants. A third avatar who is a passive observer of the …

Investigation of the use of Deep Learning and emotion detection for the improvement of Text-based medical conversational agent
B Yuana, H Aflib – Proceedings http://ceur-ws. org ISSN, 2021 – check.cerc-conf.eu
… With the development of neural language models such as word vectors , paragraph vectors , and GloVe , the transfer learning (pre-training and fine-tuning) revolution started in NLP. …

Spoken SQuAD: A study of mitigating the impact of speech recognition errors on listening comprehension
CH Li, SL Wu, CL Liu, H Lee – arXiv preprint arXiv:1804.00320, 2018 – arxiv.org
Reading comprehension has been widely studied. One of the most representative reading comprehension tasks is Stanford Question Answering Dataset (SQuAD), on which machine is …

Offline Hyperparameter Selection For Offline Reinforcement Learning
T Le Paine, C Paduraru, A Michi, C Gulcehre, K Zo?na… – offline-rl-neurips.github.io
… humanoid avatar, from visuals provided by an egocentric camera controlled by the policy. … a 56 degrees of freedom humanoid avatar from visuals provided by an egocentric camera. …

Conversational error analysis in human-agent interaction
D Aneja, D McDuff, M Czerwinski – Proceedings of the 20th ACM …, 2020 – dl.acm.org
Conversational Agents (CAs) present many opportunities for changing how we interact with information and computer systems in a more natural, accessible way. Building on research …

Medical instructed real-time assistant for patient with glaucoma and diabetic conditions
UU Rehman, DJ Chang, Y Jung, U Akhtar… – Applied Sciences, 2020 – mdpi.com
Virtual assistants are involved in the daily activities of humans such as managing calendars, making appointments, and providing wake-up calls. They provide a conversational service …

Sound analogies with phoneme embeddings
M Silfverberg, LJ Mao, M Hulden – Proceedings of the Society for …, 2018 – aclanthology.org
… ) investigate how well phonetic feature representations in English align with vector representations learned from local contexts of sound occurrence using both a neural language …

Text as Causal Mediators: Research Design for Causal Estimates of Differential Treatment of Social Groups via Language Aspects
KA Keith, D Rice, B O’Connor – arXiv preprint arXiv:2109.07542, 2021 – arxiv.org
… could customize avatars of the advocates? We note, using computer-mediated avatars to signal … Causal analysis of syntactic agreement mechanisms in neural language models. In ACL…

Using phoneme representations to build predictive models robust to asr errors
A Fang, S Filice, N Limsopatham… – Proceedings of the 43rd …, 2020 – dl.acm.org
… for the task of speech-driven talking avatar synthesis to create more realistic and … CharacterAware Neural Language Models.. In Proceedings of the Association for the Advancement …

Speech Recognition Error Prediction Approaches with Applications to Spoken Language Understanding
P Serai – 2021 – rave.ohiolink.edu
… N-gram or neural language models are obtained via maximum likelihood estimation from large bodies of text and aim at reducing the perplexity on unseen test data. However, when …

Symbolic behaviour in artificial intelligence
A Santoro, A Lampinen, K Mathewson… – arXiv preprint arXiv …, 2021 – arxiv.org
The ability to use symbols is the pinnacle of human intelligence, but has yet to be fully replicated in machines. Here we argue that the path towards symbolically fluent artificial …

Design and Analysis of a Collaborative Story Generation Game for Social Robots
E Nichols, L Gao, Y Vasylkiv, R Gomez – Frontiers in Computer …, 2021 – frontiersin.org
Storytelling plays a central role in human socializing and entertainment, and research on conducting storytelling with robots is gaining interest. However, much of this research assumes …

Intra-agent speech permits zero-shot task acquisition
C Yan, F Carnevale, P Georgiev, A Santoro… – arXiv preprint arXiv …, 2022 – arxiv.org
Human language learners are exposed to a trickle of informative, context-sensitive language, but a flood of raw sensory data. Through both social language use and internal processes …

Computational gastronomy: A data science approach to food
M Goel, G Bagler – Journal of Biosciences, 2022 – Springer
… As a demonstration, the authors converted a Japanese recipe ‘Sukiyaki’ into its French avatar … This model did not implement any advanced neural language models such as BERT, Elmo …

Text Summarization
C Room – algorithms, 2020 – devopedia.org
… The proposed approach applies a neural language model along with an attention-based input encoder. They experiment with three different encoders: bag-of-words, convolutional (…

Transflower: probabilistic autoregressive dance generation with multimodal attention
G Valle-Pérez, GE Henter, J Beskow… – ACM Transactions on …, 2021 – dl.acm.org
… Their avatar follows their movement using inverse kinematics, anchored to their real body … Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020). Tero …

Dialogue act recognition via crf-attentive structured network
Z Chen, R Yang, Z Zhao, D Cai, X He – The 41st international acm sigir …, 2018 – dl.acm.org
Dialogue Act Recognition (DAR) is a challenging problem in dialogue interpretation, which aims to associate semantic labels to utterances and characterize the speaker’s intention. …

Zipfian environments for Reinforcement Learning
SCY Chan, AK Lampinen, PH Richemond… – arXiv preprint arXiv …, 2022 – arxiv.org
As humans and animals learn in the natural world, they encounter distributions of entities, situations and events that are far from uniform. Typically, a relatively small set of experiences …

Humour-in-the-loop: Improvised Theatre with Interactive Machine Learning Systems
KW Mathewson – 2019 – era.library.ualberta.ca
Improvisation is a form of live theatre where artists perform real-time, dynamic problem solving to collaboratively generate interesting narratives. The main contribution of this thesis is …

Fostering Creative Story Writing Through Personalized Question-Asking Agents
N Hasrati, EJ Soure, BD Zimmerman – ehsanjso.me
… One category of such tools use neural language models to suggest automatically generated … -of-the-art abilities of neural language models as opposed to contributions to pedagogical …

Human instruction-following with deep reinforcement learning via transfer-learning from text
F Hill, S Mokra, N Wong, T Harley – arXiv preprint arXiv:2005.09382, 2020 – arxiv.org
… in terms of subwords (a mix of characters, common word chunks, morphemes and words) rather than the word-level vocabulary applied in more traditional neural language models. …

Investigation of Significant Features for Reviews Helpfulness
A Arshad – 2021 – thesis.cust.edu.pk
… Each token is mapped into a fixed-length real vector ie, embedding because of training neural language models [25], wherein each dimension reflects a latent idea shared through …

Creating multimodal interactive agents with imitation and self-supervised learning
DMIA Team, J Abramson, A Ahuja, A Brussee… – arXiv preprint arXiv …, 2021 – arxiv.org
A common vision from science fiction is that robots will one day inhabit our physical spaces, sense the world as we do, assist our physical labours, and communicate with us through …

AI and the Future of Disinformation Campaigns
CP Brief – 2021 – cset.georgetown.edu
The age of information enabled the age of disinformation. Powered by the speed and volume of the internet, disinformation has emerged as an instrument of strategic competition and …

Commonsense Knowledge for 3D Modeling: A Machine Learning Approach
K Hassani – 2017 – ruor.uottawa.ca
… However, instead of global matrix factorization methods such as singular value decomposition (SVD), word embeddings are learned based on neural language models in which a word …

What does bert know about books, movies and music? probing bert for conversational recommendation
G Penha, C Hauff – Fourteenth ACM Conference on Recommender …, 2020 – dl.acm.org
Heavily pre-trained transformer models such as BERT have recently shown to be remarkably powerful at language modelling, achieving impressive results on numerous downstream …

Research Of The Text Data Vectorization And Classification Algorithms Of Machine Learning
VA Kozhevnikov, ES Pankratova – Theoretical & Applied Science, 2020 – researchgate.net
The article includes information about different classification algorithms and vectorization methods. We give the advantages and disadvantages of classification methods. Also in this …

Co-Generation with GANs using AIS based HMC
T Fang, A Schwing – Advances in Neural Information …, 2019 – proceedings.neurips.cc
Inferring the most likely configuration for a subset of variables of a joint distribution given the remaining ones–which we refer to as co-generation–is an important challenge that is …

LOUIS-PHILIPPE MORENCY
C Point – cs.cmu.edu
• Selected by the World Economic Forum as one of the 40 Young Scientist under the age of 40 to participate alongside business and political leaders in the Annual Meetings of the New …

Breakingnews: Article annotation by image and text processing
A Ramisa, F Yan, F Moreno-Noguer… – IEEE transactions on …, 2017 – ieeexplore.ieee.org
… [23] proposed a system to animate a human avatar based on the emotions inferred from text. And very recently, advanced sentence parsers have been used to extract objects and their …

Language Model Pre-training Improves Generalization in Policy Learning
S Li, X Puig, Y Du, E Akyürek, A Torralba, J Andreas… – 2021 – openreview.net
Language model (LM) pre-training has proven useful for a wide variety of language processing tasks, including tasks that require nontrivial planning and reasoning capabilities. Can …

A deep-learning assisted empathetic guide for selfattachment therapy
L Alazraki – 2021 – doc.ic.ac.uk
… GPT-2 is a neural language model that uses a decoder-only transformer architecture (25). Its smallest version (and the one we will use in our implementation) has 12 layers and 117 …

Changing the narrative perspective: From deictic to anaphoric point of view
M Chen, R Bunescu – Information Processing & Management, 2021 – Elsevier
We introduce the task of changing the narrative point of view, where characters are assigned a narrative perspective that is different from the one originally used by the writer. The …

Understanding Symbolic Communication
E Cheng – 2022 – dspace.mit.edu
We quantitatively study the emergence of symbolic communication in humans with a communication game that attempts to recapitulate an essential step in the development of human …

From partners to populations: A hierarchical Bayesian account of coordination and convention.
RD Hawkins, M Franke, MC Frank, AE Goldberg… – Psychological …, 2022 – doi.apa.org
Languages are powerful solutions to coordination problems: They provide stable, shared expectations about how the words we say correspond to the beliefs and intentions in our heads…

Workshops of the seventh international brain-computer interface meeting: not getting lost in translation
JE Huggins, C Guger, E Aarnoutse… – Brain-computer …, 2019 – Taylor & Francis
… Thus, ECoG can be useful for controlling prosthetic limbs, avatars, or cursors, but can also … ’s behavior to a user or topic, both for traditional as well as neural language models [77–79]. …

Robust Instruction-Following in a Situated Agent via Transfer-Learning from Text
F Hill, S Mokra, N Wong, T Harley – 2019 – openreview.net
… in terms of subwords (a mix of characters, common word chunks, morphemes and words) rather than the word-level vocabulary applied in more traditional neural language models. …

It’s the meaning that counts: the state of the art in NLP and semantics
D Hershcovich, L Donatelli – KI-Künstliche Intelligenz, 2021 – Springer
… Here, an interactive agent is embodied as a dynamic point-of-view or avatar in a proxy situation. Key to this work is encoding an elementary understanding of how objects behave …

Text sequence modeling and deep learning
CC Aggarwal – Machine Learning for Text, 2018 – Springer
… Neural language models use neural networks to encode the grammatical structure of a language from text examples. These models can be used with arbitrary languages and …

Gesture2Vec: Clustering Gestures using Representation Learning Methods for Co-speech Gesture Generation
PJ Yazdian, M Chen, A Lim – 2021 – openreview.net
Co-speech gestures are a principal component in conveying messages and enhancing interaction experiences between humans. Similarly, the co-speech gesture is a key ingredient in …

A Review of Affective Generation Models
G Nie, Y Zhan – arXiv preprint arXiv:2202.10763, 2022 – arxiv.org
… the character rig parameters accordingly for virtual avatar animation. ExprGen was a multistage … Scherer, “Affect-lm: A neural language model for customizable affective text generation,” …

Language Modeling and Deep Learning
CC Aggarwal – Machine Learning for Text, 2022 – Springer
… Neural language models use neural networks to encode the grammatical structure of a language from text examples. These models can be used with arbitrary languages and …

Artificial Intelligence in Education
I Roll, D McNamara, S Sosnovsky, R Luckin… – 2021 – Springer
The 22nd International Conference on Artificial Intelligence in Education (AIED 2021), originally planned for Utrecht, the Netherlands, was held virtually during June 2021. AIED 2021 …

Why is AI hard and Physics simple?
DA Roberts – arXiv preprint arXiv:2104.00008, 2021 – arxiv.org
We discuss why AI is hard and why physics is simple. We discuss how physical intuition and the approach of theoretical physics can be brought to bear on the field of artificial …

Teaching machines to converse
J Li – 2017 – search.proquest.com
The ability of a machine to communicate with humans has long been associated with the general success of AI. This dates back to Alan Turing’s epoch-making work in the early 1950s, …

Machine learning for text
CC Aggarwal – 2018 – Springer
“If it is true that there is always more than one way of construing a text, it is not true that all interpretations are equal.”–Paul Ricoeur The rich area of text analytics draws ideas from …

Advanced Review Helpfulness Modeling
J Du – 2020 – vuir.vu.edu.au
… As a consequence of training neural language models [23], each token is mapped into a fixed-length real vector (ie, embedding), wherein each dimension represents a latent concept …

Survey on frontiers of language and robotics
T Taniguchi, D Mochihashi, T Nagai, S Uchida… – Advanced …, 2019 – Taylor & Francis
The understanding and acquisition of a language in a real-world environment is an important task for future robotics services. Natural language processing and cognitive robotics have …

Imitating interactive intelligence
J Abramson, A Ahuja, I Barr, A Brussee… – arXiv preprint arXiv …, 2020 – arxiv.org
A common vision from science fiction is that robots will one day inhabit our physical spaces, sense the world as we do, assist our physical labours, and communicate with us through …

Conversational information seeking
H Zamani, JR Trippas, J Dalton, F Radlinski – arXiv preprint arXiv …, 2022 – arxiv.org
… However, more recent chatbots have started incorporating spoken interactions, images, and even avatars for creating a more human-like persona. …

Recurrent neural networks
CC Aggarwal – Neural Networks and Deep Learning, 2018 – Springer
… The correctness in predicting the first few words is also helpful in predicting the subsequent words, which are also dependent on a neural language model in the target language. …

Discovering Topic Trends for Conference Analytics
P Liu – 2017 – search.proquest.com
This thesis aims to discover topic trends in an academic conference from multiple kinds of data such as paper abstracts, expert-derived conference tracks and sessions. As the manual …

Modeling Feedback in Interaction With Conversational Agents—A Review
A Axelsson, H Buschmeier, G Skantze – Frontiers in Computer …, 2022 – diva-portal.org
Intelligent agents interacting with humans through conversation (such as a robot, embodied conversational agent, or chatbot) need to receive feedback from the human to make sure …

Main Conference C6 Technology Enhanced Language Learning (TELL)
EINANE CLASS – 27th International Conference on Computers in …, 2019 – ir.canterbury.ac.nz
… promote target language production;(3) virtual representation of language learners (usually in the form of avatars) improve learners’ social presence, alleviate their foreign language …

Domain-Specific Neural Architectures
CC Aggarwal – Artificial Intelligence, 2021 – Springer
The discussion in the previous chapter introduces generic forms of neural architectures. These architectures are fully connected and layered, in the sense that the computational units …

School of Computing Science
I Ounis, R Murray-Smith, J Jose, N Pugeault, P Siebert… – 2022 – 130.209.16.93
… Pretraining large neural language models, such as BERT, has led to impressive gains on many natural language processing (NLP) tasks. However, most pretraining efforts focus on …

Ambient Intelligence for Healthcare
A Haque – 2020 – search.proquest.com
Advances in machine learning and contactless sensors have given rise to ambient intelligence–physical spaces that are sensitive and responsive to the presence of humans. In this …

Sentiment Classification for Hate Tweet Detection in Kenya on Twitter Data Using Naïve Bayes Algorithm
KK Kiilu – 2021 – ir.jkuat.ac.ke
Twitter has flourished to several hundred Million users and could present a rich information source for detecting and classifying hate speech instigator and hate targets using the platform…