Behavior Generation & Virtual Humans 2019 (53x)

Notes:

This text discusses the development of virtual humans, which are computer-based agents that simulate human cognition and behavior. The text mentions various technologies that are used to support the development of virtual humans, including speech recognition, natural language processing, and nonverbal behavior generation. The text also discusses the use of scripting in virtual human development, and the difficulties of generalizing behavior across different types of embodiment. The text also mentions systems that focus on real-time responsive behavior generation in socially interactive agents, and the use of virtual humans in applications such as crowd behavior simulation and conversation. Other topics mentioned include the use of virtual humans for rapport building, multimodal-multisensor interfaces, and the inclusion of characteristics such as beliefs, goals, desires, affect, and personality in behavior generation to increase believability.

Ubiquitous virtual humans: A multi-platform framework for embodied AI agents in XR
A Hartholt, E Fast, A Reilly, W Whitcup… – … and Virtual Reality …, 2019 – ieeexplore.ieee.org
… relevant technologies available in sup- port of developing virtual humans, from individual … researchers and developers with the creation of virtual human conversational characters … recognition [12], natural language processing [15], nonverbal behavior generation [14], nonverbal …

Speech breathing in virtual humans: An interactive model and empirical study
U Bernardet, SH Kanq, A Feng… – … IEEE Virtual Humans …, 2019 – ieeexplore.ieee.org
… AND RESULTS) 4.1 Study Design One of the key questions in the work presented here is, how rele- vant is speech breathing behavior for virtual humans. To assess the influence of speech breathing related behavior on the perception of the virtual human, conducted an …

Virtual human standardized patients for clinical training
T Talbot – Virtual reality for psychological and Neurocognitive …, 2019 – Springer
… Advanced virtual humans can engage in rich conversations (Traum et al … In this regard, Virtual Standardized Patients (VSPs), a specific kind of virtual human conversational agent, can … The response is then sent to a non-verbal behavior generator that selects animations to play …

Embodied conversational agents and interactive virtual humans for training simulators
G Chetty, M White – AVSP 2019 International Conference on …, 2019 – isca-speech.org
… to-face conversations between virtual humans (Cassell, Pelachaud et al., 1994) and between human and a virtual human … “Realtime Animation of Realistic Virtual Humans.” IEEE Computer … Lee, J., Marsella, S: Nonverbal Behavior Generator for Embodied Conversational Agents …

Virtual human questionnaire for analysis of depression, anxiety and personality
S Jaiswal, M Valstar, K Kusumam… – Proceedings of the 19th …, 2019 – dl.acm.org
… frame- work is the Virtual Human Toolkit [10] which was developed for general purpose use. It consisted of a number of modules each spe- cializing in different tasks including speech recognition, natural language processing, emotion recognition, behaviour generation, etc …

Nonverbal behavior in multimodal performances
A Cafaro, C Pelachaud, SC Marsella – The Handbook of Multimodal …, 2019 – dl.acm.org
… Alteration of the tim- ing will change what words are being emphasized and consequently change how the utterance is understood. 6.4 Approaches In this section we describe several approaches that have been adopted for multi- modal behavior generation …

Realistic Group Behaviors Based on Visibility-Voronoi (c) Diagrams
M Li, LZ Zhang, MY Zhang, T Wang… – 5th Annual …, 2019 – atlantis-press.com
… Crowd behavior simulation is an important research direction in the field of computer graphics, which studies virtual human intelligence behavior from … voronoi diagram with the gap value c. In summary, the proposed VV (c) -diagram based group behavior generation process is …

Nonverbal Behavior in
A Cafaro, C Pelachaud… – The Handbook of …, 2019 – books.google.com
… 2014] provides a collec- tion of software modules for building social robots and virtual humans … left) and Brad (right) are the two representative ECAs of the Virtual Human Toolkit … In particular, for behavior generation, it supports incremental behavior plan construction and graceful …

Multiple metaphors in metaphoric gesturing
C Saund, M Roth, M Chollet… – 2019 8th International …, 2019 – ieeexplore.ieee.org
… The implications of this finding for virtual humans is that models may want to creatively select, combine, and … understanding these gestures is to even- tually model and realize them in a virtual human capable of … Nonverbal behavior generator for embodied conversational agents …

AVEC’19: Audio/visual emotion challenge and workshop
F Ringeval, B Schuller, M Valstar, N Cummins… – Proceedings of the 27th …, 2019 – dl.acm.org
… was either controlled by a human (wizard) in another room for the WoZ interviews, or acting in a fully autonomous way, using different au- tomated perception and behaviour generation modules, for … SimSensei kiosk: A virtual human interviewer for healthcare decision support …

Integrated virtual commissioning of a ROS2-based collaborative and intelligent automation system
E Er?s, M Dahl, A Hanna, A Albo… – 2019 24th IEEE …, 2019 – ieeexplore.ieee.org
… can also be included in the commissioning loop by ”immersion”, which represents a real-time mapping from a real to a virtual human … A large amount of data can be collected during immersed commissioning which can later be used to drive behavior generation algorithms [22] …

Perception of virtual characters
E Zell, K Zibrek, R McDonnell – ACM SIGGRAPH 2019 Courses, 2019 – dl.acm.org
… Does my behaviour towards a virtual human change depending on my empathy level towards them … 2011]. Our course is the first to cover perception of virtual humans in a single resource, and addresses much more recent work than Page 2 …

Effects of Virtual Agent Gender on User Performance and Preference in a VR Training Program
X Shang, M Kallmann, AS Arif – Future of Information and Communication …, 2019 – Springer
… Virtual human-like characters are common in virtual worlds, which enable social interaction with … 9. Guadagno, RE, Blascovich, J., Bailenson, JN, Mccall, C.: Virtual humans and persuasion … eye gaze in virtual agents, social robotics and HCI: behaviour generation, user interaction …

Multimodal Analysis and Estimation of Intimate Self-Disclosure
M Soleymani, K Stefanov, SH Kang, J Ondras… – 2019 International …, 2019 – dl.acm.org
… Virtual humans are computer-based agents and have the additional ability to facilitate rapport through coordinated nonverbal displays [15] … Therefore, virtual humans are well positioned to deliver men- tal well-being benefits by leveraging both anonymity and rapport …

Towards more believable characters using personality and emotion
A Shirvani – Proceedings of the AAAI Conference on Artificial …, 2019 – ojs.aaai.org
… There are many characteristics that improve be- lievable behavior generation, including beliefs, goals, desires, affect, and personality … Emotions and affect are other characteristics extensively researched to simulate virtual humans …

Systematic Representative Design and Clinical Virtual Reality
S Mozgai, A Hartholt, AS Rizzo – Psychological Inquiry, 2019 – Taylor & Francis
… Virtual Human (VH) intelligent agents – interactive digital representations of humans who can perceive real humans and respond appropriately, both … In addition, early research outcomes suggest that virtual humans can provide benefits over human-to-human interaction by …

Meet Nadine, one of the world’s most human-like robots
E Baka, M Ramanathan, N Mishra, NM Thalmann – vi-mm.eu
… But this layer can be easily customized to other robots and virtual humans … 1129 – 1141, June 2018. 5. Z. Zhang, A. Beck, and N. Magnenat-Thalmann, “Human-like behavior generation based on head- arms model for robot tracking external targets and body parts,” IEEE Trans …

AIMER: Appraisal Interpersonal Model of Emotion Regulation, Affective Virtual Students to Support Teachers Training
A Delamarre, C Buche, C Lisetti – Proceedings of the 19th ACM …, 2019 – dl.acm.org
… The recent progress in the development of virtual humans (VHs) [21] that can simulate human social behavior, make it possible to build … 2. Non-verbal behavior generation – In order to generate re- alistic non-verbal behaviors such as facial expressions, gaze, head, and body …

Determining iconic gesture forms based on entity image representation
F Nihei, Y Nakano, R Higashinaka, R Ishii – … International Conference on …, 2019 – dl.acm.org
… [15] Jina Lee and Stacy Marsella. 2006. Nonverbal behavior generator for embodied conversational agents … 2015. Cerebella: Au- tomatic Generation of Nonverbal Behavior for Virtual Humans. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelli- gence …

AIVR 2019
G Bruder, M Anderson, A Hiyama, M Kitazaki – computer.org
… Non-Verbal Behavior Generation for Virtual Characters in Group Conversations 41 … 306 Tomer Weiss (University of California, Los Angeles) xi Page 8. Ubiquitous Virtual Humans: A Multi-Platform Framework for Embodied AI Agents in XR 308 …

Development of a Platform for RNN Driven Multimodal Interaction with Embodied Conversational Agents
HH Huang, M Fukuda, T Nishida – Proceedings of the 19th ACM …, 2019 – dl.acm.org
… Multimodal Fuser Listening behavior generation NN model Speaking behavior generation NN model #U … 2018. NADiA: Neural Network Driven Virtual Human Conversation Agents. In Proceedings of the 18th International Conference on Intelligent Virtual Agents (IVA 2018) …

First Impressions Count! The Role of the Human’s Emotional State on Rapport Established with an Empathic versus Neutral Virtual Therapist
H Ranjbartabar, D Richards, A Bilgin… – IEEE transactions on …, 2019 – ieeexplore.ieee.org
… by the goal to increase believability, research has sought to endow virtual humans with emotions … ed with a simple greeting by saying “Hello, I am Effie a virtual human.” After welcoming … Verbal-Behavior-Generator) module containing the line the character will say and for which …

Virtual reality and augmented reality
Y El Miedany – Rheumatology Teaching, 2019 – Springer
… In the development of SimCoach and the SimSensei Kiosk, a Virtual Human Toolkit (VHTK) has been utilized … NPCEditor, a statistical text classification algorithm that selects the character’s responses based on the user’s utterances; the nonverbal behaviour generator (NVBG) a …

Data-driven gaze animation using recurrent neural networks
A Klein, Z Yumak, A Beij… – Motion, Interaction and …, 2019 – dl.acm.org
… In order to simulate convincing gaze movements for a virtual human, one has to first have an understanding … A Review of Eye Gaze in Virtual Agents, Social Robotics and HCI: Behaviour Generation, User Interaction and … Real-time Expressive Gaze Animation for Virtual Humans …

Using machine learning to generate engaging behaviours in immersive virtual environments
GC Dobre – 2019 8th International Conference on Affective …, 2019 – ieeexplore.ieee.org
… All together now: Introducing the virtual human toolkit … Springer, 2006. [17] Jina Lee and Stacy Marsella. Nonverbal behavior generator for embodied conversational agents … On creating multimodal virtual humans-real time speech driven facial gesturing …

M-path: a conversational system for the empathic virtual agent
ÖN Yalç?n, S DiPaola – Biologically Inspired Cognitive Architectures …, 2019 – Springer
… includes a perceptual module that processes the inputs of the system via its sensors, a decision making module, and a behavior generation module that … T, Fast E, Gainer A, Georgila K, Gratch J, Hartholt A, Lhommet M et al (2014) Simsensei kiosk: a virtual human interviewer for …

Toward RNN Based Micro Non-verbal Behavior Generation for Virtual Listener Agents
HH Huang, M Fukuda, T Nishida – International Conference on Human …, 2019 – Springer
… Finally, an agent who does not speak cannot really perform active listening tasks, the non-verbal behavior generation model while the agent is … Wu, J., Ghosh, S., Chollet, M., Ly, S., Mozgai, S., Scherer, S.: NADiA: neural network driven virtual human conversation agents …

Multimodal databases
M Valstar – The Handbook of Multimodal-Multisensor Interfaces …, 2019 – dl.acm.org
… Virtual Human is a digital anthropomorphic representation of a system with which users can interact with. It combines realistic rendering techniques with intelligent behavior to make the virtual human as believable as possible …

Expressive Inverse Kinematics Solving in Real-time for Virtual and Robotic Interactive Characters
T Ribeiro, A Paiva – arXiv preprint arXiv:1909.13875, 2019 – arxiv.org
… By relying heavily on scripting however, it makes it difficult to generalize behaviour to any type of embodiment (most are specifically made for virtual humans), and also makes it difficult to specify certain nuances for particular characters and situations …

AVEC 2019 workshop and challenge: state-of-mind, detecting depression with AI, and cross-cultural affect recognition
F Ringeval, B Schuller, M Valstar, N Cummins… – Proceedings of the 9th …, 2019 – dl.acm.org
… In the WoZ interviews, the virtual agent is controlled by a human inter- viewer (wizard) in another room, whereas in the AI interviews, the agent acts in a fully autonomous way using different automated perception and behaviour generation modules …

Web-based embodied conversational agents and older people
G Llorach, J Agenjo, J Blat, S Sayago – Perspectives on Human-Computer …, 2019 – Springer
… 8.1). Table 8.1 Comparative analysis of different tools to create virtual humans. High-level geometry control. Detailed geometry control. Clothes, hairstyles. Textures. Facial blend shapes and skeleton. Realism. MakeHuman 1.1.0. ++. Ethnicity, age, gender. + …

Non-Verbal Behavior Generation for Virtual Characters in Group Conversations
F De Coninck, Z Yumak, G Sandino… – … Intelligence and Virtual …, 2019 – ieeexplore.ieee.org
… have been studied, however these often result in repetitive and unnatural motions [3]. Recent approaches to non-verbal behavior generation largely focus … For example, during a turn hold, the virtual human directed its gaze away from the ad- dressee during the thematic part of …

An Investigation on the Effectiveness of Multimodal Fusion and Temporal Feature Extraction in Reactive and Spontaneous Behavior Generative RNN Models for …
HH Huang, M Fukuda, T Nishida – … of the 7th International Conference on …, 2019 – dl.acm.org
… 4 RNN MODELS FOR REACTIVE SPONTANEOUS BEHAVIOR GENERATION 4.1 Model configurations We formalize the purpose of the generation task as a regression problem from the variables representing the speaker’s behaviors to the parameters representing the listener …

Emotion recognition from body movement
F Ahmed, ASMH Bari, ML Gavrilova – IEEE Access, 2019 – ieeexplore.ieee.org
Page 1. Received December 1, 2019, accepted December 17, 2019, date of publication December 30, 2019, date of current version January 21, 2020. Digital Object Identifier 10.1109/ACCESS.2019.2963113 Emotion Recognition From Body Movement …

Part-of-speech and prosody-based approaches for robot speech and gesture synchronization
L Pérez-Mayos, M Farrús, J Adell – Journal of Intelligent & Robotic Systems, 2019 – Springer
Page 1. https://doi.org/10.1007/s10846-019-01100-3 Part-of-Speech and Prosody-based Approaches for Robot Speech and Gesture Synchronization L. P érez-Mayos1 · M. Farr ´us1 · J. Adell2 Received: 30 December 2018 …

Challenge discussion: advancing multimodal dialogue
J Allen, E André, PR Cohen, D Hakkani-Tür… – The Handbook of …, 2019 – dl.acm.org
Page 1. 5Challenge Discussion: Advancing Multimodal Dialogue James Allen, Elisabeth André, Philip R. Cohen, Dilek Hakkani-Tür, Ronald Kaplan, Oliver Lemon, David Traum 5.1 Introduction Arguably, dialogue management …

Coalescing Narrative and Dialogue for Grounded Pose Forecasting
C Ahuja – 2019 International Conference on Multimodal …, 2019 – dl.acm.org
… 2006. Nonverbal behavior generator for embodied conversational agents. In International Workshop on Intelligent Virtual Agents. Springer, 243–255 … 2006. Virtual human animation in natural language visualisation. Artificial Intelligence Review 25, 1-2 (2006), 37–53 …

Virtual Movement from Natural
H Sarma – 2019 – d-nb.info
… V VB Verb VHML Virtual Human Markup Language VP Verb Phrase VR Virtual Reality VRML Virtual Reality Modeling Language X … [Ma, 2006]. This system generates animations of virtual humans from single declarative sentences containing an action verb. This system fully …

Socially-Aware Dialogue System
R Zhao – 2019 – lti.cs.cmu.edu
Page 1. Socially-Aware Dialogue System Ran Zhao CMU-LTI-19-008 Language Technologies Institute School of Computer Science Carnegie Mellon University 5000 Forbes Ave., Pittsburgh, PA 15213 www.lti.cs.cmu.edu Thesis …

Audio/Visual Emotion Challenge 2019: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognition
F Ringeval, B Schuller, M Valstar, N Cummins… – 2019 – researchgate.net
… In the WoZ interviews, the virtual agent is controlled by a human inter- viewer (wizard) in another room, whereas in the AI interviews, the agent acts in a fully autonomous way using different automated perception and behavior generation modules …

Commercialization of multimodal systems
PR Cohen, R Tumuluri – The Handbook of Multimodal-Multisensor …, 2019 – dl.acm.org
Page 1. 15Commercialization of Multimodal Systems Philip R. Cohen, Raj Tumuluri 15.1 Introduction This chapter surveys the broad and accelerating commercial activity in build- ing products incorporating multimodal-multisensor interfaces …

Multimodal conversational interaction with robots
G Skantze, J Gustafson, J Beskow – The Handbook of Multimodal …, 2019 – dl.acm.org
Page 1. 2Multimodal Conversational Interaction with Robots Gabriel Skantze, Joakim Gustafson, Jonas Beskow 2.1 Introduction Being able to communicate with machines through spoken interaction has been a long-standing vision in both science fiction and research labs …

Multimodal dialogue processing for machine translation
A Waibel – The Handbook of Multimodal-Multisensor Interfaces …, 2019 – dl.acm.org
Page 1. 14Multimodal Dialogue Processing for Machine Translation Alexander Waibel 14.1 Introduction Humans converse with each other to communicate and to develop ideas interac- tively in the presence of imprecise and under-specified information …

of deliverable First virtual coach design and model
D Simonetti, F Yunus, R Bangalore – 2019 – council-of-coaches.eu
… M Month MFCC Mel-Frequency Cepstral Coefficients MS Milestone NVBG Non-Verbal Behavior Generator OAC •Overall Activity PWR Power … NVBG (Non-Verbal Behavior Generator) (Lee & Marsella, 2006) uses the speech content and the agent’s emotional state as its inputs …

Emotion as an emergent phenomenon of the neurocomputational energy regulation mechanism of a cognitive agent in a decision-making task
M Kirtay, L Vannucci, U Albanese, C Laschi… – Adaptive …, 2019 – journals.sagepub.com
Biological agents need to complete perception-action cycles to perform various cognitive and biological tasks such as maximizing their wellbeing and their chances of genetic continuation. However, …

Responsive joint attention in human-robot interaction
AT Abelho Pereira, C Oertel, L Fermoselle… – 2019 IEEE/RSJ …, 2019 – diva-portal.org
… Several other systems that focus on real-time responsive behavior generation in socially interactive agents [35], [36], [25], [37], [21], have reported good levels of … automating attending behaviors of virtual human characters,” Autonomous Agents and Multi- Agent Systems, vol …

Designing a social robot to support children’s inquiry learning: A contextual analysis of children working together at school
DP Davison, FM Wijnen, J van der Meij… – International journal of …, 2019 – Springer
Page 1. International Journal of Social Robotics https://doi.org/10.1007/s12369-019- 00555-6 Designing a Social Robot to Support Children’s Inquiry Learning: A Contextual Analysis of Children Working Together at School Daniel …

Robot Data-Driven Imitation Learning of Human Social Behavior with Time-Persistent Interaction Context
M Robert Doering – 2019 – ir.library.osaka-u.ac.jp
… and virtual humans Liu et al … These methods rely on hand annotated data or input from human designers, so they are not appropriate for rapid development of robot behaviors to new domains. There also exist data-driven systems for designing behaviors for virtual humans …

Automatic Deictic Gestures for Animated Pedagogical Agents
SRK Kappagantula, N Adamo-Villani… – IEEE Transactions …, 2019 – ieeexplore.ieee.org
Page 1. Automatic Deictic Gestures for Animated Pedagogical Agents Sri Rama Kartheek Kappagantula , Nicoletta Adamo-Villani, Meng-Lin Wu , and Voicu Popescu Abstract—We present a system that automatically generates …

Designing with Gaze: Tama–a Gaze Activated Smart-Speaker
D McMillan, B Brown, I Kawaguchi, R Jaber… – Proceedings of the …, 2019 – dl.acm.org
Page 1. 176 Designing with Gaze Tama – a Gaze-Aware Smart Speaker Platform DONALD MCMILLAN, DSV, Stockholm University, Sweden BARRY BROWN, DSV, Stockholm University, Sweden IKKAKU KAWAGUCHI, University …

DECISION-MAKING FOR AUTONOMOUS CONSTRUCTION VEHICLES
G Marielle, C Sweta – 2019 – diva-portal.org
Page 1. Mälardalen University School of Innovation Design and Engineering Väster?as, Sweden Thesis for the Degree of Master of Science in Intelligent Embedded System – 30.0 credits DECISION-MAKING FOR AUTONOMOUS CONSTRUCTION VEHICLES Marielle Gallardo …

Social Computing and Social Media. Design, Human Behavior and Analytics: 11th International Conference, SCSM 2019, Held as Part of the 21st HCI …
G Meiselwitz – 2019 – books.google.com
… 43 Broderick Crawford, Ricardo Soto, Guillermo Cabrera, Agustín Salas-Fernández, and Fernando Paredes Toward RNN Based Micro Non-verbal Behavior Generation for Virtual Listener Agents …

Social Computing and Social Media. Communication and Social Communities: 11th International Conference, SCSM 2019, Held as Part of the 21st HCI …
G Meiselwitz – 2019 – books.google.com
… 43 Broderick Crawford, Ricardo Soto, Guillermo Cabrera, Agustín Salas-Fernández, and Fernando Paredes Toward RNN Based Micro Non-verbal Behavior Generation for Virtual Listener Agents …