Notes:
SimSensei is an intelligent virtual human (IVH) designed to “listen” to and analyze the speech and nonverbal cues of a person to make determinations about their emotional state. It uses head, facial, and body tracking, as well as the intonation of the person’s speech, to gather information about the person’s level of anxiety and other emotions.
IVHs like SimSensei are often used in healthcare, education, and other fields to provide support and assistance to people in a virtual setting. They can be particularly useful in situations where there may be barriers to accessing in-person support, such as during a pandemic or in remote areas.
Resources:
- cogitocorp.com .. empowers phone professionals to deliver a more engaging and caring customer experience
- icdvrat.org .. International Conference on Disability, Virtual Reality & Associated Technologies
- ict.usc.edu/prototypes/simsensei .. USC ICT pioneering efforts within DARPA DCAPS (Detection and Computational Analysis of Psychological Signals)
Wikipedia:
References:
- Screen Relations: The Limits of Computer-Mediated Psychoanalysis and Psychotherapy (2015)
- Would You Want Therapy From a Computerized Psychologist? (2014)
See also:
100 Best Virtual Human Videos | Affective Computing & Dialog Systems 2014 | Psychopathology & Natural Language Processing 2014 | Virtual Human Toolkit
SimSensei Kiosk: A virtual human interviewer for healthcare decision support D DeVault, R Artstein, G Benn, T Dey, E Fast… – Proceedings of the …, 2014 – dl.acm.org Abstract We present SimSensei Kiosk, an implemented virtual human interviewer designed to create an engaging face-to-face interaction where the user feels comfortable talking and sharing information. SimSensei Kiosk is also designed to create interactional situations … Cited by 25 Related articles All 9 versions
User-state sensing for virtual health agents and telehealth applications. J Gratch, LP Morency, S Scherer, G Stratou, J Boberg… – MMVR, 2013 – ict.usc.edu … 2. SimSensei and TeleCoach Use Cases SimSensei explores the feasibility of virtual health agents for mental health screening. … Figure 1. SimSensei virtual health agent (on left) and Telecoach interface concept (on right) Page 4. … Cited by 12 Related articles All 6 versions
Computers diagnose depression from our body language N Firth – New Scientist, 2013 – Elsevier … The probing questioner is SimSensei, a digital avatar that interviews humans to judge their state of mind. SimSensei is one of several new initiatives designed to partially automate one of the medical profession’s trickiest tasks: diagnosing depression. … Cited by 2 Related articles
SimSensei Demonstration: A Perceptive Virtual Human Interviewer for Healthcare Applications LP Morency, G Stratou, D DeVault… – Twenty-Ninth AAAI …, 2015 – people.ict.usc.edu Abstract We present the SimSensei system, a fully automatic virtual agent that conducts interviews to assess indicators of psychological distress. We emphasize on the perception part of the system, a multimodal framework which captures and analyzes user state for … Related articles All 2 versions
A Demonstration of Dialogue Processing in SimSensei Kiosk F Morbini, D DeVault, K Georgila, R Artstein… – 15th Annual Meeting of …, 2014 – aclweb.org Abstract This demonstration highlights the dialogue processing in SimSensei Kiosk, a virtual human dialogue system that conducts interviews related to psychological distress conditions such as depression, anxiety, and post-traumatic stress disorder (PTSD). The dialogue … Related articles All 11 versions
Detection and Computational Analysis of Psychological Signals Using a Virtual Human Interviewing Agent A Rizzo, S Scherer, D DeVault, J Gratch, R Artstein… – ict.usc.edu … This paper will focus on our current research in these areas within the DARPA- funded “Detection and Computational Analysis of Psychological Signals” project, with specific attention to the SimSensei application use case. SimSensei …
It’s only a computer: the impact of human-agent interaction in clinical interviews J Gratch, GM Lucas, AA King, LP Morency – Proceedings of the 2014 …, 2014 – dl.acm.org … 3.2 The Virtual Interviewer: SimSensei Across all conditions of the experiment, participants engaged in a spoken interaction with the SimeSensei relational VH (Figure 1). System details are described elsewhere (see [23]). … Figure 1. SimSensei, the Virtual Human Interviewer … Cited by 4 Related articles All 7 versions
A Shared, Modular Architecture for Developing Virtual Humans A Hartholt, D Traum, S Marsella, LP Morency… – ict.usc.edu … Combining advanced dialogue management with au- dio-visual sensing results in SimSensei, an agent who aids in recognition of psychological distress [4]. Sim- Sensei enables an engaging face-to-face interaction where the character reacts to the perceived user state and … Related articles
All together now A Hartholt, D Traum, SC Marsella, A Shapiro… – Intelligent Virtual …, 2013 – Springer … An initial inte- gration of FLoReS with the Toolkit has been completed and will be released shortly. An example of a hybrid virtual interviewer / listener is SimSensei, a virtual human platform to aid in recognition of psychological distress [35]. … Cited by 46 Related articles All 7 versions
Ten challenges in highlyinteractive dialog systems NG Ward, D DeVault – AAAI 2015 Spring Symposium, 2015 – cs.utep.edu … For example, the vir- tual interviewer in the SimSensei Kiosk system (DeVault et al. … SimSensei Kiosk: A virtual human interviewer for healthcare decision support. In Pro- ceedings of AAMAS. DeVault, D.; Sagae, K.; and Traum, D. 2009. Can I fin- ish? … Cited by 3 Related articles All 3 versions
Personal Use ONLY JF Morie, E Chance – researchgate.net … A “SimSensei” intelligent virtual human “listens” to what the person says, and analyzes the information from head, facial and body tracking, as well as from the intonation of the speech to make determinations about the person’s level of anxiety. … Related articles
Towards an Affective Interface for Assessment of Psychological Distress GM Lucas, J Gratch, S Scherer, J Boberg, G Stratou – 2015 – schererstefan.net … The DAIC interviews were collected as part of a larger effort named SimSensei to create a virtual agent that Figure 1. MultiSense. … [31] DeVault, D., et al. (2014) Simsensei: A virtual human interviewer for healthcare decision support. … Cited by 1
The “Ultimate Selfie”: Musings on the Future of our Human Identity JF Morie – cc.ist.psu.edu … 2011). Virtual Humans are becoming increasingly attentive to the person with whom they interact, and can seem very responsive and understanding. SIMSensei is one such character being developed at the USC ICT labs. This … Related articles
The Technology-Enabled Patient Advocate: A Valuable Emerging Healthcare Partner SM Kent, P Yellowlees – Telemedicine and e-Health, 2015 – online.liebertpub.com … 33 SimSensei took SimCoach to the next level and added cameras and the Kinect™ sensor (Microsoft, Redmond, WA) to observe and analyze face, gesture, and vocal parameters that help the agent better react to the patient’s emotional state. …
Tangible interfaces for children’s mental healthcare E Shkirando – 2013 – dspace.mah.se … Tracking physical parameters using digital technology for psychology SimSensei (Firth, 2013) is a new program that is able not only to have a real-time conversation with the user but also to recognize his/her facial expression. … Fig. 4, SimSensei, the virtual psychologist … Related articles All 4 versions
Reduced vowel space is a robust indicator of psychological distress: A cross-corpus analysis S Scherer, LP Morency, J Gratch… – Acoustics, Speech and …, 2015 – ieeexplore.ieee.org … and monitoring treatment. In total 253 subjects interacted with the automatic SimSensei 3Sample interaction between the virtual agent and a human actor can be seen here: http://youtu.be/ejczMs6b1Q4 system. Overall, 186 … Cited by 2
Maintaining and Enhancing Human-Surrogate Presence in Augmented Reality K Kim, G Welch – cs.unc.edu … Their “Simsensei Kiosk”—a VH interviewer for aiding in health care decisions—was a fully au- tonomous VH system that could recognize human user’s verbal and nonverbal behaviors, eg, natural language understanding and face detection (including gaze and expression …
Virtuality: VR as metamedia and herald of our future realities RM Lauria, JF Morie – Communication and Technology, 2015 – books.google.com … A very recent example of this from the University of Southern California’s Institute for Creative Technologies (ICT) is a Defense Advanced Research Agency (DARPA) funded research program called SimSensei (ICT 2011). … 2011. Prototype: SimSensei. Available: http://ict. usc. …
ICDVRAT 2014 A AB – 2014 – mariabeatrice.ro Page 1. The 10th International Conference on Disability, Virtual Reality and Associated Technologies Proceedings Edited by: Paul Sharkey Lena Pareto Jurgen Broeren Martin Rydmark 2 to 4 September, 2014 Gothenburg, Sweden … Related articles All 3 versions
Towards a Socially Adaptive Virtual Agent C Pelachaud, M Ochs – … IVA 2015, Delft, The Netherlands, August …, 2015 – books.google.com … However the agents’ behaviour is restricted to the face, and the agent mainly produces backchannels (ie listening behaviour). The SimSensei virtual human interviewer is designed to handle a conversation with users about psychological distress [13]. …
Towards a Socially Adaptive Virtual Agent AB Youssef, M Chollet, H Jones, N Sabouret… – Intelligent Virtual …, 2015 – Springer … However the agents’ behaviour is restricted to the face, and the agent mainly produces backchannels (ie listening behaviour). The SimSensei virtual human interviewer is designed to handle a conversation with users about psychological distress [ 13 ]. …
An Architecture for a Socially Adaptive Virtual Recruiter in Job Interview Simulations A Ben-Youssef, M Chollet, H Jones, N Sabouret… – perso.limsi.fr … However the agents’ behaviour is restricted to the face, and the agent mainly produces backchannels (ie lis- tening behaviour). The SimSensei virtual human interviewer … Simsensei kiosk: a virtual human interviewer for healthcare decision support. … Related articles All 2 versions
The effect of Oculus Rift HMD on postural stability: Proceedings of the 10th international conference on disability, virtual reality and associated technologies (ICDVRAT … AL Brooks, E Petersson – International Conference on Disability, … – forskningsbasen.deff.dk Page 1. Aalborg Universitet Effect of the Oculus Rift head mounted display on postural stability Epure, Paula; Gheorghe, Cristina; Nissen, Thomas; Toader, Laurentiu-Octavian; Nicolae, Alexandru; Nielsen, Steven SM; Juul Rosengreen … Related articles All 2 versions
Opportunities and Challenges of Internet-Based Health Interventions in the Future Internet AK Lim, C Thuemmler – Information Technology-New …, 2015 – ieeexplore.ieee.org … SimSensei which was developed by the University of Southern California (2014) provides real-time tracking and analysis of non-verbal behaviors, including facial expressions, eye gaze, body posture, and voice intonation [33]. …
Self-reported symptoms of depression and PTSD are associated with reduced vowel space in screening interviews S Scherer, G Lucas, J Gratch, A Rizzo, LP Morency – ieeexplore.ieee.org … The interviews were collected as part of a larger effort named SimSensei to create a virtual agent that interviews people and identifies verbal and nonverbal indicators of mental illness [45]. … In total 253 subjects interacted with the automatic SimSensei system. …
[BOOK] Screen Relations: The Limits of Computer-Mediated Psychoanalysis and Psychotherapy GI Russell – 2015 – books.google.com … Remote treatment is likely on an accelerating curve leading to emotionally intelligent programmes running photo-realistic avatars totally indistinguishable from what people encounter on their screens today (for a rudimentary version, just Google “SimSensei and MultiSense”). … Related articles All 2 versions
Virtual World Interview Skills Training for Students Studying Health Professions AJ Campbell, KL Amon, M Nguyen… – … of Technology in …, 2015 – Taylor & Francis … [CrossRef], [PubMed], [Web of Science ®] View all references; DeVault, et al., 201412. DeVault, D., Artstein, D., Benn, G., Dey, T., Fast, E., Gainer, A., & Morency, L.-P. (2014). SimSensei kiosk: A virtual human interviewer for healthcare decision support. … Related articles
Global/Worldwide e-Mental Health: International and Futuristic Perspectives of Telepsychiatry and the Future PM Yellowlees, DM Hilty, D Mucic – e-Mental Health, 2016 – Springer … The virtual therapist model as developed for the SimCoach and SimSensei projects could also be applied to the patient navigator role in future and opens up a whole new style of potentially automated clinical practice, and one where machine-based translation and …
Social Robots and Virtual Humans as Assistive Tools for Improving Our Quality of Life N Magnenat-Thalmann, Z Zhang – Digital Home (ICDH), 2014 …, 2014 – ieeexplore.ieee.org … 329–343, Springer, 2005. [9] D. DeVault, R. Artstein, G. Benn, T. Dey, E. Fast, A. Gainer, K. Georgila, J. Gratch, A. Hartholt, M. Lhommet, et al., “Simsensei kiosk: a virtual human interviewer for healthcare decision support,” in Proceedings of the 2014 international conference on … Related articles All 4 versions
An Eye Tracking Evaluation of a Virtual Pediatric Patient Training System for Nurses TB Pence, LC Dukes, LF Hodges, NK Meehan… – Intelligent Virtual …, 2014 – Springer … Ecological Psychology 7(1) (1995) 11. DeVault, D., Artstein, R., Benn, G., Dey, T., Fast, E., Gainer, A., Georgila, K., Gratch, J., Hartholt, A., Lhommet, M.: Simsensei kiosk: a virtual human inter- viewer for healthcare decision support. … Related articles All 2 versions
Action-Coordinating Prosody NG Ward, S Abu – cs.utep.edu … [20] D. DeVault, R. Artstein, G. Benn, T. Dey, E. Fast, A. Gainer, K. Georgila, J. Gratch, A. Hartholt, M. Lhommet, G. Lucas, S. Marsella, F. Morbini, A. Nazarian, S. Scherer, G. Stratou, A. Suri, D. Traum, R. Wood, Y. Xu, A. Rizzo, and L.-P. Morency, “SimSensei Kiosk: A virtual … Related articles All 2 versions
Dyadic behavior analysis in depression severity assessment interviews S Scherer, Z Hammal, Y Yang, LP Morency… – Proceedings of the 16th …, 2014 – dl.acm.org Page 1. Dyadic Behavior Analysis in Depression Severity Assessment Interviews Stefan Scherer USC Institute for Creative Technologies 12015 Waterfront Dr. Playa Vista, CA scherer@ict.usc.edu Zakia Hammal Carnegie Mellon … Cited by 7 Related articles All 2 versions
Advances in natural language processing J Hirschberg, CD Manning – Science, 2015 – sciencemag.org Cited by 3
A multimodal context-based approach for distress assessment S Ghosh, M Chatterjee, LP Morency – Proceedings of the 16th …, 2014 – dl.acm.org Page 1. A Multimodal Context-based Approach for Distress Assessment Sayan Ghosh*, Moitreya Chatterjee* and Louis-Philippe Morency Institute for Creative Technologies, University of Southern California 12015 E. Waterfront … Cited by 2 Related articles
Gamification to Improve Adherence to Clinical Treatment Advice: Improving Adherence to Clinical Treatment D Richards, PHY Caldwell – Handbook of Research on Holistic …, 2015 – books.google.com … Thus, participants were more comfortable talking about personal matters with a VH. The SimSensei VH used by Gratch et al. (2014) was able to achieve increased disclosure due to a feeling of being unobserved and rapport. …
Automatic audiovisual behavior descriptors for psychological disorder analysis S Scherer, G Stratou, G Lucas, M Mahmoud… – Image and Vision …, 2014 – Elsevier We investigate the capabilities of automatic audiovisual nonverbal behavior descriptors to identify indicators of psychological disorders such as depression, an. Cited by 9 Related articles All 2 versions
Exploring feedback strategies to improve public speaking: an interactive virtual audience framework M Chollet, T Wörtwein, LP Morency, A Shapiro… – Proceedings of the …, 2015 – dl.acm.org Page 1. Exploring Feedback Strategies to Improve Public Speaking: An Interactive Virtual Audience Framework Mathieu Chollet?1, Torsten Wörtwein†1, Louis-Philippe Morency‡, Ari Shapiro? , Stefan Scherer? ? Institute for …
Eye on the Prize D Palm – planarkitekterna.se Page 1. Master Thesis Computer Science Thesis no: MCS-2013-09 09 2013 Eye on the Prize Enhancing Realism during Interaction towards Non-Player-Characters with Natural Eye Movements Daniel Palm School of Computing … Related articles All 7 versions
A data-driven passing interaction model for embodied basketball agents D Lala, T Nishida – Journal of Intelligent Information Systems, 2015 – Springer Page 1. J Intell Inf Syst DOI 10.1007/s10844-015-0386-z A data-driven passing interaction model for embodied basketball agents Divesh Lala1 ·Toyoaki Nishida1 Received: 5 December 2014 / Revised: 13 September 2015 …
Automatic Audiovisual Behavior Descriptors for Psychological Disorder Analysis ASR Gratcha, LP Morencya – researchgate.net Page 1. Automatic Audiovisual Behavior Descriptors for Psychological Disorder Analysis Stefan Scherera, Giota Stratoua, Gale Lucasa, Marwa Mahmouda,b, Jill Boberga, Jonathan Gratcha, Albert (Skip) Rizzoa, Louis-Philippe Morencya … Related articles
Variational Infinite Hidden Conditional Random Fields K Bousmalis, SP Zafeiriou, LP Morency, M Pantic… – ieeexplore.ieee.org Page 1. 0162-8828 (c) 2013 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This … Related articles All 5 versions
A Study of Multimodal Addressee Detection in Human-Human-Computer Interaction TJ Tsai, A Stolcke, M Slaney – 2015 – ieeexplore.ieee.org Page 1. 1520-9210 (c) 2015 IEEE. Personal use is permitted, but republication/ redistribution requires IEEE permission. See http://www.ieee.org/ publications_standards/publications/rights/index.html for more information. This …