Emotion-Oriented Systems: The Humaine Handbook


Emotion-Oriented Systems: The Humaine Handbook (2011) .. edited by Paolo PettaCatherine PelachaudRoddy Cowie


Table of Contents

Preface … xiii

PART 1: FOUNDATIONS … 1

Chapter 1. Contemporary Theories and Concepts in the Psychology of Emotions … 3
Géraldine COPPIN and David SANDER

1.1. Introduction … 3

1.2. Emergence of a scientific approach to emotions … 4
1.2.1. The emotional sequence: James-Lange versus Cannon-Bard … 4
1.2.2. Schachter’s two-factor theory … 6

1.3. Basic emotions theories … 7

1.3.1. Premises of basic emotions theories … 7
1.3.2. Characteristics of basic emotions … 8
1.3.3. Criticisms of basic emotions theories … 10

1.4. Bi-dimensional theories of emotion … 11

1.4.1. Premises of bi-dimensional theories of emotion … 11
1.4.2. Criticisms of bi-dimensional theories of emotion … 12

1.5. Appraisal theories of emotions … 14

1.5.1. Premises of appraisal theories of emotion … 14
1.5.2. Specific models of this theoretical trend … 15
1.5.3. Criticisms of appraisal theories of emotion … 18

1.6. Conclusion … 19

1.7. Glossary … 20

1.8. Bibliography … 21

Chapter 2. Emotion and the Brain … 33
Andy CHRISTEN and Didier GRANDJEAN

2.1. Introduction … 33

2.1.1. Emotions and the brain: the emergence of affective neuroscience as an independent discipline … 34

2.2. The major role of affective neuroscience in understanding emotions … 35

2.2.1. Emotion and the brain: from a unitary entity to processing, from structure to neural networks … 36
2.2.2. Levels of processing in emotional processes … 38
2.2.3. Emotion and cognition … 39

2.3. The historical and conceptual legacy of early conceptions of emotions and the brain … 40

2.3.1. Forerunners of affective neuroscience … 40
2.3.1.1. Charles Darwin … 40
2.3.1.2. The James-Lange peripheralist theory … 41

2.4. Initial neuro-anatomical emotion theories … 41

2.4.1. Canon-Bard’s centralist theory … 41
2.4.2. Papez’s circuit … 42
2.4.3. MacLean’s limbic theory … 43

2.5. Structures in the brain and their functions in emotional processes … 44

2.5.1. Amygdala … 44
2.5.2. Amygdala and emotional learning processes … 45
2.5.2.1. Amygdala and classical conditioning … 45
2.5.2.2. The amygdala: a structure with two processing streams … 46
2.5.3. The amygdala and emotional perception: hypotheses around the specificity of processing within the amygdala … 48
2.5.3.1. The amygdala as a fear module … 48
2.5.3.2. The amygdala and arousal … 50
2.5.3.3. The amygdala as a relevance detector … 51
2.5.4. The amygdala and memory processing … 52

2.6. The prefrontal cortex … 53

2.6.1. The prefrontal cortex and bodily signals … 55
2.6.2. The prefrontal cortex and the top-down regulation of behavior … 56
2.6.3. The prefrontal cortex and the motivational component of emotion … 57

2.7. The anterior cingulate cortex … 58

2.8. The role of the insula in disgust … 58

2.9. Temporal dynamic of brain processes in emotional genesis … 59

2.10. Functional connectivity … 60

2.10.1. Investigations of the connectivity using brain imaging techniques (MRI) … 60

2.10.2. Investigations into connectivity using electroencephalographic (EEG) techniques … 61
2.10.3. Benefits of brain connectivity studies … 62

2.11. Conclusion … 63

2.12. Bibliography … 64

PART 2: NON-VERBAL BEHAVIOR … 77

Chapter 3. Emotional Corpora: from Acquisition to Modeling … 79
Laurence DEVILLERS and Jean-Claude MARTIN

3.1. Introduction … 79

3.2. Building corpora: “acted”, “induced” and real-life emotions … 81

3.2.1. Acted data … 82
3.2.2. Induced data … 83
3.2.3. Real-life data … 84
3.2.4. Comparison of different types of data … 85

3.3. Current emotional corpora … 86

3.4. Coding schemes … 86

3.4.1. Emotional annotation protocols … 89
3.4.1.1. Definitions of annotated features … 89
3.4.1.2. Annotation … 90
3.4.2. Annotating context … 91
3.4.2.1. Validation … 92

3.5. Complex emotions in spontaneous data … 93

3.6. Applications for corpora … 97

3.6.1. Detecting and deciphering emotions in speech … 97
3.6.2. Designing an expressive agent from corpora … 98

3.7. Conclusion … 100

3.8. Bibliography … 101

Chapter 4. Visual Emotion Recognition: Status and Key Issues … 107
Alice CAPLIER

4.1. Introduction … 107

4.2. What is a facial expression? … 109

4.2.1. Definition … 109
4.2.2. Description … 109
4.2.3. Ekman’s universal expressions … 111
4.2.4. An ideal system … 112

4.3. Overview of facial expression recognition methods … 112

4.3.1. Databases … 112
4.3.2. Preprocessing: extracting the face … 114
4.3.3. Extracting facial characteristics … 115
4.3.4. Classification … 117
4.3.5. Performance … 117

4.4. Spontaneous facial expressions … 118

4.4.1. Position of the problem … 118
4.4.2. Databases … 119
4.4.3. Recognizing spontaneous expressions … 122

4.5. Expression intensity … 124

4.6. Dynamic analysis … 126

4.7. Multimodality … 128

4.8. Conclusion … 131

4.9. Bibliography … 132

Chapter 5. Recognition of Acoustic Emotion … 139
Chloé CLAVEL and Gaël RICHARD

5.1. Introduction … 139

5.2. Principles of automatic emotion-recognition systems … 140

5.3. Acoustic descriptors … 141

5.3.1. Voiced versus unvoiced content … 142
5.3.2. A temporal unit for emotional analysis … 143
5.3.3. Prosodic descriptors … 143
5.3.3.1. Fundamental frequency (pitch) … 144
5.3.3.2. Intensity … 145
5.3.3.3. Rhythm descriptors … 146
5.3.4. Voice quality descriptors … 146
5.3.4.1. Normalized amplitude quotient … 147
5.3.4.2. Frequency modulation (jitter) … 147
5.3.4.3. Amplitude modulation (shimmer) … 148
5.3.4.4. Rate of unvoiced windows … 148
5.3.4.5. Harmonic to noise ration … 148
5.3.5. Cepstral and spectral descriptors … 149
5.3.5.1. Formant parameters … 149
5.3.5.2. Mel-frequency cepstral coefficients … 149
5.3.5.3. Bark band energy … 150
5.3.5.4. Spectral centroide … 151

5.4. Automatic emotion classification … 151

5.4.1. Choosing descriptors … 152
5.4.1.1. Normalizing descriptors … 152
5.4.1.2. Reduction in data representation space … 152
5.4.2. Learning algorithms … 154
5.4.2.1. Separators with vast margin … 154
5.4.2.2. Gaussian mixture models … 155

5.5. Performance and assessment … 157

5.5.1. First factor: data and classes of emotions … 157
5.5.2. Second factor: the problem of “ground truths” … 159
5.5.3. Third factor: manual preprocessing … 160
5.5.4. Fourth factor: learning algorithms … 160
5.5.5. Fifth factor: learning conditions … 161

5.6. Conclusion … 161

5.7. Bibliography … 163

Chapter 6. Modeling Facial Expressions of Emotions … 169
Sylwia Julia HYNIEWSKA, Radoslaw NIEWIADOMSKI and Catherine PELACHAUD

6.1. Expressive conversational agents … 169

6.2. Expressions and their emotional states … 170
6.2.1. Expressing discrete emotions … 171
6.2.2. Dimensional approaches to emotional expression … 171
6.2.3. Componential expression of emotions … 173

6.3. Computational models for facial expressions of emotions … 174

6.3.1. A discrete representation of facial expressions … 174
6.3.2. Dimensional representation of facial expressions … 174
6.3.3. Componential approaches to facial expressions … 176
6.3.4. Mixtures of emotions and social constraints … 178
6.3.5. Sequences of emotional expressions … 181

6.4. Conclusion … 183

6.5. Acknowledgements … 184

6.6. Bibliography … 184

Chapter 7. Emotion Perception and Recognition … 191
Ioana VASILESCU

7.1. Introduction … 191

7.2. Perception in vocal communication of emotion … 193

7.3. Experimental paradigms and emotion-oriented automatic systems … 194

7.3.1. Experiments validating emotional content and/or annotation strategies … 195
7.3.1.1. Experimental design and stimuli … 195
7.3.1.2. Test populations … 196
7.3.1.3. Perceptual categorization into emotional classes … 196
7.3.1.4. Choosing emotional labels … 197
7.3.1.5. Discussion … 199
7.3.2. Tests for validating measurable parameters of emotional information … 202
7.3.2.1. Discussion … 205
7.3.3. Tests comparing human and automated emotion recognition … 205
7.3.3.1. Discussion … 208

7.4. Conclusion … 208

7.5. Bibliography … 209

PART 3: FUNCTIONS … 215

Chapter 8. The Role of Emotions in Human-Machine Interaction … 217
Valérie MAFFIOLO and Magalie OCHS

8.1. Introduction … 217

8.2. Interactive information and assistance systems … 219

8.2.1. Uses of emotions in interactive systems … 220
8.2.2. Current research and tools … 220
8.2.2.1. Collecting and annotating expressions of emotions … 221
8.2.2.2. Automatic emotion recognition … 222
8.2.2.3. Interface expressivity … 223

8.3. Video games … 227

8.3.1. The importance of emotions in video games … 228
8.3.1.1. Emotions in gaming systems … 228
8.3.1.2. Player emotions … 229
8.3.2. Current research and tools … 231
8.3.2.1. Towards believable emotional virtual characters … 231
8.3.2.2. Towards games that account for player emotions … 233

8.4. Intelligent tutoring systems (ITS) … 235

8.4.1. The importance of emotions in ITS … 235
8.4.1.1. The learner’s emotions … 235
8.4.1.2. Emotions in ITS … 235
8.4.2. Current research and tools … 235

8.5. Discussion and research perspectives … 237

8.6. Bibliography … 238

Chapter 9. Music and Emotions … 247
Donald GLOWINSKI and Antonio CAMURRI

9.1. The growing importance of music in society … 247

9.2. Recognizing emotions and structural characteristics in music … 249

9.2.1. Understanding listeners’ emotional reactions … 249
9.2.2. A categorical or dimensional approach? … 250

9.3. Rules for modeling musical expression of emotions … 251

9.4. Towards a continuous measure of emotional reactions to music … 252

9.5. Multimodality in musical experience … 253

9.5.1. A multimodal research platform for musical expression … 255
9.5.1.1. Level 1 … 257
9.5.1.2. Level 2 … 258
9.5.1.3. Level 3 … 259
9.5.1.4. Level 4 … 259
9.5.1.5. Overview … 259

9.6. Multimodal emotional synthesis in a musical context … 260

9.7. The social active listening paradigm: the collective aspect of emotion … 262

9.7.1. Example: Mappe per Affetti Erranti … 262

9.8. Conclusion and perspectives … 263

9.9. Bibliography … 263

Chapter 10. Literary Feelings in Interactive Fiction … 271
Marc CAVAZZA and David PIZZI

10.1. Introduction: emotions and feelings … 271

10.2. French novels and the representation of feelings … 273

10.3. Madame Bovary: plot and scenes … 275

10.4. Interactive fiction and emotional planning … 280

10.5. Linguistic interaction and emotions … 284

10.6. Emma Bovary’s virtuality … 290

10.7. Conclusion … 294

10.8. Bibliography … 295

Chapter 11. The Design of Emotions: How the Digital is Making Us More Emotional … 299
Annie GENTÈS

11.1. Representing, interpreting and evoking emotions … 299

11.2. Emotion, mimicry and technical devices … 301

11.2.1. Representing emotions and catharsis … 301

11.3. Devices as an alternate source of emotion: photography … 301

11.4. Art and computers: formal beginnings … 303

11.5. The human behind the mechanics and the mechanics behind the human … 305

11.6. Mirror interaction as an emotional vehicle … 307

11.7. Trompe l’œil versus explicit expression … 309

11.8. Three-dimensional universes: an empathetic experience … 311

11.9. Empathy and identifying emotions … 315

11.9.1. From empathy to shared emotions … 315

11.10. Making human?machine interaction and dialog effective … 317

11.11. Conclusion: “revenge of the emotions” … 318

11.12. Bibliography … 318

List of Authors … 321

Index … 325

(Visited 3 times, 1 visits today)