Robots that Talk and Listen: Technology and Social Impact


Robots that Talk and Listen: Technology and Social Impact (2015) .. edited by Judith Markowitz


Contents

List of contributing authors … XIII
Preface … 1

Part I: Images

Steve Mushkin
My robot … 5
I Introduction … 5
II Generative-research methodology … 6
III What children want from technology … 6

  • A Methodology … 7
  • B Findings and discussion … 7

IV Robots … 10

  • A Prior research … 10
  • B Robot study … 12

V Conclusions … 18

Judith Markowitz
Cultural icons … 21
I Introduction … 21
II Robot as killer … 22

  • A The androids of R.U.R. … 23
  • B The frankenstein monster … 26
  • C Managing killer robots … 29

III Robot as servant … 31

  • A Karakuri ningyo … 32
  • B Golem … 34

IV Robot as lover … 39

  • A Pygmalion’s statue … 39
  • B Robots loving humans … 43
  • C Managing robot love … 45

V Conclusion … 46

David F. Dufty
Android aesthetics: Humanoid robots as works of art … 55
I Introduction … 55
II The history of automata for art and entertainment … 56

  • A Automata in ancient times … 57
  • B Clockmaking and automata … 57
  • C Karakuri ningyo … 58
  • D Animatronics … 58
  • E An android as a self portrait: the work of Hiroshi Ishiguro … 59
  • F Hanson robotics … 60

III The Philip K. Dick android … 61
IV Robot components … 62
V Building a dialogue system … 63

  • A Gricean maxims … 63
  • B Architecture … 64
  • C Conversing like Philip K. Dick … 65

VI Conversational competence … 66

  • A Be orderly … 66
  • B Background noise … 68
  • C Cooperative principle … 68
  • D Competence … 69

VII The scientific value of androids … 71
VIII The value of androids as art … 72
IX The uncanny valley: a possible obstacle to artistic androids … 73
X Consciousness … 74
XI Conclusion … 75

Part II: Frameworks and Guidelines

Bilge Mutlu, Sean Andrist, and Allison Sauppé
Enabling human-robot dialogue … 81
I Introduction … 81
II Review of the literature … 82

  • A Multimodal, multiparty dialogue … 83
  • B Situated interaction … 84
  • C Joint action … 85
  • D Linguistic and nonverbal effectiveness … 87
  • E Adaptive dialogue … 88

III A framework for human-robot dialogue-systems … 89

  • A Multimodal language processing … 90
  • B Domain processing … 90
  • C Task model … 90
  • D Dialogue model … 91
  • E Multimodal production … 91
  • F Adaptive dialogue … 92

IV Enabling effective human-robot dialogue … 93

  • A Task model for instruction and repair … 93
  • B A production model for expert robot speech … 103
  • C Summary … 115

V Opportunities and challenges for future work … 116

  • A Linking task and dialogue models … 116
  • B Development of reusable models … 117
  • C Open sharing of models and components … 117

VI Conclusion … 118

Fumiko Nazikian
Robots can talk – but can they teach? … 125
I Introduction … 125
II Androids in the classroom … 126
III Foreign-language teaching … 128

  • A The evolution of foreign-language teaching … 129
  • B The ACTFL guidelines … 130
  • C Assessing accuracy in Japanese … 132

IV Robots and the ACTFL guidelines … 133
V Identifying the difficulties facing Japanese-language learners … 134

  • A Learning sounds and prosody … 134
  • B Grammar … 136
  • C Sociocultural aspects … 139
  • D Pragmatic strategies … 141
  • E Can a speech-enabled robot teach? … 142

VI Global communication and the intercultural speaker … 143

  • A Dialogue and the intercultural speaker … 144
  • B Robots as an intercultural link … 144

VII Conclusion … 145

Nicole Mirnig and Manfred Tscheligi
Comprehension, coherence and consistency: Essentials of Robot Feedback … 149
I Introduction … 149
II Prior work … 152
III A framework for human-robot interaction … 154

  • A Introduction … 154
  • B Robot feedback … 154
  • C Mental models … 156
  • D Three basic principles … 158

IV Conclusion … 166

Part III: Learning

Jonathan H. Connell
Extensible grounding of speech for robot instruction … 175
I Introduction … 175

  • A Eldercare as a domain … 175
  • B Language and learning … 177
  • C Cultural bootstrapping … 179

II Grounding substrate … 180

  • A Object finding … 181
  • B Object properties … 183
  • C Gesture recognition … 185
  • D Speech interpretation … 187
  • E Manipulation routines … 189

III Demonstration of abilities … 191

  • A Scene understanding … 192
  • B Object naming … 193
  • C Semantic web access … 195
  • D Procedure learning … 197

IV Adding motivation … 199
Alan R. Wagner
Lies and deception: Robots that use falsehood as a social strategy … 203
I Introduction … 203
II Prior work … 205
III Basic elements … 206
IV Framework … 208

  • A Representing an interaction … 208
  • B Outcome-matrix transformation … 209
  • C Stereotyping … 211

V Implementation … 213

  • A Examining the factors influence the decision to lie … 216
  • B Using stereotypes and partner modeling to predict the cost of lying … 218

VI Summary and future work … 222
VII Conclusion … 223

Joerg C. Wolf and Guido Bugmann
Robotic learning from multimodal instructions: a card game case study … 227
I Introduction … 227
II Related work … 228
III Human-to-human instruction … 228
IV System components: instructor input … 231

  • A Overview … 231
  • B Speech … 231
  • C Non-verbal input … 233
  • D Multimodal integration … 235
  • E Temporal and semantic integration … 237

V Robot agent learning … 238

  • A Overview … 238
  • B Rule frames … 238
  • C Action selection at execution time … 240
  • D Mapping issues … 240

VI Dialogue management (DM) … 242
VII System evaluation … 243

  • A Approach … 243
  • B Experiment 1: dealing instruction … 244
  • C Experiment 2: teaching four rules … 247
  • D Errors per rule … 249

VIII Discussion of errors … 251

  • A Human error … 251
  • B Dialogue errors … 251
  • C Grammar coverage … 252
  • D Manipulation recognition and multimodal integration … 252

IX Summary and conclusions … 253

  • A Summary … 253
  • B Corpus-based approach … 253
  • C Demonstration channel … 254
  • D Multimodal integration … 255
  • E Conclusion … 255

Part IV: Design

François Grondin and François Michaud
Real-time audition system for autonomous mobile robots … 263
I Introduction … 263
II Issues and challenges in robot audition … 264

  • A Microphones … 265
  • B Reverberation … 265
  • C Environmental noise … 267
  • D Ego noise … 268
  • E Real-time performance … 268

III ManyEars: an open framework for robot audition … 268

  • A Localization … 269
  • B Tracking … 271
  • C Separation … 272
  • D Post-filtering … 274

IV Recognition … 275

  • A Automatic speech recognition (ASR) … 275
  • B Speaker recognition … 278
  • C Emotion, music and daily sounds recognition … 279

V Conclusion … 280

Sandra Y. Okita and Victor Ng-Thow-Hing
The effects of design choices on human-robot interactions in children and adults … 285
I Introduction … 285
II Prior work – the evolution of the role of robots … 286
III Social schemas and social metaphors … 288
IV Design choices in lower-level communication modalities … 291

  • A Type of voice … 291
  • B Speed setting of gestures … 294

V Design choices in higher-level communication modalities … 297

  • A Proxemics and social schemas … 297
  • B Level of attention … 299

VI Effects of developmental differences on design choices … 303

  • A Level of intelligent behavior in robots … 304
  • B Realism and contingency level … 307

VII Conclusions and future work … 309

Part V: Conclusion

Roger K. Moore
From talking and listening robots to intelligent communicative machines … 317
I Introduction … 317
II Looking for solutions … 319

  • A Beyond speech … 320
  • B Beyond words … 322
  • C Beyond meaning … 323
  • D Beyond communication … 324
  • E Beyond dialogue … 325
  • F Beyond one-off interactions … 326

III Towards intelligent communicative machines … 327

  • A Achieving an appropriate balance of capabilities … 328
  • B A consolidated perspective … 329
  • C Beyond human abilities … 330

IV Conclusion … 330

Index … 337