Developing Enterprise Chatbots: Learning Linguistic Structures (2019) .. by Boris Galitsky
Contents
1 Introduction … 1
1.1 Introduction … 1
1.2 Current Chatbot Trends … 3
1.3 Current Status of Bot Development … 4
1.4 How to Read This Book and Build a Chatbot That Can Be Demoed … 8
References … 10
2 Chatbot Components and Architectures … 13
2.1 Introduction to Chatbots Architecture … 13
2.1.1 Definitions … 13
2.1.2 Dialogue Manager … 14
2.1.3 Multimodal Interaction … 18
2.1.4 Context Tracking … 19
2.1.5 Topic Detection … 20
2.1.6 Named Entities and Their Templates … 20
2.1.7 Information Retrieval … 22
2.1.8 Personalization … 22
2.1.9 Architecture of a Task-Oriented Chatbot … 23
2.2 History of Chatbots … 25
2.3 Deterministic Dialogue Management … 28
2.3.1 Handling Context … 29
2.3.2 Turn-Taking … 29
2.3.3 Action Selection … 31
2.3.4 Dialogue Management with Manually Coded RULES … 31
2.3.5 Finite-State Machines … 33
2.3.6 Rule-Based Dialogue Management … 34
2.3.7 Frame and Template-Based Dialogue Management … 35
2.4 Dialogue Management Based on Statistical Learning … 36
2.4.1 Bayesian Networks … 37
2.4.2 Neural Networks … 38
2.4.3 Markov Models … 39
2.5 Dialogue Management Based on Example-Based, Active and Transfer Learning … 41
2.6 Conclusions … 47
References … 47
3 Explainable Machine Learning for Chatbots … 53
3.1 What Kind of Machine Learning a Chatbot Needs … 53
3.1.1 Accuracy vs Explainability … 53
3.1.2 Explainable vs Unexplainable Learning … 54
3.1.3 Use Cases for the ML System Lacking Explainability … 56
3.1.4 Automated Detection of a Request to Explain … 59
3.2 Discriminating Between a User Question and User Request … 60
3.2.1 Examples of Questions and Transactional Requests … 63
3.2.2 Nearest Neighbor-Based Learning for Questions vs Transactional Requests Recognition … 64
3.3 A Decision Support Chatbot … 65
3.3.1 Example of a Decision Support Session … 66
3.3.2 Computing Decisions with Explanations … 68
3.4 Explanation-Based Learning System Jasmine … 72
3.4.1 A Reasoning Schema … 73
3.4.2 Computing Similarity Between Objects … 79
3.5 Conclusions … 80
References … 82
4 Developing Conversational Natural Language Interface to a Database … 85
4.1 Introduction … 85
4.1.1 History … 86
4.2 Statistical and Deep Learning in NL2SQL Systems … 88
4.2.1 NL2SQL as Sequence Encoder … 89
4.2.2 Limitations of Neural Network Based Approaches … 96
4.3 Advancing the State-of-the-Art of NL2SQL … 96
4.3.1 Building NL2SQL via Multiword Mapping … 96
4.3.2 Sketch-Based Approach … 99
4.3.3 Extended Relational Algebra to Handle Aggregation and Nested Query … 100
4.3.4 Interpreting NL Query via Parse Tree Transformation … 103
4.4 Designing NL2SQL Based on Recursive Clause Building, Employing Thesauri and Implementing Via Chatbot … 106
4.4.1 Selecting Deterministic Chatbot-Based Approach … 106
4.4.2 Interpreting Table.Field Clause … 107
4.4.3 Collecting Information on a Database and Thesaurus for NL2SQL … 108
4.4.4 Iterative Clause Formation … 109
4.4.5 Clause Building by Matching the Phrase with Indexed Row … 109
4.4.6 Extracting Focus Clause … 112
4.5 Resolving Ambiguities in Query Interpretation via Chatbot … 112
4.6 A Sample Database Enabled with NL2SQL … 114
4.7 Conclusions … 117
References … 119
5 Assuring Chatbot Relevance at Syntactic Level … 121
5.1 Introduction … 121
5.2 Syntactic Generalization in Search and Relevance Assessment … 124
5.3 Generalizing Portions of Text … 128
5.3.1 Generalizing at Various Levels: From Words to Paragraphs … 130
5.3.2 Equivalence Transformation on Phrases … 132
5.3.3 Simplified Example of Generalization of Sentences … 133
5.3.4 From Syntax to Inductive Semantics … 137
5.3.5 Nearest Neighbor Learning of Generalizations … 138
5.4 Evaluation of a Generalization-Based Search Engine … 139
5.4.1 User Interface of Search Engine … 139
5.4.2 Qualitative Evaluation of Search … 141
5.4.3 Evaluation of Web Search Relevance Improvement … 143
5.4.4 Evaluation of Product Search … 144
5.5 Evaluation of Text Classification Problems … 147
5.5.1 Comparative Performance Analysis in Text Classification Domains … 147
5.5.2 Example of Recognizing Meaningless Sentences … 149
5.6 Implementation of OpenNLP.Similarity Component … 150
5.6.1 First Use Case of Similarity Component: Search … 150
5.6.2 Solving a Content Generation Problem … 151
5.6.3 Filtering Out Meaningless Speech Recognition Results … 152
5.6.4 Comparison with Bag-of-Words Approach … 153
5.7 Related Work … 154
5.8 Conclusions … 157
References … 160
6 Semantic Skeleton Thesauri for Question Answering Bots … 163
6.1 Introduction … 163
6.2 Defining Semantic Headers of Answers … 165
6.3 Defining Semantic Skeletons for Common Sense … 167
6.4 SSK Handling of Complex Questions … 169
6.5 Evaluation of Relevance Improvement Due to SSK … 171
6.6 Discussion and Conclusions … 173
References … 174
7 Learning Discourse-Level Structures for Question Answering … 177
7.1 Introduction … 177
7.2 Parse Thickets and Their Graph Representation … 180
7.2.1 Extending Phrases to Span Across Sentences … 181
7.2.2 Computing Structural Distance Between a Question and an Answer … 182
7.3 Dimensions of Sentence-Level Generalization … 187
7.4 Generalization of Parse Thickets … 188
7.4.1 A High-Level View … 189
7.4.2 Generalization for RST Arcs … 191
7.4.3 Generalization for Communicative Action Arcs … 192
7.4.4 Kernel Learning for Parse Thickets … 195
7.4.5 From Matching to Learning Parse Thickets … 197
7.5 Evaluation of Search Relevance Improvement … 198
7.5.1 Evaluation Settings … 199
7.5.2 Query Is a Sentence and Answer Is a Sentence … 200
7.5.3 Query Is a Paragraph and Answer Is a Paragraph … 201
7.5.4 Extended Tree Kernel Learning for Individual Search Sessions … 203
7.5.5 Comparison of Search Performance with Other Studies … 205
7.6 Implementation of Generalization at Many Levels … 207
7.7 Related Work … 208
7.8 Conclusions … 214
References … 216
8 Building Chatbot Thesaurus … 221
8.1 Introduction … 221
8.2 Improving Chatbot Relevance by Thesauri … 224
8.2.1 Defining the is_about Relation for a Query … 224
8.2.2 Thesaurus-Based Answer Selection … 225
8.2.3 Thesaurus-Based Relevance Verification Algorithm … 229
8.3 Building Thesauri … 229
8.3.1 Thesaurus Construction as a Process of Learning and Web Mining … 229
8.3.2 Thesaurus-Building Algorithm … 232
8.3.3 An Example of Thesaurus Learning Session … 233
8.3.4 Thesaurus Snapshot … 234
8.4 Evaluation of Chatbot Relevance Boost … 235
8.4.1 Evaluation of Search Relevance Improvement … 235
8.4.2 Industrial Evaluation of Thesaurus-Based Text Similarity … 242
8.5 Thesaurus Builder as a Part of OpenNLP … 246
8.5.1 Running Thesaurus Learner … 246
8.6 Related Work … 247
8.7 Conclusions … 249
References … 250
9 A Content Management System for Chatbots … 253
9.1 Introduction … 253
9.1.1 From Search to Personalized Recommendations to Chatbots … 256
9.2 Relevance-Related Problems in a Content-Management System … 257
9.2.1 Content Pipeline Architecture … 258
9.2.2 The Engines Assuring CMS Relevance … 260
9.2.3 Content Processing Units … 261
9.3 Generalization of Expressions of Interest … 269
9.3.1 Personalization Algorithm as Intersection of Likes … 270
9.3.2 Mapping Categories of Interest/Thesauri … 270
9.3.3 Defeasible Logic Programming-Based Rule Engine … 271
9.4 The Algorithms for High-Relevance CMS … 275
9.4.1 De-duplication Algorithms … 275
9.4.2 Analyzing Sentiments by Parse Tree Navigation … 277
9.4.3 Agglomerative Clustering of Search Results … 280
9.5 Building Conclusive Answers … 285
9.5.1 Concluding a Question Answering Session … 285
9.5.2 Building a Structure of Conclusive Answer … 286
9.5.3 Content Compilation Algorithm … 292
9.5.4 A Brief Example of the Content Generation Flow … 295
9.5.5 Modeling the Content Structure of Texts … 297
9.5.6 Related Work on Conclusive Answers … 300
9.6 Evaluation … 302
9.6.1 Performance Analysis of the Content Pipeline Components … 302
9.6.2 Performance Analysis of Personalized Recommendations … 308
9.6.3 Performance Analysis of SG-Supported Search Relevance … 310
9.7 Related Work and Discussions … 313
9.8 Conclusions … 318
9.8.1 From Search Engines to Chatbots … 318
9.8.2 Relevance in a CMS … 319
References … 323
10 Rhetorical Agreement: Maintaining Cohesive Conversations … 327
10.1 Introduction … 327
10.1.1 Request and Response Utterances … 329
10.1.2 Correct and Incorrect Response-Request Pairs … 333
10.2 Communicative Discourse Trees … 333
10.2.1 Relying on VerbNet to Represent Communicative Actions … 337
10.3 Classification Settings for Request-Response Pairs … 339
10.3.1 Nearest Neighbor Graph-Based Classification … 340
10.3.2 Tree Kernel Learning for CDT … 342
10.3.3 Additional Rules for RR Agreement and RR Irrationality … 343
10.4 Evaluation … 344
10.4.1 Evaluation Domains … 344
10.4.2 Recognizing Valid and Invalid Answers … 345
10.4.3 Measuring RR Agreement in Our Evaluation Domains … 347
10.5 Handling Natural Language Descriptions of Algorithms … 349
10.6 Related Work … 351
10.6.1 Managing Dialogues and Question Answering … 352
10.6.2 Dialog Games … 355
10.6.3 Rhetorical Relations and Argumentation … 356
10.7 Conclusion … 358
References … 360
11 Discourse-Level Dialogue Management … 365
11.1 Introduciton … 365
11.2 Introduction: Maintaining Cohesive Session Flow … 366
11.2.1 Limitations of Keyword Learning-Based Approaches … 367
11.2.2 Datasets for Evaluation … 368
11.3 Dialogue Management via Extended Discourse Trees … 369
11.3.1 Clarification-Based Domain Exploration Scenarios … 370
11.3.2 Navigating the Extended Discourse Tree … 373
11.3.3 Example of Navigating an Extended Discourse Tree for Three Documents … 375
11.3.4 Constructing EDT … 377
11.3.5 Manipulating with Discourse Trees … 378
11.3.6 Multi-document Navigation Without Discourse Trees … 381
11.3.7 Extended Discourse Tree for FAQ Pages … 384
11.3.8 Evaluation: Information Access Efficiency in Chatbots Versus Search Engines … 385
11.3.9 Related Work on Discourse Disentanglement … 387
11.4 Building Dialogue Structure from a Discourse Tree of an Initial Question … 388
11.4.1 Setting a Dialogue Style and Structure by a Query … 389
11.4.2 Building Dialogue Structure in Customer Support Dialogues … 391
11.4.3 Finding a Sequence of Answers to Be in Agreement with a Question … 394
11.4.4 Searching for Answers with Specified RR for Dialogue Construction … 396
11.4.5 Evaluation of the Dialogue Construction from the First Query … 397
11.5 Constructing Imaginary Discourse Trees for Dialogue Management … 400
11.5.1 Answering Questions via Entities and Discourse Trees … 401
11.5.2 Question Answer Filtering Algorithm … 404
11.5.3 Experiments with Answering Convergent Questions … 405
11.6 Dialogue Management Based on Lattice Walking … 406
11.6.1 Formal Concept Analysis … 408
11.6.2 Pattern Structure and Projections … 408
11.6.3 Measures for Pattern Concepts … 409
11.6.4 Lattice Walker Example … 410
11.6.5 The Structure of the Datasets … 412
11.6.6 Principles of Query Refinement … 413
11.7 Related Work … 414
11.7.1 Visualization of Discourse Trees and Discourse Features … 418
11.8 Open Source Implementation … 420
11.9 Conclusions … 421
References … 422
12 A Social Promotion Chatbot … 427
12.1 Introduction … 427
12.2 The Domain of Social Promotion … 430
12.3 CASP Architecture … 431
12.4 Use Cases of CASP … 433
12.5 Evaluation of Relevance … 435
12.6 Evaluation of Extraction of Communicative Action … 439
12.7 Evaluation of Trust … 440
12.8 Replying to Multiple Posts … 444
12.8.1 Introducing Lattice Querying … 444
12.8.2 Sentence-Based Lattice Queries … 446
12.8.3 Paragraph-Level Lattice Queries … 448
12.8.4 Evaluation of Web Mining via Lattice Queries … 450
12.9 Correction of Obtained Post Candidate … 453
12.9.1 Meaningless Phrases Substitution Algorithm … 454
12.10 More Examples of Conversations … 456
12.11 Discussion and Conclusions … 458
References … 461
13 Enabling a Bot with Understanding Argumentation and Providing Arguments … 465
13.1 Introduction … 465
13.2 Finding Valid Argumentation Patterns and Identifying Fake Content … 469
13.2.1 Handling Heated Arguments … 477
13.3 Evaluation of Logical Argument Detection … 479
13.3.1 Dataset for General Argumentation … 479
13.3.2 Specific Argumentation Patterns Dataset … 481
13.3.3 Evaluation Setup and Results … 483
13.3.4 CDT Construction Task … 485
13.4 Evaluation of Affective Argument Detection … 487
13.4.1 Detecting Sentiments at the Discourse Level … 487
13.4.2 Dataset and Evaluation Setup … 488
13.4.3 Extending Compositionality Semantics Towards Discourse … 490
13.4.4 Evaluation Results … 491
13.5 Assessing Validity of the Extracted Argument Patterns via Dialectical Analysis … 492
13.5.1 Building a Defeasible Logic Program … 493
13.5.2 Evaluation of Validation of Arguments … 498
13.6 Assessment of Text Integrity … 501
13.6.1 Discourse Structure and Text Integrity … 503
13.6.2 Sentiment Profile of a Dialogue … 504
13.6.3 Evaluation of Text Integrity Assessment … 509
13.7 Tackling Noisy Discourse Trees … 510
13.7.1 Discourse Trees Alignment … 512
13.7.2 Example of Building NCDT … 514
13.7.3 Evaluation of Learning Discourse Trees for Noisy Text … 514
13.8 Related Work … 517
13.8.1 Argument Mining … 518
13.8.2 Logical Argument and Discourse Linguistics … 519
13.8.3 Discourse Structures and Sentiment Analysis … 521
13.8.4 Discourse Parses and Ranking of Results … 522
13.8.5 Text Readability … 523
13.9 Conclusions … 525
References … 526
14 Rhetorical Map of an Answer … 533
14.1 A Rhetorical Map of an Answer: Which DT-Answer Nodes Should Match the Question and Which Should Not … 533
14.1.1 From Discourse Tree to a Rhetorical Map of an Answer … 534
14.1.2 Definition and Construction Algorithm … 540
14.1.3 How Rhetorical Maps Improve Search Precision … 541
14.2 A Rhetorical Map of an Answer: What to Index and What Not to Index … 543
14.2.1 Informative and Uninformative Parts of an Answer … 544
14.2.2 How a Discourse Tree Indicates What to Index and What Not to Index … 546
14.2.3 How Rhetorical Relations Determine Indexing Rules … 549
14.2.4 Forming Alternative Questions … 550
14.2.5 Classifying EDUs as Informative or Not … 551
14.3 Conclusions … 553
References … 554
15 Conclusions … 557
Contents xv