The field of Natural Language Processing (NLP) is one of the most important and useful application areas of Artificial Intelligence (AI). NLP is now rapidly evolving, as new methods and toolsets converge with an ever-expanding wealth of available data. This state-of-the-art handbook addresses all aspects of formal analysis for Natural Language Processing. Following a review of the field’s history, it systematically introduces readers to the rule-based model, statistical model, neural network model, and pre-training model in Natural Language Processing.
At a time characterized by the steady and vigorous growth of Natural Language Processing, this handbook provides a highly accessible introduction and much-needed reference guide to both the theory and method of NLP. It can be used for individual study, as the textbook for courses on Natural Language Processing or computational linguistics, or as a supplement to courses on Artificial Intelligence, and offers a valuable asset for researchers, practitioners, lecturers, graduate and undergraduate students alike.
Having been widely used in NLP in recent years, neural networks and Deep Learning have gradually become the mainstream technology in NLP research. Therefore, this chapter will present some details about models based on neural networks and Deep Learning, including the evolution of neural networks, neural networks of our brain, artificial neural networks, Machine Learning, Deep Learning, word vectors, word embedding, dense word vectors, perceptrons, feedforward neural networks, convolutional neural networks, recurrent neural networks, attention mechanisms, external memory, and pretrained models (such as Transformer and BERT).
Author(s): Zhiwei Feng
Publisher: Springer
Year: 2023
Language: English
Pages: 802
Preface
Contents
About the Author
Acronyms
Part I: History Review
Chapter 1: Past and Present of Natural Language Processing
1.1 What Is Natural Language Processing?
1.2 History Review of Natural Language Processing
1.2.1 Embryo
1.2.2 Growth
1.2.3 Boom
1.3 Characteristics of Current Trends in Natural Language Processing
Bibliography
Chapter 2: Pioneers in the Study of Language Computing
2.1 Markov Chains
2.2 Zipf´s Law
2.3 Shannon´s Work on Entropy
2.4 Bar-Hillel´s Categorial Grammar
2.5 Harris´s Approach of Linguistic String Analysis
2.6 . . ´s Linguistic Set Theory Model
Bibliography
Part II: Formal Models
Chapter 3: Formal Models Based on Phrase Structure Grammar
3.1 Chomsky´s Hierarchy of Grammar
3.2 Finite State Grammar and Its Limitations
3.3 Phrase Structure Grammar
3.4 Recursive Transition Networks and Augmented Transition Networks
3.5 Bottom-Up and Top-Down Analysis
3.6 General Syntactic Processor and Chart Parsing
3.7 Earley Algorithm
3.8 Left-Corner Analysis
3.9 Cocke-Younger-Kasami Algorithm
3.10 Tomita Algorithm
3.11 Government and Binding Theory and Minimalist Program
3.12 Joshi´s Tree Adjoining Grammar
3.13 Formal Description of the Structure of Chinese Characters
3.14 Hausser´s Left-Associative Grammar
Bibliography
Chapter 4: Formal Models Based on Unification
4.1 Multiple-Branched and Multiple-Labeled Tree Analysis (MMT)
4.2 Kaplan´s Lexical Functional Grammar
4.3 Martin Kay´s Functional Unification Grammar
4.4 Gazdar´s Generalized Phrase Structure Grammar
4.4.1 Syntactic Rule System
4.4.2 Meta-Rules
4.4.3 The Feature Restriction System
4.4.3.1 Feature Co-occurrence Restriction (FCR)
4.4.3.2 Feature Specification Defaults (FSD)
4.4.3.3 Head Feature Convention (HFC)
4.4.3.4 Foot Feature Principle (FFP)
4.4.3.5 Control Agreement Principle (CAP)
4.4.3.6 Linear Precedence Statement (LPS)
4.5 Shieber´s PATR
4.6 Pollard´s Head-Driven Phrase Structure Grammar
4.7 Pereira and Warren´s Definite Clause Grammar
Bibliography
Chapter 5: Formal Models Based on Dependency and Valence
5.1 Origin of Valence
5.2 Tesnière´s Dependency Grammar
5.2.1 Connexion
5.2.2 Translation
5.3 Application of Dependency Grammar in NLP
5.4 Valence Grammar
5.5 Application of Valence Grammar in NLP
Bibliography
Chapter 6: Formal Models Based on Lexicalism
6.1 Gross´s Lexicon-Grammar
6.2 Link Grammar
6.3 Lexical Semantics
6.4 Ontology
6.5 WordNet
6.6 HowNet
6.7 Pustejovsky´s Generative Lexicon Theory
Bibliography
Chapter 7: Formal Models of Automatic Semantic Processing
7.1 Sememe Analysis
7.2 Semantic Field
7.2.1 Taxonomic Field Type
7.2.2 Component Field Type
7.2.3 Ordered Field Type
7.2.4 Opposition Field Type
7.2.5 Synonymous Field Type
7.3 Semantic Network
7.4 Montague´s Semantics
7.5 Wilks´ Preference Semantics
7.6 Schank´s Conceptual Dependency Theory
7.7 Mel´chuk´s Meaning Text Theory
7.8 Fillmore´s Deep Case and Frame Semantics
7.9 Word Sense Disambiguation Methods
7.9.1 WSD by Selecting the Most Common Sense
7.9.2 WSD by Using Parts of Speech
7.9.3 WSD Based on Selectional Restrictions
7.9.4 Robust WSD Methods
7.9.5 Supervised Learning Approaches
7.9.5.1 Naïve Bayes Classifier
7.9.5.2 Decision List Classifier
7.9.6 Bootstrapping Approaches to WSD
7.9.7 Unsupervised WSD Methods
7.9.8 Dictionary-Based WSD Approach
Bibliography
Chapter 8: Formal Models of Automatic Situation and Pragmatic Processing
8.1 Basic Concepts of Systemic Functional Grammar
8.2 Application of Systemic Functional Grammar in Natural Language Processing
8.3 Speech Act Theory and Conversation Intelligent Agent
Bibliography
Chapter 9: Formal Models of Discourse Analysis
9.1 Reference Resolution
9.2 Reasoning Techniques in Text Coherence
9.3 Mann and Thompson´s Rhetorical Structure Theory
Bibliography
Chapter 10: Formal Models of Probabilistic Grammar
10.1 Context-Free Grammar and Sentence Ambiguity
10.2 Fundamentals of Probabilistic Context-Free Grammar
10.3 Three Assumptions of Probabilistic Context-Free Grammar
10.4 Probabilistic Lexicalized Context-Free Grammar
Bibliography
Chapter 11: Formal Models of Neural Network and Deep Learning
11.1 Development of Neural Network
11.2 Brain Neural Network and Artificial Neural Network
11.3 Machine Learning and Deep Learning
11.4 Word Vector and Word Embedding (CBOW, Skip-Gram)
11.5 Dense Word Vector (Word2vec)
11.6 Perceptron
11.7 Feed-Forward Neural Network (FNN)
11.8 Convolutional Neural Network (CNN)
11.9 Recurrent Neural Network (RNN)
11.10 Attention Mechanism
11.11 External Memory
11.12 Pretrained Models (Transformer and BERT)
Bibliography
Chapter 12: Knowledge Graphs
12.1 Types of Knowledge Graphs
12.2 Knowledge Representation
12.3 Knowledge Merging
12.4 Entity Recognition and Disambiguation
12.5 Relation Extraction
12.6 Event Extraction
12.7 Knowledge Storage
Bibliography
Chapter 13: Concluding Remarks
Epilogue
