This book explores AI methodologies for the implementation of affective states in intelligent learning environments. Divided into four parts, Multimodal Affective Computing: Technologies and Applications in Learning Environments begins with an overview of Affective Computing and Intelligent Learning Environments, from their fundamentals and essential theoretical support up to their fusion and some successful practical applications. The basic concepts of Affective Computing, Machine Learning, and Pattern Recognition in Affective Computing, and Affective Learning Environments are presented in a comprehensive and easy-to-read manner. In the second part, a review on the emerging field of Sentiment Analysis for Learning Environments is introduced, including a systematic descriptive tour through topics such as building resources for sentiment detection, methods for data representation, designing and testing the classification models, and model integration into a learning system. The methodologies corresponding to Multimodal Recognition of Learning-Oriented Emotions are presented in the third part of the book, where topics such as building resources for emotion detection, methods for data representation, multimodal recognition systems, and multimodal emotion recognition in learning environments are presented. The fourth and last part of the book is devoted to a wide application field of the combination of methodologies, such as Automatic Personality Recognition, dealing with issues such as building resources for personality recognition, methods for data representation, personality recognition models, and multimodal personality recognition for affective computing.
Author(s): Ramón Zatarain Cabada; Héctor Manuel Cárdenas López; Hugo Jair Escalante
Publisher: Springer International Publishing
Year: 2023
Language: English
Pages: 211
1 Affective Computing
1.1 Introduction
1.2 Theories of Emotions, Sentiments, and Affect
1.3 Theories of Personality and Learning
1.3.1 Main Personality Theories
1.3.2 The Effect of Personality on Learning
1.4 Cognitive Processing and Learning-Oriented Emotions
1.5 Emotions, Sentiment, Personality, and the Machine
1.6 Discussion
References
2 Machine Learning and Pattern Recognition in Affective Computing
2.1 Introduction
2.2 Input Data in Affective Computing
2.3 Machine Learning Variants and Models
2.3.1 Supervised Learning
2.3.2 Unsupervised Learning
2.3.3 Other Learning Variants
2.4 Dimensionality Reduction
2.5 Deep Learning
2.5.1 Neural Networks
2.5.2 Convolutional Neural Networks
2.5.3 Sequential Models
2.5.4 Other DL-Based Models
2.6 Discussion
References
3 Affective Learning Environments
3.1 Introduction
3.2 The Dynamics of Teaching and Learning
3.3 Theoretical Models for the Role of Affect in Learning
3.4 Design of Affective Learning Environments
3.5 Discussion
References
Part II Sentiment Analysis for Learning Environments
4 Building Resources for Sentiment Detection
4.1 Introduction
4.2 Experimental Setup Design
4.3 Data Mining System Design and Implementation
4.4 Data Mining Challenges
4.5 Discussion
References
5 Methods for Data Representation
5.1 Introduction
5.2 Tokenization
5.3 Parsing
5.4 Stemming and Lemmatization
5.5 Word Embeddings
5.6 Discussion
References
6 Designing and Testing the Classification Models
6.1 Introduction
6.2 Lexicon-Based Sentiment Analysis
6.3 Multilayer Perceptron
6.4 Convolutional Neural Networks
6.5 Long Short-Term Memory Neural Networks
6.6 Evaluation Protocols
6.7 Discussion
References
7 Model Integration to a Learning System
7.1 Introduction
7.2 Building Resources
7.3 Dataset Focused on the Programming Language Domain
7.4 Creation of a Dictionary of Emotions Focused on Learning (SentiDICC)
7.5 Model Selection Process
7.6 Evaluation Metrics
7.7 Model Training and Validation
7.8 Affective Learning Environment
7.8.1 Model Implementation
7.8.2 Affective Tutoring Agent
7.9 Discussion
References
Part III Multimodal Recognition of Learning-Oriented Emotions
8 Building Resources for Emotion Detection
8.1 Introduction
8.2 Experimental Setup Design
8.2.1 Selecting Data Modalities
8.2.2 Labeling Process
8.2.3 Work Environment
8.2.4 Emotion Elicitation
8.3 Discussion
References
9 Methods for Data Representation
9.1 Introduction
9.2 Image-Based Data Representation for Facial Expressions
9.3 Spectrogram-Based Data Representation for Speech
9.4 Signal-Based Data Representation for Physiological Data
9.5 Practical Considerations for Choosing Data Representation Methods
9.6 Discussion
References
10 Multimodal Recognition Systems
10.1 Introduction
10.2 Data Fusion Techniques
10.3 Convolutional Neural Networks in Multimodal Emotion Recognition
10.4 Long Short-Term Memory in Multimodal Emotion Recognition
10.5 Evaluation Protocols
10.6 Discussion
References
11 Multimodal Emotion Recognition in Learning Environments
11.1 Introduction
11.2 Enhancing the Student Motivation, Engagement, and Cognitive Processing
11.3 Dataset Creation
11.3.1 Labeling Process
11.3.2 Fusing Different Datasets
11.4 Defining DL Architectures
11.4.1 Convolutional Neural Networks
11.4.2 Long Short-Term Memory
11.5 Evaluation Protocols
11.6 Model Deployment
11.6.1 Data Pipelines
11.6.2 Model Interpretation
11.7 Affective Tutoring Agent
11.8 Discussion
References
Part IV Automatic Personality Recognition
12 Building Resources for Personality Recognition
12.1 Introduction
12.2 Data Structure Design
12.3 Personality Data Annotation
12.4 Applications for Data Collection
12.5 Discussion
References
13 Methods for Data Representation
13.1 Introduction
13.2 Speech Data Representation
13.3 Text Data Representation
13.4 Facial Expressions Data Representation
13.5 Physiological Signals Data Representation
13.6 Differences Between Emotion and Personality Data Representation
13.7 Discussion
References
14 Personality Recognition Models
14.1 Introduction
14.2 Unimodal Architectures
14.3 Multimodal Architectures
14.4 Discussion
References
15 Multimodal Personality Recognition for Affective Computing
15.1 Introduction
15.2 Design of a Data Structure
15.2.1 Collecting a Dataset for APP
15.2.2 Creating a Dataset for APR
15.2.3 Apparent Personality Perception (APP) Versus Automatic Personality Recognition (APR)
15.3 An Application for Data Collection
15.3.1 Architectural Model of the Platform
15.4 Data Recollection Process
15.5 Adapting a Dataset to a Working Environment
15.5.1 Image Preprocessing
15.5.2 Sound Preprocessing
15.6 Personality Recognition Model Design
15.6.1 Image-Based Models
15.6.2 Sound-Based Models
15.6.3 Multimodal Models
15.7 Laboratory Tests
15.8 Models as a Service
15.9 Personality Recognition in Education
15.10 Discussion
References
Glossary
