Multimodal AI in Healthcare: A Paradigm Shift in Health Intelligence

Preface
Contents
Contributors
Abbreviations
Multimodal Artificial Intelligence: Next Wave of Innovation in Healthcare and Medicine
1 Introduction
2 Clinical and Biomedical Applications of Multimodal AI and Data Science
3 Advances in AI Technologies and Data Analytics in Healthcare
References
Unsupervised Numerical Reasoning to Extract Phenotypes from Clinical Text by Leveraging External Knowledge
1 Introduction
2 Related Work
3 Methodology
3.1 External Knowledge
3.2 Number and Lexical Candidates Extraction
3.3 Contextualized Embeddings for Numeric Entities and Lexical Candidates
3.4 Embedding Similarity and Deterministic HPO Assignment
4 Experiment Design
4.1 Datasets
4.2 Implementation Details
4.3 Baselines and Evaluation Methods
5 Results and Discussion
5.1 Quantitative Analysis
5.2 Qualitative Analysis
5.3 Ablation Studies
6 Conclusions and Future Works
References
Domain-specific Language Pre-training for Dialogue Comprehension on Clinical Inquiry-Answering Conversations
1 Introduction
2 Domain-Specific Language Pre-training
2.1 Conversation-based Sample Construction
2.2 Experiment Setup of Pre-training
3 Dialogue Comprehension on Clinical Inquiry-Answering Conversations
3.1 Task Definition
3.2 Clinical Dialogue Corpus
3.3 Baseline Models
3.4 Training Configuration
3.5 Evaluation: Comparison with Baselines
3.6 Evaluation in Low-Resource Scenarios
3.7 Evaluation: Pre-training Scheme Comparison
4 Conclusions
References
Clinical Dialogue Transcription Error Correction Using Seq2Seq Models
1 Introduction
2 Related Work
3 Clinical Dialogue Transcription
3.1 Gastrointestinal Clinical Dialogue Dataset
4 Methods
4.1 General Purpose Base Language Models
4.2 PubMed Gastrointestinal Dataset
4.3 Fine-Tune Using Self-Supervision
5 Evaluation
5.1 Performance Metric
5.2 Comparison of Base Language Models
5.3 Comparison of Fine-Tuned Language Models
6 Discussion
7 Conclusion
References
Customized Training of Pretrained Language Models to Detect Post Intents in Online Health Support Groups
1 Introduction
2 Background and Related Work
3 Tweet2Quit Dataset
3.1 Data Collection
3.2 Identification of the Intents to Annotate
3.3 Reliability
3.4 Annotation Process
4 Models
4.1 Random Forest (Baseline)
4.2 Pretrained Language Models
5 Adapting to the Labels' Relationships
5.1 Customized Loss Functions
6 Experiments
6.1 Experimental Setup
6.2 Pretrained Language Models
6.3 Loss Functions and Adjusted Metrics
7 Conclusion and Discussion
References
EXPECT-NLP: An Integrated Pipeline and User Interface for Exploring Patient Preferences Directly from Patient-Generated Text
1 Introduction
2 Aspect Based Sentiment Analysis
3 Data Model
4 Aspect Extraction Results
5 EXPECT-NLP Interface
6 Real-World Use Cases
7 Conclusion and Future Work
References
Medication Error Detection Using Contextual Language Models
1 Introduction
2 Proposed Methodology
2.1 Problem Formalization
2.2 Contextual Language Models
3 Experimentation
3.1 Dataset Generation
3.2 ASR Implementation
3.3 Results and Analysis
4 Conclusions
References
Latent Representation Weights Learning of the Indefinite Length of Views for Conception Diagnosis
1 Introduction
2 Integration and Methods
2.1 Latent Representation Weight Learning
2.2 Optimization
3 Results and Discussion
3.1 Data Collection
3.2 Experimental Setup
3.3 Comparison with the Baseline
4 Conclusion
References
Phenotyping with Positive Unlabelled Learning for Genome-Wide Association Studies
1 Introduction
2 Related Work
3 Methods
3.1 Problem Formulation
3.2 Anchor Learning
3.3 Using BERT as Anchor Classifier
3.4 Baselines
3.5 Anchor Performance Metrics
3.6 Hyperparameters
4 Experiments and Results
4.1 UK Biobank data
4.2 Anchor Classifier Performance and Robustness to Control Noise
4.3 Evaluating Phenotypes with GWAS
5 Discussion
References
Out-of-Distribution Detection for Medical Applications: Guidelines for Practical Evaluation
1 Introduction
2 Related Work
3 Methods
3.1 Considerations
3.2 Designing OOD Tests
3.3 Dataset
3.4 Models
3.5 AUC Score of OOD Detection
4 Experimental Results
5 Discussion
References
A Robust System to Detect and Explain Public Mask Wearing Behavior
1 Introduction
2 Background and Related Work
2.1 Face Mask Detection
2.2 Explanation
3 Methodology
3.1 Detection Architecture
3.2 Explanation Architecture
3.3 Transfer Learning
3.4 Individual and Aggregate Explanation
4 Experimentation
4.1 Dataset
4.2 Augmentation
4.3 Experiment Setup
5 Evaluation
5.1 Qualitative Results
5.2 Quantitative Results
5.3 Sanity Check
6 Conclusion
References
A Federated Cox Model with Non-proportional Hazards
1 Introduction
2 Background and Related Work
3 Model
4 Experiments
4.1 Datasets
4.2 Setup
4.3 Results
5 Conclusion
Appendix A Additional Figures
References
A Step Towards Automated Functional Assessment of Activities of Daily Living
1 Introduction
2 Related Works
3 Dataset
4 Proposed Approach
5 Training and Evaluation
6 Experiments and Results
6.1 Ablation Study
7 Conclusion
References
The Interpretation of Deep Learning Based Analysis of Medical Images—An Examination of Methodological and Practical Challenges Using Chest X-ray Data
1 Introduction
1.1 Previous Work
2 Performance Quantification
3 Model Training
4 Analysis
4.1 Understanding Data and Findings Interpretation
5 Summary and Conclusions
References
Predicting Drug Functions from Adverse Drug Reactions by Multi-label Deep Neural Network
1 Introduction
2 Related Work
3 Proposed Methodology and Working Architecture
3.1 Description of the Datasets
3.2 Problem Statement
3.3 Proposed Methodology
4 Performance Evaluation of Experiments
4.1 Performance Measurement
4.2 Experimental Results and Discussions
5 Conclusion
References
Pattern Discovery in Physiological Data with Byte Pair Encoding
1 Introduction
2 Related Work
3 Method
3.1 PAA Transformation
3.2 Discretization
3.3 Identifying Patterns
3.4 Handling Consecutive Identical Symbols
3.5 Post Processing
3.6 Extension to Multivariate Series
3.7 Classification/Regression
3.8 Hyper-Parameter Tuning
4 Data
5 Results
5.1 Computation Speed
5.2 Interpretability
6 Conclusion
References
Predicting ICU Admissions for Hospitalized COVID-19 Patients with a Factor Graph-based Model
1 Introduction
2 Model
2.1 Model Overview
2.2 Variable Selection
2.3 Factor Function Construction
2.4 Inference Algorithms
3 Experimental Setup
3.1 Dataset
3.2 Model Evaluation
4 Results
4.1 Predictive Biomarkers
4.2 Model Validation
5 Limitations and Future Work
6 Conclusion
References
Semantic Network Analysis of COVID-19 Vaccine Related Text from Reddit
1 Introduction
2 Data
3 Methods
4 Results
5 Discussion
6 Conclusion
References
Towards Providing Clinical Insights on Long Covid from Twitter Data
1 Introduction
2 Related Work
2.1 Social Media Platform for COVID-19
2.2 Clinical Information Extraction
2.3 Interpretability
3 Dataset
3.1 Data Acquisition and De-identification
3.2 Filtering Long COVID Self Reports
4 Methodology
5 Results and Discussion
References
Predicting Infections in the Covid-19 Pandemic—Lessons Learned
1 Introduction
2 Related Work
3 Learning-based Models for Predicting the Number of Infections
4 Experiments
5 Conclusion
References
Improving Radiology Report Generation with Adaptive Attention
1 Introduction
2 Related Work
3 Method
3.1 Encoder-Decoder Framework
3.2 Multi-Head Adaptive Attention
3.3 Choice of Pretrained Visual Extractor
4 Experiments and Results
4.1 Datasets
4.2 Model Evaluation
4.3 Model Development and Hyper Parameter Tuning
4.4 Ablation Study
4.5 Quantitative Evaluation
4.6 Qualitative Analysis
5 Conclusion
References
Instantaneous Physiological Estimation Using Video Transformers
1 Introduction
2 Related Work
2.1 Video Based Physiology Extraction
2.2 Transformers
3 Methods
3.1 Optical Basis of Video-Based Bio-Signal Extraction
3.2 Video Transformer for Physiological Estimation
3.3 Loss Formulation
4 Results
4.1 Implementation Details
4.2 Datasets and Evaluation Protocol
4.3 Heart Rate Estimation Results
4.4 Spatial Attention Mask
4.5 Respiration Rate Estimation Results
5 Conclusion
References
Automated Vision-Based Wellness Analysis for Elderly Care Centers
1 Introduction
2 Related Work
3 Proposed Wellness Analysis System
3.1 Facial Analysis
3.2 Activity Analysis
3.3 Interaction Analysis
3.4 Analysis of Long-Term Pattern and Trend
4 Evaluation
4.1 Data Collection
4.2 Results
5 Conclusion
References
Efficient Extraction of Pathologies from C-Spine Radiology Reports Using Multi-task Learning
1 Introduction
2 Datasets
3 Description of the Workflow
4 Methods
5 Results
6 Empirical Evidence Behind MultiTasking Models
7 Conclusion
References
Benchmarking Uncertainty Quantification on Biosignal Classification Tasks Under Dataset Shift
1 Introduction
2 Related Work
3 Uncertainty Quantification Approaches
4 Bencmark Tasks and Experiemnts
4.1 Biosignal Classification Tasks
4.2 Dataset Shift and Evaluation Protocol
4.3 Metrics
5 Results and Analysis
6 Conclusions
References
Mining Adverse Drug Reactions from Unstructured Mediums at Scale
1 Introduction
2 Related Work
3 Approach
3.1 Classification
3.2 Named Entity Recognition
3.3 Relation Extraction
4 Experimental Setup
4.1 Datasets
4.2 Experiments
4.3 Results
5 Conclusion
References
A Graph-based Imputation Method for Sparse Medical Records
1 Introduction
2 Method
3 Results
4 Conclusion
References
Using Nursing Notes to Predict Length of Stay in ICU for Critically Ill Patients
1 Introduction
2 Related Works
3 Dataset
4 Proposed Architecture of ICU LOS Prediction
4.1 Transformer Based Document Representation
4.2 TF-IDF Vector
4.3 Severity of Illness (SOI) Score
4.4 Training the Model
5 Evaluation of the Proposed Predictive Model
5.1 Results and Discussions
6 Conclusion
References
Automatic Classification of Dementia Using Text and Speech Data
1 Introduction
2 Related Work
3 Materials and Methods
3.1 Dataset: DementiaBank Pitt and WLS Corpora
3.2 Data Pre-processing
3.3 Feature Extraction
4 Ensemble Model
4.1 Deep Learning Model
4.2 Model Evaluation
5 Discussion
6 Conclusion
References
Unified Tensor Network for Multimodal Dementia Detection
1 Introduction
2 Background
2.1 Alzheimer's Disease and Dementia
2.2 Multimodal Machine Learning
3 Unified Tensor Analysis Pipeline
3.1 Dataset
3.2 Preprocessing
3.3 Representation
3.4 Fusion
3.5 Evaluation
4 Experiments and Results
5 Conclusion
References