Transfer learning deals with how systems can quickly adapt themselves to new situations, tasks and environments. It gives machine learning systems the ability to leverage auxiliary data and models to help solve target problems when there is only a small amount of data available. This makes such systems more reliable and robust, keeping the machine learning model faced with unforeseeable changes from deviating too much from expected performance. At an enterprise level, transfer learning allows knowledge to be reused so experience gained once can be repeatedly applied to the real world. For example, a pre-trained model that takes account of user privacy can be downloaded and adapted at the edge of a computer network. This self-contained, comprehensive reference text describes the standard algorithms and demonstrates how these are used in different transfer learning paradigms. It offers a solid grounding for newcomers as well as new insights for seasoned researchers and developers.
Author(s): Qiang Yang, Yu Zhang, Wenyuan Dai, Sinno Jialin Pan
Publisher: Cambridge University Press
Year: 2020
Language: English
Pages: 393
Contents......Page 6
Preface......Page 9
PART I FOUNDATIONS OF TRANSFER LEARNING......Page 14
1.1 AI, Machine Learning and Transfer Learning......Page 16
1.2 Transfer Learning: A Definition......Page 20
1.3 Relationship to Existing Machine Learning Paradigms......Page 24
1.4 Fundamental Research Issues in Transfer Learning......Page 26
1.5 Applications of Transfer Learning......Page 27
1.6 Historical Notes......Page 30
1.7 About This Book......Page 31
2.1 Introduction......Page 36
2.2 Instance-Based Noninductive Transfer Learning......Page 38
2.3 Instance-Based Inductive Transfer Learning......Page 41
3.1 Introduction......Page 47
3.2 Minimizing the Domain Discrepancy......Page 48
3.3 Learning Universal Features......Page 54
3.4 Feature Augmentation......Page 56
4.1 Introduction......Page 58
4.2 Transfer through Shared Model Components......Page 60
4.3 Transfer through Regularization......Page 63
5.1 Introduction......Page 71
5.3 Relation-Based Transfer Learning Based on MLNs......Page 74
6.1 Introduction......Page 81
6.2 The Heterogeneous Transfer Learning Problem......Page 83
6.3 Methodologies......Page 84
6.4 Applications......Page 103
7.1 Introduction......Page 106
7.2 Generative Adversarial Networks......Page 107
7.3 Transfer Learning with Adversarial Models......Page 110
7.4 Discussion......Page 117
8.1 Introduction......Page 118
8.2 Background......Page 120
8.3 Inter-task Transfer Learning......Page 126
8.4 Inter-domain Transfer Learning......Page 135
9.1 Introduction......Page 139
9.3 Multi-task Supervised Learning......Page 141
9.4 Multi-task Unsupervised Learning......Page 150
9.6 Multi-task Active Learning......Page 151
9.8 Multi-task Online Learning......Page 152
9.10 Parallel and Distributed Multi-task Learning......Page 153
10.1 Introduction......Page 154
10.2 Generalization Bounds for Multi-task Learning......Page 155
10.3 Generalization Bounds for Supervised Transfer Learning......Page 158
10.4 Generalization Bounds for Unsupervised Transfer Learning......Page 161
11.1 Introduction......Page 164
11.2 TTL over Mixed Graphs......Page 166
11.3 TTL with Hidden Feature Representations......Page 171
11.4 TTL with Deep Neural Networks......Page 175
12.1 Introduction......Page 181
12.2 The L2T Framework......Page 182
12.3 Parameterizing What to Transfer......Page 183
12.4 Learning from Experiences......Page 184
12.6 Connections to Other Learning Paradigms......Page 187
13.1 Introduction......Page 190
13.2 Zero-Shot Learning......Page 191
13.3 One-Shot Learning......Page 197
13.4 Bayesian Program Learning......Page 200
13.5 Poor Resource Learning......Page 203
13.6 Domain Generalization......Page 206
14.1 Introduction......Page 209
14.2 Lifelong Machine Learning: A Definition......Page 210
14.3 Lifelong Machine Learning through Invariant Knowledge......Page 211
14.4 Lifelong Machine Learning in Sentiment Classification......Page 212
14.5 Shared Model Components as Multi-task Learning......Page 216
14.6 Never-Ending Language Learning......Page 217
PART II APPLICATIONS OF TRANSFER LEARNING......Page 222
15.1 Introduction......Page 224
15.2 Differential Privacy......Page 225
15.3 Privacy-Preserving Transfer Learning......Page 228
16.1 Introduction......Page 234
16.2 Overview......Page 235
16.3 Transfer Learning for Medical Image Analysis......Page 242
17.2 Transfer Learning in NLP......Page 247
17.3 Transfer Learning in Sentiment Analysis......Page 254
18.1 Introduction......Page 270
18.3 Transfer Learning in Spoken Language Understanding......Page 272
18.4 Transfer Learning in Dialogue State Tracker......Page 275
18.5 Transfer Learning in DPL......Page 276
18.6 Transfer Learning in Natural Language Generation......Page 281
18.7 Transfer Learning in End-to-End Dialogue Systems......Page 282
19.1 Introduction......Page 292
19.2 What to Transfer in Recommendation......Page 293
19.3 News Recommendation......Page 297
19.4 VIP Recommendation in Social Networks......Page 301
20.1 Introduction......Page 306
20.2 Machine Learning Problems in Bioinformatics......Page 307
20.3 Biological Sequence Analysis......Page 308
20.5 Systems Biology......Page 312
20.6 Biomedical Text and Image Mining......Page 314
20.7 Deep Learning for Bioinformatics......Page 315
21.2 Transfer Learning for Wireless Localization......Page 320
21.3 Transfer Learning for Activity Recognition......Page 329
22.1 Introduction......Page 337
22.2 “What to Transfer” in Urban Computing......Page 338
22.3 Key Issues of Transfer Learning in Urban Computing......Page 339
22.4 Chain Store Recommendation......Page 340
22.5 Air-Quality Prediction......Page 343
23 Concluding Remarks......Page 347
References......Page 349
Index......Page 390
