'Introduction to Algorithms for Data Mining and Machine Learning' introduces the essential ideas behind all key algorithms and techniques for data mining and machine learning, along with optimization techniques. Its strong formal mathematical approach, well selected examples, and practical software recommendations help readers develop confidence in their data modeling skills so they can process and interpret data for classification, clustering, curve-fitting and predictions. Masterfully balancing theory and practice, it is especially useful for those who need relevant, well explained, but not rigorous (proofs based) background theory and clear guidelines for working with big data.
Author(s): Xin-She Yang
Publisher: Academic Press
Year: 2019
Language: English
Commentary: Real PDF
Pages: 173
Front Cover......Page 1
Title......Page 4
Copyright......Page 5
Preface......Page 8
About the author......Page 6
Acknowledgments......Page 10
1.1.1 Essence of an algorithm......Page 11
1.1.3 Types of algorithms......Page 13
1.2.1 A simple example......Page 14
1.2.2 General formulation of optimization......Page 17
1.2.3 Feasible solution......Page 19
1.3 Unconstrained optimization......Page 20
1.3.1 Univariate functions......Page 21
1.3.2 Multivariate functions......Page 22
1.4 Nonlinear constrained optimization......Page 24
1.4.1 Penalty method......Page 25
1.4.2 Lagrange multipliers......Page 26
1.4.3 Karush-Kuhn-Tucker conditions......Page 27
1.5 Notes on software......Page 28
2 Mathematical foundations......Page 29
2.1.1 Linear and affine functions......Page 30
2.1.2 Convex functions......Page 31
2.2 Computational complexity......Page 32
2.2.1 Time and space complexity......Page 34
2.2.2 Complexity of algorithms......Page 35
2.3.1 Norms......Page 36
2.3.2 Regularization......Page 38
2.4.1 Random variables......Page 39
2.4.2 Probability distributions......Page 40
2.4.3 Conditional probability and Bayesian rule......Page 42
2.4.4 Gaussian process......Page 44
2.5 Bayesian network and Markov models......Page 45
2.6 Monte Carlo sampling......Page 46
2.6.2 Metropolis-Hastings algorithm......Page 47
2.7.1 Entropy and cross entropy......Page 49
2.7.2 DL divergence......Page 50
2.8 Fuzzy rules......Page 51
2.10 Notes on software......Page 52
3.1.1 Newton's method......Page 55
3.1.2 Newton's method for multivariate functions......Page 57
3.1.3 Line search......Page 58
3.2 Variants of gradient-based methods......Page 59
3.2.1 Stochastic gradient descent......Page 60
3.2.2 Subgradient method......Page 61
3.2.3 Conjugate gradient method......Page 62
3.3 Optimizers in deep learning......Page 63
3.4 Gradient-free methods......Page 66
3.5.1 Genetic algorithm......Page 68
3.5.2 Differential evolution......Page 70
3.5.4 Bat algorithm......Page 71
3.5.6 Cuckoo search......Page 72
3.5.7 Flower pollination algorithm......Page 73
3.6 Notes on software......Page 74
4.1 Sample mean and variance......Page 77
4.2.1 Maximum likelihood......Page 79
4.2.2 Liner regression......Page 80
4.2.3 Linearization......Page 85
4.2.4 Generalized linear regression......Page 87
4.2.5 Goodness of fit......Page 90
4.3 Nonlinear least squares......Page 91
4.3.1 Gauss-Newton algorithm......Page 92
4.3.3 Weighted least squares......Page 95
4.4 Overfitting and information criteria......Page 96
4.5 Regularization and Lasso method......Page 98
4.6 Notes on software......Page 100
5.1 Logistic regression......Page 101
5.3 Principal component analysis......Page 106
5.4 Linear discriminant analysis......Page 111
5.5 Singular value decomposition......Page 114
5.6 Independent component analysis......Page 115
5.7 Notes on software......Page 118
6 Data mining techniques......Page 119
6.1.2 Distance metric......Page 120
6.2 Hierarchy clustering......Page 121
6.3 k-Nearest-neighbor algorithm......Page 122
6.4 k-Means algorithm......Page 123
6.5.1 Decision tree algorithm......Page 125
6.5.2 ID3 algorithm and C4.5 classifier......Page 126
6.5.3 Random forest......Page 130
6.6.1 Naive Bayesian classifier......Page 131
6.6.2 Bayesian networks......Page 133
6.7.1 Characteristics of big data......Page 134
6.7.3 Mining big data......Page 135
6.8 Notes on software......Page 137
7.1 Statistical learning theory......Page 139
7.2 Linear support vector machine......Page 140
7.3 Kernel functions and nonlinear SVM......Page 143
7.4 Support vector regression......Page 145
7.5 Notes on software......Page 147
8.1 Learning......Page 149
8.2.1 Neuron models......Page 150
8.2.2 Activation models......Page 151
8.2.3 Artificial neural networks......Page 153
8.3 Back propagation algorithm......Page 156
8.4 Loss functions in ANN......Page 157
8.6 Network architecture......Page 159
8.7.1 Convolutional neural networks......Page 161
8.7.1.1 Convolution and activation......Page 162
8.7.1.2 Pooling......Page 164
8.7.1.3 Flattening......Page 165
8.7.1.4 Fully connected neural network......Page 166
8.7.2 Restricted Boltzmann machine......Page 167
8.7.3 Deep neural nets......Page 168
8.7.4 Trends in deep learning......Page 169
8.8 Tuning of hyperparameters......Page 170
8.9 Notes on software......Page 171
Bibliography......Page 173
Index......Page 181
Back Cover......Page 184
