Following a brief introduction and overview, early chapters cover the basic algebraic relationships of entropy, relative entropy and mutual information, AEP, entropy rates of stochastics processes and data compression, duality of data compression and the growth rate of wealth. Later chapters explore Kolmogorov complexity, channel capacity, differential entropy, the capacity of the fundamental Gaussian channel, the relationship between information theory and statistics, rate distortion and network information theories. The final two chapters examine the stock market and inequalities in information theory. In many cases the authors actually describe the properties of the solutions before the presented problems.
Author(s): Thomas M. Cover, Joy A. Thomas
Edition: 99th
Publisher: Wiley-Interscience
Year: 1991
Language: English
Pages: 565
Elements of Information Theory......Page 1
Wiley Series in Telecommunications......Page 3
Copyright......Page 5
Preface......Page 7
Acknowledgments......Page 11
Contents......Page 13
List of Figures......Page 19
Ch1 Introduction & Preview......Page 24
Ch2 Entropy, Relative Entropy & Mutual Information......Page 35
Ch3 Asymptotic Equipartition Property......Page 73
Ch4 Entropy Rates of Stochastic Process......Page 83
Ch5 Data Compression......Page 101
Ch6 Gambling & Data Compression......Page 148
Ch7 Kolmogorov Complexity......Page 167
Ch8 Channel Capacity......Page 206
Ch9 Differential Entropy......Page 247
Ch10 Gaussian Channel......Page 262
Ch11 Maximum Entropy & Spectral Estimation......Page 289
Ch12 Information Theory & Statistics......Page 302
Ch13 Rate Distortion Theory......Page 359
Ch14 Network Information Theory......Page 397
Ch15 Information Theory & Stock Market......Page 482
Ch16 Inequalities in Information Theorv......Page 505
Bibliography......Page 533
List of Symbols......Page 549
Index......Page 552
