Information geometry

Information Geometry
Preface
Acknowledgements
Contents
Chapter 1: Introduction
1.1 A Brief Synopsis
1.2 An Informal Description
1.2.1 The Fisher Metric and the Amari-Chentsov Structure for Finite Sample Spaces
1.2.2 In nite Sample Spaces and Functional Analysis
1.2.3 Parametric Statistics
1.2.4 Exponential and Mixture Families from the Perspective of Differential Geometry
1.2.5 Information Geometry and Information Theory
1.3 Historical Remarks
1.4 Organization of this Book
Chapter 2: Finite Information Geometry
2.1 Manifolds of Finite Measures
2.2 The Fisher Metric 2.3 Gradient Fields2.4 The m- and e-Connections
2.5 The Amari-Chentsov Tensor and the alpha-Connections
2.5.1 The Amari-Chentsov Tensor
2.5.2 The alpha-Connections
2.6 Congruent Families of Tensors
2.7 Divergences
2.7.1 Gradient-Based Approach
2.7.2 The Relative Entropy
2.7.3 The alpha-Divergence
2.7.4 The f-Divergence
2.7.5 The q-Generalization of the Relative Entropy
2.8 Exponential Families
2.8.1 Exponential Families as Af ne Spaces
2.8.2 Implicit Description of Exponential Families
2.8.3 Information Projections
2.9 Hierarchical and Graphical Models
2.9.1 Interaction Spaces 2.9.2 Hierarchical Models2.9.3 Graphical Models
Chapter 3: Parametrized Measure Models
3.1 The Space of Probability Measures and the Fisher Metric
3.2 Parametrized Measure Models
3.2.1 The Structure of the Space of Measures
3.2.2 Tangent Fibration of Subsets of Banach Manifolds
3.2.3 Powers of Measures
3.2.4 Parametrized Measure Models and k-Integrability
3.2.5 Canonical n-Tensors of an n-Integrable Model
3.2.6 Signed Parametrized Measure Models
3.3 The Pistone-Sempi Structure
3.3.1 e-Convergence
3.3.2 Orlicz Spaces
3.3.3 Exponential Tangent Spaces Chapter 4: The Intrinsic Geometry of Statistical Models4.1 Extrinsic Versus Intrinsic Geometric Structures
4.2 Connections and the Amari-Chentsov Structure
4.3 The Duality Between Exponential and Mixture Families
4.4 Canonical Divergences
4.4.1 Dual Structures via Divergences
4.4.2 A General Canonical Divergence
4.4.3 Recovering the Canonical Divergence of a Dually Flat Structure
4.4.4 Consistency with the Underlying Dualistic Structure
4.5 Statistical Manifolds and Statistical Models
4.5.1 Statistical Manifolds and Isostatistical Immersions 4.5.2 Monotone Invariants of Statistical Manifolds4.5.3 Immersion of Compact Statistical Manifolds into Linear Statistical Manifolds
4.5.4 Proof of the Existence of Isostatistical Immersions
4.5.5 Existence of Statistical Embeddings
Chapter 5: Information Geometry and Statistics
5.1 Congruent Embeddings and Suf cient Statistics
5.1.1 Statistics and Congruent Embeddings
5.1.2 Markov Kernels and Congruent Markov Embeddings
5.1.3 Fisher-Neyman Suf cient Statistics
5.1.4 Information Loss and Monotonicity
5.1.5 Chentsov's Theorem and Its Generalization