Free

# Gaussian Processes for Machine Learning

By Carl Edward Rasmussen and Christopher K. I. Williams

Free

Book Description

Table of Contents

- Series Foreword
- Preface
- Symbols and Notation
- Introduction
- A Pictorial Introduction to Bayesian Modelling
- Roadmap

- Regression
- Weight-space View
- The Standard Linear Model
- Projections of Inputs into Feature Space

- Function-space View
- Varying the Hyperparameters
- Decision Theory for Regression
- An Example Application
- Smoothing, Weight Functions and Equivalent Kernels
- * Incorporating Explicit Basis Functions
- Marginal Likelihood

- History and Related Work
- Exercises

- Weight-space View
- Classification
- Classification Problems
- Decision Theory for Classification

- Linear Models for Classification
- Gaussian Process Classification
- The Laplace Approximation for the Binary GP Classifier
- Posterior
- Predictions
- Implementation
- Marginal Likelihood

- * Multi-class Laplace Approximation
- Implementation

- Expectation Propagation
- Predictions
- Marginal Likelihood
- Implementation

- Experiments
- A Toy Problem
- One-dimensional Example
- Binary Handwritten Digit Classification Example
- 10-class Handwritten Digit Classification Example

- Discussion
- * Appendix: Moment Derivations
- Exercises

- Classification Problems
- Covariance Functions
- Preliminaries
- * Mean Square Continuity and Differentiability

- Examples of Covariance Functions
- Stationary Covariance Functions
- Dot Product Covariance Functions
- Other Non-stationary Covariance Functions
- Making New Kernels from Old

- Eigenfunction Analysis of Kernels
- * An Analytic Example
- Numerical Approximation of Eigenfunctions

- Kernels for Non-vectorial Inputs
- String Kernels
- Fisher Kernels

- Exercises

- Preliminaries
- Model Selection and Adaptation of Hyperparameters
- The Model Selection Problem
- Bayesian Model Selection
- Cross-validation
- Model Selection for GP Regression
- Marginal Likelihood
- Cross-validation
- Examples and Discussion

- Model Selection for GP Classification
- * Derivatives of the Marginal Likelihood for Laplace's Approximation
- * Derivatives of the Marginal Likelihood for EP
- Cross-validation
- Example

- Exercises

- Relationships between GPs and Other Models
- Reproducing Kernel Hilbert Spaces
- Regularization
- * Regularization Defined by Differential Operators
- Obtaining the Regularized Solution
- The Relationship of the Regularization View to Gaussian Process Prediction

- Spline Models
- * A 1-d Gaussian Process Spline Construction

- * Support Vector Machines
- Support Vector Classification
- Support Vector Regression

- * Least-squares Classification
- Probabilistic Least-squares Classification

- * Relevance Vector Machines
- Exercises

- Theoretical Perspectives
- The Equivalent Kernel
- Some Specific Examples of Equivalent Kernels

- * Asymptotic Analysis
- Consistency
- Equivalence and Orthogonality

- * Average-case Learning Curves
- * PAC-Bayesian Analysis
- The PAC Framework
- PAC-Bayesian Analysis
- PAC-Bayesian Analysis of GP Classification

- Comparison with Other Supervised Learning Methods
- * Appendix: Learning Curve for the Ornstein-Uhlenbeck Process
- Exercises

- The Equivalent Kernel
- Approximation Methods for Large Datasets
- Reduced-rank Approximations of the Gram Matrix
- Greedy Approximation
- Approximations for GPR with Fixed Hyperparameters
- Subset of Regressors
- The Nyström Method
- Subset of Datapoints
- Projected Process Approximation
- Bayesian Committee Machine
- Iterative Solution of Linear Systems
- Comparison of Approximate GPR Methods

- Approximations for GPC with Fixed Hyperparameters
- * Approximating the Marginal Likelihood and its Derivatives
- * Appendix: Equivalence of SR and GPR Using the Nyström Approximate Kernel
- Exercises

- Further Issues and Conclusions
- Multiple Outputs
- Noise Models with Dependencies
- Non-Gaussian Likelihoods
- Derivative Observations
- Prediction with Uncertain Inputs
- Mixtures of Gaussian Processes
- Global Optimization
- Evaluation of Integrals
- Student's t Process
- Invariances
- Latent Variable Models
- Conclusions and Future Directions

- Appendix Mathematical Background
- Joint, Marginal and Conditional Probability
- Gaussian Identities
- Matrix Identities
- Matrix Derivatives
- Matrix Norms

- Cholesky Decomposition
- Entropy and Kullback-Leibler Divergence
- Limits
- Measure and Integration
- Lp Spaces

- Fourier Transforms
- Convexity

- Appendix Gaussian Markov Processes
- Fourier Analysis
- Sampling and Periodization

- Continuous-time Gaussian Markov Processes
- Continuous-time GMPs on R
- The Solution of the Corresponding SDE on the Circle

- Discrete-time Gaussian Markov Processes
- Discrete-time GMPs on Z
- The Solution of the Corresponding Difference Equation on PN

- The Relationship Between Discrete-time and Sampled Continuous-time GMPs
- Markov Processes in Higher Dimensions

- Fourier Analysis
- Appendix Datasets and Code
- Bibliography
- Author Index
- Subject Index

You May Also Like

Pattern Recognition and Machine Learning (Information Science and Statistics)

By Christopher M. Bishop

Also Available On

Categories

Arts & Photography473Biographies & Memoirs72Business & Money73Children's Books1444Christian Books & Bibles864Computers & Technology707Cookbooks, Food & Wine31Crafts, Hobbies & Home204Education & Teaching3653Health, Fitness & Dieting12History5749Humor & Entertainment160Law142Literature & Fiction19594Medical Books1Mystery, Thriller & Suspense16Other3105Parenting & Relationships6Politics & Social Sciences1382Professional & Technical19Reference8Religion & Spirituality1633Romance236Science & Math1128Science Fiction & Fantasy204Self-Help48Sports & Outdoors45Teen & Young Adult125Test Preparation77Travel112

Curated Lists

#### Free Machine Learning Books

11 Books

- Pattern Recognition and Machine Learning (Information Science and Statistics)
- by Christopher M. Bishop
- Data mining
- by I. H. Witten
- The Elements of Statistical Learning: Data Mining, Inference, and Prediction
- by Various

#### Free Chemistry Textbooks

9 Books

- CK-12 Chemistry
- by Various
- Concept Development Studies in Chemistry
- by John Hutchinson
- An Introduction to Chemistry - Atoms First
- by Mark Bishop

#### Free Mathematics Textbooks

21 Books

- Microsoft Word - How to Use Advanced Algebra II.doc
- by Jonathan Emmons
- Advanced Algebra II: Activities and Homework
- by Kenny Felder
- de2de
- by

#### Free Children Books

38 Books

- The Sun Who Lost His Way
- by
- Tania is a Detective
- by Kanika G
- Firenze_s-Light
- by

#### Free Java Books

10 Books

- Java 3D Programming
- by Daniel Selman
- The Java EE 6 Tutorial
- by Oracle Corporation
- JavaKid811
- by