Contents

• Introduction
• Supervised Learning
  • Nonparametric Regression
  • Classification and Time Series Prediction
• Linear Models
  • Radial Functions
  • Radial Basis Function Networks
• Least Squares
  • The Optimal Weight Vector
  • The Projection Matrix
  • Incremental Operations
  • The Effective Number of Parameters
  • Example
• Model Selection Criteria
  • Cross-Validation
  • Generalised Cross-Validation
  • Example
• Ridge Regression
  • Bias and Variance
  • Optimising the Regularisation Parameter
  • Local Ridge Regression
  • Optimising the Regularisation Parameters
  • Example
• Forward Selection
  • Orthogonal Least Squares
  • Regularised Forward Selection
  • Regularised Orthogonal Least Squares
  • Example
• Appendices
  • Notational Conventions
  • Useful Properties of Matrices
  • Radial Basis Functions
  • The Optimal Weight Vector
  • The Variance Matrix
  • The Projection Matrix
  • Incremental Operations
    • Adding a new basis function
    • Removing an old basis function
    • Adding a new training pattern
    • Removing an old training pattern
  • The Effective Number of Parameters
  • Leave-one-out Cross-validation
  • A Re-Estimation Formula for the Global Parameter
  • Optimal Values for the Local Parameters
  • Forward Selection
  • Orthogonal Least Squares
  • Regularised Forward Selection
  • Regularised Orthogonal Least Squares
• References
• About this document ...