Tutorial 1: 9:30 hrs: Kernel Methods - Prediction with Gaussian
Processes and Support Vector Machines
Tutors: Bernard Schölkopf (GMD-FIRST, Berlin, Germany) and
Christopher K.I. Williams (University of Edinburgh, UK)
Abstract:
This self-contained tutorial is aimed at people who have an interest and
some familiarity with supervised learning problems, but does not presuppose
prior knowledge of Support Vector Machine (SVM) or Gaussian Process (GP)
predictors.
Over recent years, SVMs and GPs have become popular approaches to supervised
learning problems. Although they have rather different origins, many of the
ideas behind the two techniques are very similar. The aims of this tutorial
are to present the two methods and explain the connections between them.
In detail, we will
- discuss the general framework for regression and classification
problems
- introduce kernel-based algorithms (Support Vector Machines and
Gaussian process predictors) for regression and
classification problems
- discuss the Bayesian interpretation of Gaussian process predictors
- describe algorithms used for (2) above
- give methods for choosing kernels for particular problems
- describe some example applications
- describe additional kernel algorithms
12.30 hrs Close of session
Tutorial 2: 09:30 hrs: Neural Development
Tutor: Geoffrey J. Goodhill (Georgetown University, Washington, USA)
Abstract:
This tutorial is aimed at neural network researchers with an interest in
neuroscience. Neural network researchers have long recognized the importance
of understanding how real brains develop for building smarter learning machines.
However, development is one of the most dynamic areas of neuroscience, and
in the past few years dramatic progress has been made in uncovering the
underlying biological mechanisms and principles by which brains are built.
In addition, the interest of the neural network community has so far mostly
been focussed on a very narrow range of problems in neural development. The
goal of this tutorial is to present a broad overview of the whole of neural
development, and to describe some of the most recent and exciting experimental
discoveries and what they might imply for theoretical modelling.
The first half of the tutorial will broadly outline the complete sequence
of stages of neural development:
- neural induction and pattern formation
- migration
- cell fate determination
- axon pathfinding
- synapse formation
- cell death and neurotrophic factors
- activity-dependent processes.
The second half will focus more closely on some particular cases where
theoretical modeling has made or has immediate potential to make important
contributions, including:
- patterning and regionalization
- axon pathfinding
- map formation
- activity-dependent refinement of synaptic strengths
12:30 hrs: Close of session
Tutorial 3: 9:30 hrs: Neuro-fuzzy Systems
Tutor: Hans-Heinrich Bothe (Technical University of Denmark, Lyngby)
Abstract:
Modern scientists and engineers must have knowledge on management of uncertain
information. Fuzzy Logic (FL) and Artificial Neural Networks (ANN) are keys
to intelligent systems modelling and information processing, dealing with
real-world tasks. For more than 25 years, FL and ANN paradigms were developed
by separate scientific communities, and have since become essential parts
of computer related scientific and engineering education. In the past years,
an extensive interest has been shown in developing and applying neuro-fuzzy
methods which synthesize the strategic advantages and computational properties
of FL and ANN.
This tutorial consists of two parts, each of which is illustrated by a number
of technical applications. Part 1, 'Foundations of FL and ANN', introduces
important basic and advanced topics on fuzzy logic in a broader sense, and
on advanced ANN topologies and learning rules. Part 1 begins with a course
on biological foundations, details different fuzzy inference methods from
a unique perspective, and leads to an introduction to continuous and discrete
time ANN as functional representations of assemblies of biological neurons.
Whereas FL is seen from a perspective of cognitive science, ANN are treated
as models of the organ that is responsible for decision making: the brain.
Part 2, 'Computational Neuro-fuzzy Paradigms', compares a set of advanced
schemes of joint FL and ANN integration from a computational point of view.
We will introduce algorithms for automatic rule extraction from numerical
data, examples for a 'fuzzification' of existing ANN paradigms, and a number
of sophisticated hybrid neuro-fuzzy approaches for pattern recognition and
control tasks.
The applications include perception- and cognition-based pattern recognition
or action control, and describe continuous and discrete time models.
12:30 hrs: Close of session
Tutorial 4: 14:00 hrs: Graphical models
Tutor: Christopher M. Bishop (Microsoft Research, Cambridge and
University of Edinburgh, UK)
Abstract:
Graphical models allow complex probability distributions to be constructed
from relatively simple components, thereby greatly simplifying the development
of new probabilistic models. Furthermore, they allow the independence properties
of a distribution to be seen explicitly in terms of graphical connectivity.
Graphical models also provide a powerful framework for solving inference
and learning problems, allowing highly complex tasks to be tackled using
graphical manipulations.
This tutorial will provide an introduction to theory and practice of
probabilistic graphical models. After explaining the basic concepts of directed
and undirected graphs I will show how many standard models, such as factor
analysis, hidden Markov models, Kalman filters, and neural networks, can
be viewed as specific probabilistic graphs. I will then explain the central
concept of conditional independence and show how the Markov properties of
complex distributions can be determined easily using graphical procedures.
Next I will turn to the problems of inference and learning in graphical models,
and explain the role of the EM algorithm. I will then introduce the junction
tree algorithm for finding exact solutions to inference problems in graphical
models, noting that many standard procedures, such as the forward-backward
algorithm for hidden Markov models, are special cases. Next I will discuss
the limitations of the junction tree algorithm, and introduce the variational
framework for approximate inference in graphical models. Finally I will
illustrate these ideas with some example practical applications.
No previous knowledge of graph theory will be assumed. However, some basic
familiarity with probabilities may be helpful.
17:00 hrs: Close of session
Tutorial 5: 14:00 hrs: Computational Approach to the Functioning of
the Hippocampus: From Natural to Artificial Information Processing
Tutor: Peter Erdi (KFKI, Budapest, Hungary)
Abstract:
The hippocampus has a crucial role in cognitive processes, such as learning,
memory formation and spatial navigation. In the tutorial an integrated approach
to the structure, function and dynamics of the hippocampus is given. Structural
facts, physiological mechanisms and related computational algorithms are
reviewed.
The goals of this tutorial are twofold. First, it will be demonstrated how
the understanding of the functional organization of the hippocampus may
contribute for creating biology-inspired information processing systems.
Second, new methods of Computational Neuroscience are explained through specific
applications. The following topics will be covered:
ANATOMICAL ORGANIZATION
- Internal structure
- Input-output system
RHYTHMIC ACTIVITY: single cell and network properties
- Physiologial background
- Compartmental models: single cell and network activity
- Population models: normal and epileptic behaviour; wave phenomena
BRAIN STATES and LONG-TERM POTENTIATION
- mechanism for memory trace formation
- Hebbian synaptic modification algorithms
COGNITIVE MAPS and NAVIGATION
- Spatial representation: neurophysiological mechanisms and computational
algorithms
- formation of place cells: facts and models
- Models of navigation
HUMAN MEMORY
- Forming but not storing memories: review of experimental background
- Modeling the encoding and retrieval of memory traces
The tutorial is based on the updated version of Chapter 6 (Hippocampus) of
the book Arbib, MA, Erdi P and Szentagothai J: Neural Organization. Structure,
Function and Dynamics. MIT Press 1997.
17:00 hrs: Close of session
Tutorial 6: 14:00 hrs: Neural networks, automata and formal models of
computation
Tutor: Mikel L. Forcada (Universitat d'Alacant, Spain)
Abstract:
Participants of this course will learn to examine neural networks and neural
learning tasks (usually non-symbolic in nature) using the tools of current
formal language theory and the theories of computation (which may be seen
as basically symbolic).
The relationship between neural networks (and particularly recurrent neural
networks) and traditional models of computation has been the subject of
historical papers (such as McCulloch and Pitts 1943) and, in the last ten
years, of a large body of recent work (including the work by Jordan, Pollack,
Giles, Elman, Fanelli, Hush, Kremer, Casey, Siegelmann and others). The tutorial
aims at providing tools to build a coherent framework for the study of this
relationship, which may be very beneficial both to (a) researchers and
practitioners approaching neural networks from the formal language arena
in search of fault-tolerance, generalization and robustness against noise
and (b) neural network researchers willing to understand or discover the
underlying symbolic or subsymbolic nature of the processes they study. The
growth of this field attests to the relevance of such a coherent framework.
The following topics will be covered:
Introduction
- An overview of automata and language theory: a hierarchy of machines, grammars,
and language classes.
- The "neural" beginnings of automata and language theory: McCulloch-Pitts,
Kleene, Minsky.
The computational power of feedforward neural networks
- Single-layer perceptrons
- Multi-layer perceptrons
Neural state machines: recurrent neural networks
- Recurrent neural networks as state machines: Moore and Mealy machines.
- Recurrent neural networks as language recognizers: robust neural modelling
of finite state acceptors.
- A taxonomy of recurrent neural networks: computational capabilities; first
and second-order recurrent neural nets; subclasses of automata: definite-memory
machines and finite-memory machines; observability of state.
Learning languages and automata from examples using recurrent neural networks.
Comparison with algorithmic approaches.
- Inducing neural language acceptors: learning finite-state automata and
pushdown automata from positive and negative samples.
- Inducing neural language generators: learning stochastic regular languages
from positive examples.
- Learning asynchronous translators with recurrent neural networks.
Turing and super-Turing computability with neural networks
- The computational equivalence of recurrent neural networks and Turing
machines.
- Super-Turing capabilities of neural-like analog computing devices.
The tutorial is aimed at researchers (mainly in Computer Science, Computer
Engineering, Electrical Engineering, and Applied Mathematics Departments)
of all levels, both coming from the formal-language and computational theory
fields and from the broader neural networks community; graduate students
in these fields; research planners or evaluators. The following background
is assumed: (a) basic neural network concepts, (b) familiarity with the notation
and basic concepts of discrete mathematics and set theory and (c) very basic
concepts of language theory and computation: finite automata, context-free
grammars, Turing machines (will be reviewed).
17:00 hrs: Close of session
