6 Oct Mini-workshop on machine learning and neural representation by the University of Helsinki

Mini-workshop on machine learning and neural representation Tue 6th October 2009, University of Helsinki (Kumpula)

11:30 Aapo Hyvärinen (University of Helsinki)
Introduction

12:00 Jörg Lücke (Goethe-Universität Frankfurt)
Linear, maximal, and occlusive causes for component extraction
(abstract given below)

13:00 Lunch break (on your own)

14:00 Peter Földiák (University of St Andrews)
Semantic representation in neural codes (abstract given below)

15:00 Coffee break

15:30 Michael Gutmann (University of Helsinki)
Learning models of natural images by noise-contrastive estimation
(abstract given below)

16:30 End of workshop

Location: Lecture hall CK112 (basement, next to Unicafe) Exactum Building, Gustaf Hällströmin katu 2b Kumpula Campus, University of Helsinki

Please register for the workshop by reply email (so that I know how much coffee to order).

Welcome!

Aapo Hyvärinen

-------------------------------------------------------------------------

Abstracts of the talks

Jörg Lücke: Linear, maximal, and occlusive causes for component extraction.
Abstract: In the nervous system of humans and animals sensory data are represented as combinations of the data's elementary components. Such representations are considered as `higher-level' because they allow for a more direct read-out of crucial or at least useful information. In machine learning, algorithms such as independent component analysis
(ICA) or non-negative matrix factorization (NMF) have been extensively studied because they allow for the extraction of such elementary components from unlabelled data. They have consequently been highly successful in the analysis of data and have broadly been applied in science and technology. In my talk I will discuss component extraction with discrete latent variables, i.e., I consider algorithms that try to explain the data by combinations of components with discrete factors.
The novel approaches are developed within the framework of probabilistic generative models, and I will start by studying models that combine their components linearly. A novel training method based on approximate EM will be introduced and it is show that the method can efficiently maximize the data likelihood. I will discuss the applicability of the approach to a broader class of models including those with non-linear combinations of components. As one example a model is introduced in which components combine according to a maximum rule (MCA). As another example, I discuss models that use non-linear combinations according to component positions in depth (Occlusive Components Analysis). For all models, results on applications to artificial benchmarks (e.g., bars
tests) and natural data will be shown and evaluated. Finally, I will discuss the implications of the presented results for biological information processing and draw relations to component extraction in neural networks.

Peter Földiák: Semantic representation in neural codes.
Abstract: The question of how activity patterns of neurons represent objects in the world has so far mainly been addressed by asking the question of how the identity of stimuli be decoded from the neural signals. However, an even more interesting question is how the structure of the relationships between items and categories can be represented in a sparse and explicit neural code. The duality between "sets of objects" and "sets of features" have been extensively studied by the field of lattice theory called "Formal Concept Analysis" (FCA).
FCA is proposed as a useful method for analysing a neural code because of this explicit structure. Examples from monkey inferotemporal cortex will be presented. Probabilistic extensions of FCA, and some possible practical computational applications for categorisation, optimal cacheing and semantic memory systems will be discussed.

Michael Gutmann: Learning models of natural images by noise-contrastive estimation.
Abstract: Noise-contrastive estimation is an estimation principle that we have recently developed to learn sophisticated statistical models.
The idea is to train a classifier to discriminate between the observed data and some artificially generated noise. We have shown that this leads to a consistent (convergent) estimator of the parameters. An advantage of the method is that it can be applied to the estimation of unnormalized models (i.e. models where the density function does not integrate to one). Statistical models of natural images are typically
unnormalized: Examples will be shown where we used noise contrastive estimation to learn models of natural images.