11 Sep 10:15 Michael Gutmann: Learning features by contrasting natural images with noise

HIIT seminar, Fri Sep 11, 10:15 a.m. (coffee from 10), Exactum D122

Dr. Michael Gutmann
Neuroinformatics Group
Helsinki Institute for Information Technology HIIT University of Helsinki, Department of Computer Science

Learning features by contrasting natural images with noise

Abstract:
Modeling the statistical structure of natural images is interesting for reasons related to neuroscience as well as engineering. Currently, this modeling relies heavily on generative probabilistic models. The estimation of such models is, however, difficult, especially when they consist of multiple layers. If the goal lies only in estimating the features, i.e. in pinpointing structure in natural images, one could also estimate instead a discriminative probabilistic model where multiple layers are more easily handled. For that purpose, we propose to estimate a classifier that can tell natural images apart from reference data which has been constructed to contain some known structure of natural images. The features of the classifier then reveal the interesting structure. Here, we use a classifier with one layer of features and reference data which contains the covariance-structure of natural images. We show that the features of the classifier are similar to those which are obtained from generative probabilistic models. Furthermore, we investigate the optimal shape of the nonlinearity that is used within the classifier.

[This is joint work with Aapo Hyvärinen. A similar talk will be presented at the conference "ICANN2009".]