Machine infers person’s interests from his eye-movement

A method developed in Aalto University retrieves information to data glasses about people and objects a person is looking at. Equipment including an eye tracker implements machine learning methods and infers from context and gaze direction what the viewer may be interested in, to learn more about. Following that the machine displays additional information on viewer’s data glasses. For instance, a researcher may learn about scientific results of a researcher passing by and singles may see on their data glasses the relationship status of a person they meet. By following the eye-movement the machine also infers when additional information should be displayed.

In the test versions of the augmented reality (AR) method displayed information is retrieved from a database with information keyed in beforehand. However, the method can also be set to retrieve information from internet and social media. In this case searches – using Google among others - can be made by looking through the data glasses only.
The equipment recognizes persons looked at. It requires that images of the persons to be recognized have been inserted into database earlier. At the moment, it is not possible to identify totally unknown persons.

Our basic research on machine learning methods has made it possible to create new advanced user interfaces in which the machine infers without direct human commands what the viewer is interested in, tells professor Samuel Kaski from Aalto University.

As a result of our research we have created a prototype that has been successfully tested in laboratories. However, there is still a way to go before this can be commercialized, reminds researcher Jorma Laaksonen from Aalto University.

Study about the new machine learning methods is to be published in the Virtual Reality journal. The research results have been reached with funding from private companies and organizations to Aalto University’s MIDE research programme.

Video about the equipment:
http://www.youtube.com/watch?v=gtuGSWDVdQU

Photos with AR views:
http://www.flickr.com/photos/35622531@N08/tags/uiart/

Information about the research consortium:
http://mide.tkk.fi/en/UI-ART

Preprint version of the article to be published in Virtual Reality:
http://research.ics.tkk.fi/mi/papers/vire10.pdf (copy the link to your browser)

More information:
Samuel Kaski, Professor
tel. +358 50 305 8694
samuel.kaski@hiit.fi

Jorma Laaksonen, Researcher
tel. +358 50 305 8719
jorma.laaksonen@tkk.fi

Aalto University is a new multidisciplinary science and art community in the fields of technology and science, business and economics, and art and design. Aalto University is founded on Finnish strengths, and its goal is to develop as a unique entity to become one of the world's top universities. Aalto University's cornerstones are its strengths in education and research. At Aalto University, there are 20,000 basic degree and graduate students as well as a staff of 4,500 of which 300 are professors. The total budget for 2009 was close to EUR 368 million. In 2009, a total of 1,567 Master's Degrees and 180 Doctorates were completed at the three Schools.

Aalto University, Communications
viestinta@aalto.fi
tel. +358 50 569 9410