Using the DeepGaze III model to decompose spatial and dynamic contributions to fixation placement over time
Abstract
Humans view images in scanpaths of fixations, where they move their gaze over the image to explore interesting parts of the image. Which factors govern the principles of such scanpaths and how they change over time has been the subject of substantial research. The deep learning based DeepGaze III model currently sets the state-of-the-art in free-viewing scanpath prediction on natural images. It combines a spatial prediction module, which captures the influence of scene content on fixation placement, with a scanpath history module that captures the influence of earlier fixations and therefore the dynamics of the scanpath. Here, we conduct a series of ablation studies to train variants of DeepGaze III with no access to scene content, scanpath history or both and analyse how well fixations are predicted over the course of free-viewing scanpaths. We find that the overall predictability of fixations decays substantially …
Matthias Kümmerer
Postdoc
I’m interested in understanding how we use eye movements to gather information about our environment. This includes building saliency models and models of eye movement prediction such as my line of DeepGaze models. I also work on the question of how to evaluate model quality and benchmarking and I’m the main organizer of the MIT/Tuebingen Saliency Benchmark.
Matthias Bethge
Professor for Computational Neuroscience and Machine Learning & Director of the Tübingen AI Center
Matthias Bethge is Professor for Computational Neuroscience and Machine Learning at the University of Tübingen and director of the Tübingen AI Center, a joint center between Tübingen University and MPI for Intelligent Systems that is part of the German AI strategy.