The appearance of objects in an image can change dramatically depending on their pose, distance, and illumination. Learning representations that are invariant against such appearance changes can be viewed as an important preprocessing step which removes distracting variance from a data set, so that downstream classifiers or regression estimators perform better. Complex cells in primary visual cortex are commonly seen as building blocks for such invariant image representations (eg Riesenhuber Poggio 2000). While complex cells, like simple cells, respond to edges of particular orientation they are less sensitive to the precise location of the edge. A variety of neural algorithms have been proposed that aim at explaining the response properties of complex cells as components of an invariant representation that is optimized for the spatio-temporal statistics of the visual input. For certain classes of transformations (eg translations, scalings, and rotations), it is possible to analytically derive features that are invariant under these transformations, and the design of such invariant features has been studied extensively in computer vision. The range of naturally occurring transformations, however, is much more variable and not precisely known. Thus, an analytical design of invariant features does not seem feasible. Instead one can seek to find features that may not be perfectly invariant anymore but which on average change as slowly as possible under the transformations occurring in the data (Földiák 1991). The best known instantiation of this approach is slow feature analysis (SFA) which has been proposed to underlie the formation of complex …