Sparse Overcomplete Linear Models

Overcomplete basis
Subset of a four times overcomplete basis learned on whitened 16x16 image patches.


isa.tar.gz (35 MB)

Links (Python) (C++)


The code linked above implements the Gibbs sampler and the persistent variant of expectation maximization for overcomplete linear models described in this paper. The archive also contains 8x8 image patches as well as some example results. If you are interested in reproducing or better understanding the results of the paper, you should take a look at the Python code.

However, the Python code becomes quite memory hungry for larger data and I recommend giving the more efficient C++ implementation a try if you are interested in using the sampler in your own projects. The C++ code comes with Python interface.

The paper only describes overcomplete linear models with independent sources (ICA), but both implementations also already work with independent subspaces (ISA).

If you have questions, please contact Lucas Theis.


L. Theis, J. Sohl-Dickstein, and M. Bethge
Training sparse natural image models with a fast Gibbs sampler of an extended state space
Advances in Neural Information Processing Systems 25, 2012
#natural image statistics, #ica, #overcompleteness
Code, PDF, Supplemental, Poster, BibTex
University of Tuebingen BCCN CIN MPI