Deep Learning and the Representation of Natural Data

A key component in systems that can understand natural data is a module that turns the raw data into an suitable internal representation. The main objective of 'Deep Learning' is to come up with learning methods that can automatically produce good representations of data from labeled or unlabeled samples.The convolutional network model (ConvNet) is a particular type of deep architecture that is somewhat inspired by biology, and consist of multiple stages of filter banks, interspersed with non-linear operations, and spatial pooling. The most recent speech recognition and image understanding systems deployed by Facebook, Google, IBM, Microsoft, Baidu, NEC and others use deep learning, and many use convolutional networks. Such systems use very large and very deep ConvNets with billions of connections, trained using backpropagation with stochastic gradient, with heavy regularization. But many new applications require the use of unsupervised feature learning methods. A number of methods based on sparse auto-encoder will be presented.