### Other study groups

- Machine Learning for Imaging, in the Computer Laboratory
- Machine Learning Reading Group @ CUED
- NLIP seminar series, in the Computer Laboratory

### Some topics we might study:

#### Optimization methods

Introduction, basic unsupervised learning methods (PCA, k-means), empirical risk minimization, standard loss functions, linear classification, stochastic optimizers, hidden layers, deep feedforward networks, backpropagation, regularization ideas, batch normalization

- Large-scale machine learning with stochastic gradient descent, Bottou, 2010
- Deep learning, LeCun, Bengio, Hinton, 2015
- Efficient BackProp, LeCun et al., 1998
- Dropout: a simple way to prevent neural networks from overfitting, Srivastava et al., 2014
- Batch normalization: accelerating deep network training by reducing internal covariate shift, Ioffe et al., 2015

#### Images

Convolutional networks (CNNs), classifying images, popular architectures, attaining maximal performance in image classiﬁcation. Deep image generation: generating CNNs, adversarial networks, deep computer vision with feedback loop

- Gradient-based learning applied to document recognition, LeCun et al., 1998
- ImageNet classification with deep convolutional neural networks, Krizhevsky et al., 2012

#### Autoencoding

GANs, autoencoders, variational autoencoders, image analogies

- Auto-encoding variational Bayes, Klingma et al., 2014
- Generative adversarial nets, Goodfellow et al., 2014

#### Sequences

Recurrent neural networks, deep learning on sequences, deep RNNs, LSTMs, GRUs, deep machine translation

- Generating sequences with recurrent neural networks, Graves, 2013

## Other reading

- STATS 385: theories of deep learning, a Stanford graduate course, reviewing papers that seek to build a theoretical understanding of neural networks
- CS231n: convolutional neural networks for visual recognition, a Stanford graduate course: an excellent introduction to the theory and practice of training deep neural nets, with online videos available
- Deep Learning, Goodfellow, Bengio, Courville. A book describing the state of the art, by leading researchers [pdf]
- Computer Age Statistical Inference, Efron and Hastie. A book about the impact of computers on the evolution of statistical thinking [pdf]
- A neural algorithm of artistic style, Gatys, Ecker, Bethge, 2015. This was used to “translate” a photo of the mathematical bridge into “art”, with software from https://turbo.deepart.io/
- Deepmath: a website collecting papers on the mathematical theory of deep neural networks
- Off the convex path: a blog on (nonconvex) optimization and applications to machine learning (with a focus on theory)
- A nice reading list [pdf]
- Nathan Benaich’s blog about the state of AI from the point of view of the tech industry