Some Christmas challenges:
- Train a recurrent network on tweets, at the word level, then apply it to Charles Dickens. Splice emoji into Dickens.
- Invent a hybrid character-and-word RNN. There can be character-based units, plus short-circuit links between words (inspired by Residual Network). This might help address the challenge of what to do if the vocabulary is very different between Dickens and twitter.
- Could we recompose Dickens into tweets? This sounds like a translation task, but we don’t have a bilingual dataset. Maybe there are word or sentence embeddings, that can simply be transposed?
- Lyrics for Christmas songs and carols
- 50k tweets from Christmas 2016, full of emoji
- A Christmas Carol, Charles Dickens
Figuring out how to use TensorFlow:
- A starting point: the rather terse TensorFlow tutorial on RNNs
- Stanford course CS20SI: TensorFlow for deep learning research
In a reading group early next term, let’s have a show-and-tell, and a discussion of TensorFlow and PyTorch. Also, there’s a Slack channel for any discussion over the holidays.