In past posts, I’ve described how Recurrent Neural Networks (RNNs) can be used to learn patterns in sequences of inputs, and how the idea of unrolling can be used to train them. It turns out that there are some significant limitations to the types of patterns that a typical RNN can learn, due to the way their weight matrices are used. As a result, there has been a lot of interest in a variant of RNNs called Long ShortTerm Memory networks (LSTMs). As I’ll describe below, LSTMs have more control than typical RNNs over what they remember, which allows them to learn much more complex patterns.

Recent Posts
Recent Comments
Dahabina Omar on LSTMs Pvalues  A bunch o… on The geometry of linear re… Genetic algorithms a… on Optimization VachSoch on Statistics vs. Heuristics Rolling and Unrollin… on Recurrent Neural Networks Archives
Categories
Meta