In past posts, I’ve described how Recurrent Neural Networks (RNNs) can be used to learn patterns in sequences of inputs, and how the idea of unrolling can be used to train them. It turns out that there are some significant limitations to the types of patterns that a typical RNN can learn, due to the way their weight matrices are used. As a result, there has been a lot of interest in a variant of RNNs called Long ShortTerm Memory networks (LSTMs). As I’ll describe below, LSTMs have more control than typical RNNs over what they remember, which allows them to learn much more complex patterns.

Recent Posts
Recent Comments
Jesse Johnson on Kmodes Wiisco on Kmodes Peter Ranacher on Kmodes Recurrent Neural Net… on Neural Networks 2: Evalua… Neural Networks 1: T… on Linear Separation and Support… Archives
Categories
Meta