Backpropagation Through Time (BPTT), vanishing gradients, forget gates, and long-range sequence memory.
For a small example, a sequence 2, -1, 3 arrives one item at a time. Update a scalar hidden state after each item and compare outputs when the sequence order changes.
Recent measurements influence the next reading. Window sequences, prevent future leakage, compare an LSTM with naive and linear baselines, and inspect long-horizon decay.
Initializing Interactive Playground...
Predict how one control changes the RNNs & LSTMs result.
The starter deliberately follows a tempting but flawed assumption: Leaking future observations through preprocessing
Check: Your saved hypothesis predicts a direction, the experiment tests it, and the result changes or supports your explanation.
Explanations after each answer · 80% to pass · RNNs & LSTMs
Chapter 1 of 6 · Core
Sequential data (time-series, speech, natural language text) possesses temporal dependencies where the current token depends on past sequence history . Standard feedforward neural networks cannot process variable-length inputs without fixed context windows. Recurrent Neural Networks (RNNs) introduce a recurrent hidden state vector passed forward step-by-step across time steps .
Key equations
MIT 6.S191: Recurrent Neural Networks
MIT Introduction to Deep Learning
Follow recurrent state through sequences and connect its limitations to gated architectures such as LSTMs.
Primary sources and free technical reading for RNNs & LSTMs.