How RNNs Actually Memorize Sequences — and What It Looks Like
Gradient magnitudes don't usually make headlines, but for researchers trying to understand why one recurrent neural network outperforms another, they might be the most honest signal available. A new visualization approach uses input-output connectivity — essentially, how strongly a model's prediction depends on each prior character in a sequence — to expose something that accuracy scores alone can't: whether a model is actually learning context, or just getting lucky on short patterns.
Why accuracy metrics miss the point
Recurrent Neural Networks are designed to carry information across time. Feed them a sequence, and in theory, each new prediction can draw on everything that came before. In practice, the notorious vanishing gradient problem means that long-range dependencies tend to fade during training, leaving models that are technically functional but contextually shallow.
The standard benchmarks used to evaluate recurrent units — Penn Treebank, text8, Chinese Poetry Generation — measure cross-entropy loss or character-level accuracy. These numbers are useful, but they're easy to game. A model can score well by nailing predictions that only require the last few characters of input, while completely failing on cases where understanding depends on something said ten or twenty tokens earlier. The aggregate score hides that failure entirely.
This is the core problem the visualization method addresses. Rather than asking "how accurate is this model overall?", it asks "which past inputs is this model actually using when it makes a prediction?" Those are very different questions, and the second one turns out to be far more revealing.
The autocomplete task as a diagnostic tool
The researchers chose autocomplete as their test problem deliberately. Unlike raw character prediction, autocomplete produces output that humans can immediately evaluate — you either recognize the suggested word as contextually appropriate, or you don't. The task also naturally requires both short-term and long-term understanding: short-term to complete a word once several letters are visible, long-term to anticipate what word is coming based on earlier sentence context.
Three model architectures were compared: GRU, LSTM, and Nested LSTM. All were trained on the text8 dataset using two-layer networks with 600 units each. The connectivity metric used is the gradient of the output with respect to each input character — a measure of how much each past character influenced the current prediction.
The results are striking. When predicting the word "learning" with only its first two characters visible, the GRU model draws strongly on the word "advanced" appearing earlier in the sentence and correctly surfaces "learning" as a top suggestion. The LSTM does the same, though with slightly lower confidence. The Nested LSTM, by contrast, shows almost no connectivity to prior context and falls back on common words starting with "l" — statistically reasonable, but contextually blind.
A second test case, predicting "grammar" on its second appearance in a passage, shows the GRU correctly identifying the word from a single character, while both LSTM and Nested LSTM require four characters before converging on the right answer. The GRU is visibly using the earlier occurrence of the word as a contextual anchor.
One particularly interesting edge case involves the LSTM predicting "schools." The connectivity visualization shows the LSTM drawing on words from nearly the entire preceding sentence — which looks like sophisticated long-range memory. But its actual suggestions are wrong, bearing little relationship to the context it appears to be using. The authors interpret this as a model capable of long-term memorization that hasn't learned to translate that memory into meaningful contextual understanding. It's a subtle but important distinction.
What this means for how we evaluate recurrent architectures
The broader implication here is methodological. Comparing recurrent units by their benchmark scores has always been somewhat unsatisfying — results vary by dataset, hyperparameters, and implementation details, and it's rarely clear what's actually driving the difference. The connectivity visualization offers a complementary lens: instead of asking which model wins, it asks how each model reasons.
That said, the authors are careful about the limits of their approach. Connectivity analysis is inherently example-specific. Observing that Nested LSTM shows weak long-range connectivity on one passage doesn't mean it will behave the same way on a different dataset or with different hyperparameters. The visualization is a diagnostic, not a verdict.
The method also suggests a more targeted quantitative metric: measure prediction accuracy as a function of how many characters from the target word are already visible. This captures the short-term versus long-term tradeoff more directly than aggregate loss, though it sacrifices the simplicity of a single summary number.
The gap between memorization and contextual understanding — the ability to store information versus the ability to use it meaningfully — turns out to be where the real differences between recurrent architectures live, and gradient-based connectivity analysis is one of the cleaner ways to see it.
Accuracy metrics alone don't tell the full story when comparing recurrent neural network architectures — and that's precisely the central argument this research makes visible through a connectivity visualization approach that exposes how GRU, LSTM, and Nested LSTM models actually differ in practice.
When identical accuracy scores hide fundamentally different behavior
The GRU and LSTM models evaluated here achieve nearly the same overall accuracy and cross entropy loss on an autocomplete task. On the surface, that looks like a draw. But the connectivity visualization tells a different story: the GRU model draws far more heavily on long-range context — repeating words and semantic meaning from earlier in the input sequence — while the LSTM model leans toward short-term context. Two models, similar numbers, meaningfully different internal behavior.
This gap is larger than the raw metrics would suggest. Cross entropy and accuracy, taken alone, flatten out distinctions that matter when you're deciding which architecture to build on or how to improve one. The qualitative analysis here fills that gap by showing not just whether a prediction is correct, but how the model arrived at it — and whether a wrong prediction was at least contextually reasonable, like a synonym that fits the surrounding text.
What the Nested LSTM architecture adds — and whether it delivers
The Nested LSTM unit takes the opposite approach from GRU. Where GRU simplifies the standard LSTM by removing internal memory, Nested LSTM adds more of it. The cell value update — normally a direct arithmetic operation combining input and forget gates — is replaced by a full inner LSTM unit. That inner unit can itself be replaced recursively, stacking additional memory states as deep as needed.
The motivation is straightforward: more internal memory should, in theory, support longer-term memorization. The gate activation functions follow the same conventions as vanilla LSTM, with sigmoid functions on the input, forget, and output gates, and the identity function applied to the candidate state to avoid redundant non-linearities. The full equations governing the inner LSTM unit are defined in the appendix of the original work.
In practice, the Nested LSTM model tested here uses one layer of 600 units with two internal memory states, compared to two layers of 600 units each for GRU and LSTM. The results, however, don't show the Nested LSTM pulling ahead on long-term contextual understanding — a finding the author notes is likely sensitive to hyperparameter choices and the specific application rather than a definitive verdict on the architecture.
Why visualization methods matter for model development
The deeper contribution here isn't the ranking of these three architectures — it's the argument for how models should be evaluated. Quantitative metrics like cross entropy loss are necessary but not sufficient. They can't show you that one model is using semantic relationships across a long input window while another is essentially reacting to the last few characters. Connectivity visualization makes that difference legible, intuitively and without requiring deep familiarity with the underlying math.
That has practical value at two stages: when selecting a model for deployment, and when designing better architectures. Knowing that GRU leverages long-range word repetition and meaning more aggressively than LSTM — under these conditions, with these hyperparameters — is the kind of insight that shapes future research directions. The original Nested LSTM paper by Joel Ruben Antony Moniz and David Krueger is credited as a direct inspiration for this line of inquiry, reinforcing that the recurrent unit, despite its familiarity, remains an open and active area of investigation.
The dataset used for evaluation is drawn from the text8 corpus, split 90/10/5/5 across training, validation, and test sets, with sequences capped at 200 characters and a vocabulary of 16,384 output words. Training ran for 14,278 mini-batches using Adam optimization with default parameters. The full implementation is available at https://github.com/distillpub/post--memorization-in-rnns, and peer reviews from Abhinav Sharma, Dylan Cashman, and Ruth Fong — linked below — contributed substantially to the final quality of the work.
Review 1 - Abhinav Sharma
Review 2 - Dylan Cashman
Review 3 - Ruth Fong
The broader takeaway is that model evaluation needs both dimensions — the numbers that measure performance and the visual tools that reveal mechanism. Neither replaces the other, and research that treats them as complementary rather than competing is likely to produce more durable insights about why certain architectures work, not just whether they do.
Diagrams and text are licensed under Creative Commons Attribution CC-BY 4.0 with the source available on GitHub, unless noted otherwise. Figures reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from …"
For attribution in academic contexts, please cite this work as
