This is a history-of-ideas explainer of language modeling before the transformer, framed as one recurring problem: carrying information across a long sequence, scored by two quantities, the path length information must travel between two related tokens and how much of that travel is inherently sequential and cannot be parallelized. It proceeds through five links. First, n-gram models, tracing to Shannon 1948, apply the Markov assumption to approximate the next-token probability by conditioning only on the last k minus one words and estimating by counting; they fail because of a fixed horizon (blind beyond the window) and combinatorial sparsity, since the number of k-grams is vocabulary size to the power k, and because words are atomic symbols they generalize nothing. Second, recurrent neural networks (Elman 1990) keep a fixed-size hidden state updated at each token as h at t equals tanh of W_h times the previous hidden state plus W_x times the input plus a bias, so in principle the state summarizes unbounded history, but the vanishing and exploding gradient problem (Hochreiter 1991, Bengio 1994) makes the learning signal decay exponentially with distance because the gradient is a product of k Jacobians, and the n steps are inherently sequential giving O(n) path length and O(n) sequential operations. Third, Long Short-Term Memory networks (Hochreiter and Schmidhuber 1997) add a cell state, described by Christopher Olah as a conveyor belt, updated additively and guarded by forget, input, and output gates, so the new cell equals the forget gate times the previous cell plus the input gate times the candidate, letting gradients flow with gain near one and extending usable dependency length to dozens or hundreds of tokens, but it is still sequential and still eventually forgets. Fourth, sequence-to-sequence models (Sutskever, Vinyals, Le 2014; Cho 2014) use an encoder LSTM to compress the whole source into one fixed-length context vector and a decoder LSTM to generate the target, reaching 34.8 BLEU on WMT 2014 English to French versus 33.3 for phrase-based statistical machine translation, but the single fixed vector is a bottleneck that makes quality collapse on long sentences, and reversing the source sentence raised BLEU from 25.9 to 30.6 by shortening paths. Fifth, Bahdanau, Cho, and Bengio (2014) introduced attention, computing a fresh context vector for each output step as a weighted sum of all encoder states with softmax alignment weights, giving the decoder a direct length-one path to any source token and learning alignment end to end, which removes the fixed-vector bottleneck and reaches across word-order differences such as the English European Economic Area mapping onto the reversed French zone economique europeenne. Finally, the bottlenecks that remained were that the recurrent backbone is inherently sequential, which precludes parallelization within training examples, and that encoder self-dependencies still travel O(n); Vaswani et al. (2017) removed recurrence entirely in favor of self-attention, where Table 1 shows self-attention has O(n squared times d) complexity per layer, O(1) sequential operations, and O(1) maximum path length, versus recurrent layers with O(n times d squared), O(n), and O(n), trading a sequential bottleneck for a quadratic compute one; the big Transformer reached 28.4 BLEU on English to German and 41.8 on English to French after 3.5 days on eight GPUs. The post ends by connecting to the modern campaign to keep O(1) paths while shrinking the O(n squared) cost (KV caching, multi-query and grouped-query attention, FlashAttention, sliding-window and linear attention, and Mamba). It includes an interactive path-length comparator that shows sequential steps and path length growing with sequence length for RNNs while self-attention stays at one, and an interactive Bahdanau alignment demo.