This is an explainer of the 2019 Sparse Transformer by Child, Gray, Radford, and Sutskever, the first serious escape from attention’s quadratic cost. Its central idea is that full attention connects all n squared pairs of positions in one hop, but you can connect all pairs in two hops with far fewer edges by letting every token attend to a local window plus a strided set of relay positions, chosen so any token can reach any other through a shared relay, and picking the stride to be the square root of n means each token needs only order square root of n connections for a total cost of order n times square root of n instead of n squared, with global reach intact. It presents the strided pattern with a chessboard rook metaphor: fold the length n sequence into a grid of side square root of n, and give each position two factorized attention heads, a local head that attends to the previous stride positions which is the rest of its row, and a strided head that attends to every stride-th position which is its column; formally the local set is the previous ell positions and the strided set is positions j less than or equal to t where t minus j is divisible by ell. Like a rook reaching any square in at most two moves, column then row, any position can attend to any earlier position within two layers even though each attention touches only order square root of n positions, and this suits images and audio with 2-D or periodic structure. It then presents the fixed pattern for text where periodic structure is arbitrary: attend to a local block of ell consecutive tokens plus a small fixed set of landmark summary columns, the last position of every previous block, which act as relays that each block funnels information into and every later token reads, giving the same two-hop principle token to landmark to token but with fixed content-agnostic summary slots. A Sparse Transformer interleaves the two factorized heads across layers, and the general recipe is p attention steps so any position reaches any other within p hops, with p equals two the workhorse. The square root of n is derived by splitting each token’s cost into a local part of size ell and a strided or landmark part of size n over ell, so cost per token is ell plus n over ell, minimized when the two terms are equal at ell equals square root of n, giving per-token cost two square root of n and total order n times square root of n, which is n to the three halves; this is not linear but the jump from n squared to n to the 1.5 is a thousand-fold reduction at n equals one million. Results, reported qualitatively: with architectural and initialization changes to train hundreds of layers deep, gradient recomputation for memory, and custom sparse attention kernels, it modeled sequences tens of thousands of timesteps long, feasible to a million or more, roughly a 30 times increase in practically modelable length, and set new state-of-the-art density modeling on Enwik8 text, CIFAR-10 and ImageNet-64 images, while generating coherent audio, showing one factorized-sparsity idea works for any long serialized sequence. Caveats: the patterns are hand-designed not learned so you must match pattern and stride to your data’s structure, which later learned-sparsity methods like Routing Transformer and Reformer addressed; two-hop reach is weaker than one-hop because distant tokens communicate only indirectly through relays; it is still super-linear at n to the 1.5 and loses to truly linear methods at extreme length; and FlashAttention later made exact full attention fit and run fast, reducing the pressure to approximate at moderate lengths. It sits at the head of the efficient-attention lineage in the attend-to-less family, with descendants Longformer and BigBird adding global tokens, sliding-window attention keeping just the local part, and routing and LSH methods learning the sparsity, all tracing to the question of whether every token must attend to every other, answered not directly but within a couple of hops, a design space that reaches to Kimi K3’s linear-attention core. Two interactive widgets let the reader click a query cell on a 4-by-4 grid and toggle Full, Strided, and Fixed patterns to see the causal attention set and edge counts, and drag the sequence length to compare full n squared versus sparse n square-root-n connections with the optimal stride and savings factor.