The ring that forgets and then remembers

Entropy is a count laid out Boltzmann’s answer to why heat flows one way: entropy is the logarithm of the number of microscopic arrangements compatible with what is seen, and disordered states have overwhelmingly more arrangements than ordered ones, so a system wandering among its arrangements is overwhelmingly likely to be found in a disordered one. The same essay recorded the two objections that were raised against Boltzmann almost at once. Josef Loschmidt pointed out in 1876 that the laws of mechanics are reversible, so for every motion in which entropy rises there is a reversed motion in which it falls. Ernst Zermelo pointed out in 1896 that a bounded mechanical system returns arbitrarily close to its starting state, given enough time, so entropy cannot rise for ever.

Both objections are correct, and Boltzmann’s answer to both is correct too, and the argument about how they fit together has never entirely stopped. In 1956 Mark Kac invented a model simple enough to be solved exactly, in which every part of the argument can be seen at once: the reversible rules, the irreversible-looking behaviour, the recurrence, and the one assumption in Boltzmann’s equation that turns the first into the second.

A ring with marked gaps #

Place $N$ sites round a circle, each holding a ball that is white or black. Between neighbouring sites are $N$ gaps, and some fixed set of them, chosen at random with probability $\mu$ each, is marked. At every tick every ball moves one site clockwise, and a ball that crosses a marked gap changes colour. That is the whole of the dynamics.

The rule has every property of microscopic mechanics that matters to the argument. It is deterministic: the state at one tick fixes the state at the next. It is reversible: moving every ball one site anticlockwise, changing colour at the same marks, undoes a tick exactly. And it conserves the number of possible states, mapping each arrangement to exactly one other, as Liouville’s theorem says Hamiltonian mechanics does for the volume of phase space. Nothing in the rule prefers mixing to unmixing.

Started all white, the ring nonetheless mixes. A ball turns black when it crosses one mark, white again at the next, and after enough ticks each ball has crossed a different, effectively random number of marks, so its colour is effectively random. After forty ticks the sixty balls in the figure are about half white and half black with no visible pattern, the state any observer would call equilibrium.

Boltzmann’s equation for the ring #

Boltzmann would describe the ring by the numbers of white and black balls, $W$ and $B$, and ask how they change. In one tick, a white ball becomes black if the gap ahead of it is marked, and a black ball white. If whether a ball is white is independent of whether the gap ahead of it is marked — his Stosszahlansatz, the assumption of molecular chaos — then on average a fraction $\mu$ of each colour changes, and

$$W_{t+1} - B_{t+1} = (1 - 2\mu)(W_t - B_t), \qquad \Delta_t = (1-2\mu)^t,$$

where $\Delta$ is the excess of white over black as a fraction of all the balls. The excess decays exponentially to zero. From $\Delta$ an entropy can be computed — the entropy of the colours alone, $-p\ln p - (1-p)\ln(1-p)$ per ball with $p$ the fraction white — and it rises monotonically to its maximum, $\ln 2$. That is Boltzmann’s H-theorem, for the ring.

For the first few dozen ticks the exact ring follows Boltzmann’s curve so closely that the two cannot be told apart. Then the exact excess wanders about zero by a few hundredths, while Boltzmann’s goes to zero and stays. And at tick 400 the exact ring does something Boltzmann’s equation never can: it is all white again. Every ball has gone once round the ring, crossed every one of the 52 marks, and changed colour an even number of times. Had the number of marks been odd, it would have taken two turns, 800 ticks. The recurrence is not approximate: the state at that tick is exactly the starting state.

The assumption that carries the arrow #

Kac proved something sharper than the figure shows. Average over all the ways of choosing which gaps are marked, each with probability $\mu$, and the average excess is exactly $(1-2\mu)^t$ — Boltzmann’s equation holds exactly — for every $t$ less than $N$. The reason is that a ball has crossed $t$ different gaps by tick $t$, and as long as it has not yet come back to a gap it crossed before, those gaps are independent random choices. The Stosszahlansatz is literally true on average, for that long.

What fails at a recurrence is exactly that independence. After one full turn, each ball meets the same gaps again, and its colour is no longer uncorrelated with whether the gap ahead is marked — it is completely determined by it. The marks are fixed, the balls carry the memory of every mark they have crossed in their colours, and the memory comes back in full. Boltzmann’s equation is right as long as the past it forgets does not return, and wrong the moment it does.

That is the general structure of the problem in a gas. The molecules in a gas collide, and Boltzmann assumed that two molecules about to collide are uncorrelated, which is very nearly true because they have usually never met before. After the collision they are correlated — their velocities are related by the collision — and the correlation is carried away and diluted among more and more molecules in later collisions, until it is too spread out to matter for any measurement of a few molecules at a time. It is not destroyed. A gas, like the ring, carries every correlation it has ever formed, in a form no coarse measurement can see.

Boltzmann’s own replies to his critics fit the ring closely. To Loschmidt he is said to have answered, in effect, “then go ahead and reverse them”: the reversed motion exists, but preparing it requires knowing and controlling every molecule, which nobody can. To Zermelo he answered with arithmetic: the recurrence exists, but its time is so long that nothing physical will ever wait for it. In the ring both answers can be checked, because both the reversal and the recurrence are within reach of a calculation, and both answers are right — the reversal needs the exact arrangement, and the recurrence needs a full turn of the ring.

An entropy that knows only the count #

The entropy that rises in the ring is the entropy of a description: the count of white balls, ignoring which ones. It is a coarse-grained entropy, in the sense mixing what is already mixed made precise — what counts as mixed depends on what is being distinguished. The fine-grained description, the full arrangement of colours, has no entropy to gain: the dynamics maps each arrangement to exactly one other, so if the starting arrangement is known exactly, the arrangement at every later tick is known exactly. Nothing is lost. The coarse description loses track because it throws away which balls are which, and the ring’s dynamics systematically moves information out of the count and into the arrangement, where the count cannot see it.

That makes the second law a statement about descriptions as well as about things, and it is why the entropy of the ring can fall to zero in a single tick at the recurrence without violating anything. The arrangement just before the recurrence is extraordinarily special — it is one tick from all white — but the count of colours cannot show that, any more than a shuffled deck’s count of red and black cards shows whether it is one cut away from being sorted. The bit that has to be paid for found the thermodynamic cost of forgetting a bit; the ring forgets nothing, and it is the coarse description that pays.

Large rings are reliable, single rings are not #

Boltzmann’s law is exact on average over the ways of marking the ring. A single ring, with one particular set of marks, follows it only approximately.

A small ring follows the decay roughly and then wanders widely about zero. A large one follows it closely and then hardly wanders at all. The scatter about Boltzmann’s law shrinks roughly as one over the square root of the number of balls, the scaling the second law with a probability attached found in the probability of entropy running backwards for a small system. For a ring with as many sites as there are molecules in a cubic centimetre of gas, about $10^{19}$, the deviations would be a part in a few billion: invisible.

So Boltzmann’s equation is a law about large numbers, as he said it was. In the limit of a huge ring, at any fixed number of ticks, every ring follows it exactly. The recurrence time also grows with the ring, in proportion to $N$. The order of the two limits matters: let the ring grow first and then wait, and the approach to equilibrium is irreversible and exact; wait first at a fixed size and then look, and it recurs. Real gases are so large, and their recurrence times so long — Boltzmann estimated them at a number of years with more digits than there are atoms in the universe — that the first order is the physically relevant one.

Running it backwards #

Loschmidt’s objection can be acted out exactly. Run the ring forwards for sixty ticks, until it is thoroughly mixed, and then reverse every ball’s direction. The excess of white climbs back — slowly at first, while the reversed arrangement is still far from its origin, then steeply — and after sixty reversed ticks the ring is all white. Boltzmann’s equation, given the mixed count at tick sixty, can only predict more mixing. Its arrow of time is not in the dynamics; it is in the equation’s choice to forget the arrangement.

That reversal is not purely an artifice of a toy model. Erwin Hahn showed in 1950 that the precessing nuclear spins in a liquid, which dephase until their combined magnetisation vanishes, can be made to rephase by a single radio-frequency pulse that reverses their relative phases, so that the magnetisation reappears as an echo. The spin echo is Loschmidt’s reversal performed on a real, large system, and it works because the dephasing, like the ring’s mixing, moved the information out of the measured quantity without destroying it. The equation that only runs forwards found the same arrow in diffusion; there, too, the irreversibility belongs to a description that has averaged over what it cannot follow.

The urn that plays dice instead #

Fifty years before Kac, Paul and Tatyana Ehrenfest had built a model with the opposite character. Put $N$ numbered balls in two urns, and at each step pick a number at random and move that ball to the other urn. The number in the first urn drifts towards half, as the ring’s colours do, but here the dynamics is random rather than deterministic, and the question of recurrence takes a different form. Every arrangement will come back, with certainty, but how long it takes depends on how special it is: the mean time to return to a state is one over the fraction of time the system spends in it. The state with all balls in one urn has probability $2^{-N}$ at equilibrium, so its mean recurrence time is $2^N$ steps — for a hundred balls, about $10^{30}$. The half-and-half state, by contrast, comes back every few dozen steps.

The two models bracket the real case. The ring recurs on schedule, after exactly $N$ or $2N$ ticks whatever the starting state, because its dynamics is a rigid rotation. The urn recurs on average after a time set by the state’s probability, because its dynamics explores all states at random. A real gas is deterministic like the ring but chaotic, so it explores its states like the urn, and its recurrence times are of the urn’s kind: astronomically long for any state that looks ordered, because ordered states are an astronomically small fraction of all of them. The walk that comes home found that a random walker in one or two dimensions always returns to its start and in three dimensions may not; returning to a whole arrangement of $10^{19}$ molecules is a walk in a space of $10^{19}$ dimensions, and in such a space “always returns” can mean “returns after a time that no clock will ever measure”.

Recurrence is still seen in real systems small or orderly enough to show it. The first computer experiment on a nonlinear chain, run by Fermi, Pasta, Ulam and Tsingou in 1955, found its energy returning almost exactly to its starting mode instead of spreading out — the near-recurrence the energy that refuses to be shared followed. Quantum wave packets in a box revive at fixed times, the return the return a classical cloud never makes described. Both are the ring’s lesson in other clothes: when a system’s motion is close enough to regular, the memory of its start comes back on a timetable.

Where the ring stops being a gas #

The ring is a caricature, and its differences from a gas matter. Its “collisions” are with fixed scatterers rather than between balls, so the balls never correlate with each other, only with the marks; in a gas, correlations between molecules build up and spread, and there is no exact model of how. Its dynamics is not chaotic: a small error in the initial arrangement does not grow, which is why the reversal works perfectly. In a chaotic system, as the error that doubles on a schedule found, a reversal must be exact to a precision that grows exponentially with the time elapsed, and any real attempt at Loschmidt’s reversal in a gas fails for that reason as much as for any other. The ring’s recurrence time is exactly $N$ or $2N$; a gas’s Poincaré recurrence is vastly longer and only approximate. And the ring has only two states per site, so its entropy saturates at $\ln 2$. The domain of the figures is rings of a hundred to twenty thousand sites with ten per cent of the gaps marked, over times up to a little beyond one recurrence.

Still open: why the start was all white #

The ring explains how a reversible rule produces irreversible-looking behaviour, and it makes plain the one thing it does not explain: the starting state. The ring mixes because it was started all white, an arrangement of extraordinarily low coarse-grained entropy. Started from a typical mixed arrangement, it would stay mixed — and run backwards from a mixed arrangement, it would also stay mixed. The arrow in the figures points from the special state to the typical ones, and it points that way because the special state was put at the beginning. In the universe the same question is why the early universe was in a state of extraordinarily low entropy, the condition sometimes called the past hypothesis. Whether that is a fact about initial conditions to be accepted, a consequence of some deeper law of cosmology, or a selection effect is argued by physicists and philosophers, and the ring, which makes everything else transparent, makes it clear that this part of the arrow is not in the dynamics at all.

The model itself fits in one statement. Kac’s ring is deterministic, reversible and conserves its states, yet started all white its excess of colour decays as Boltzmann’s $(1-2\mu)^t$ and its coarse-grained entropy rises to $\ln 2$ — exactly, on average over the marks, for every tick before the first recurrence, and within fluctuations shrinking as $1/\sqrt N$ for one ring — because a ball meets only fresh marks until it comes round again; at tick $N$ or $2N$ it is all white once more, and reversed at any tick it retraces its steps exactly. The arrow of time in Boltzmann’s equation is real for large systems and finite times, and it lives in the decision to count rather than to track.