The error that doubles on a schedule

Two pendulums are joined end to end and released from rest with both arms out at 120°. A second pair is released from 120.00000001° — a difference of a hundred-millionth of a degree, which is smaller than the thermal expansion of the apparatus over a temperature change too small to measure. Both are then integrated forward from the same free-body accounting every mechanics problem uses, with no noise, no friction and no approximation beyond the arithmetic.

The interesting thing is not that a small difference grows. It is that it grows at a rate, and that the rate is a property of the system rather than of the difference.

The measurement #

That number, the largest Lyapunov exponent, is what turns a qualitative statement into a quantitative one. A system is sensitive to initial conditions if a small difference grows; it is chaotic if the growth is exponential, because only then is there a time constant and only then does the question “how long is the forecast good for” have a numerical answer.

And the answer has an unwelcome shape. If the separation grows as $e^{\lambda t}$, the time for an initial uncertainty δ to reach a stated size Δ is

$$t = \frac{1}{\lambda}\ln\frac{\Delta}{\delta},$$

a logarithm of the precision. Improving a measurement by a factor of a thousand buys $\ln(1000)/\lambda$ — for this pendulum, 3.4 seconds.

That is the whole of what chaos costs, and it is worth stating in its most deflating form: the reward for a heroic improvement in measurement is a linear improvement in forecast, and the improvement needed is exponential.

A note on what is being measured a distance between. The separation plotted is a distance in phase space — the four numbers that specify the state, two angles and two angular rates — rather than a distance between the bobs. That matters, because two configurations can look identical and be moving quite differently, and the whole point is to track the state rather than the picture. Using the bob positions instead gives a curve that oscillates wildly as the pendulums swing past each other, with the same exponential envelope hidden inside it.

What the number is a limit of #

The exponent has a definition that the figure above approximates rather than evaluates, and the gap between the two is worth knowing about, because it is where most bad measurements of this quantity come from.

Properly, $\lambda$ is a limit: take two states an infinitesimal distance $\delta_0$ apart, follow both, and take

$$\lambda = \lim_{t\to\infty}\ \lim_{\delta_0\to 0}\ \frac{1}{t}\ln\frac{\delta(t)}{\delta_0}.$$

Both limits are doing work. The inner one is needed because the growth law is a statement about the linearised equations — the separation has to stay small enough that the difference between the two trajectories obeys a linear equation with the first one’s Jacobian in it — and the outer one because the local stretching rate varies from place to place and only its long-time average is a property of the system.

Neither limit can be taken on a computer, and the compromise shows in the figure directly. Start too close together and the separation is buried in rounding error; start far enough apart to see it and the linear regime ends after a few decades, which is the flattening at the top of the curve where the two pendulums are already as different as two pendulums can be. Nine decades of straight line between those two failures is what the measurement gets, and it is enough.

The standard repair is to stop letting the separation grow. Follow the pair for a short interval, record the factor by which they separated, then pull the second trajectory back along the same direction to its original tiny distance and continue. Summing the logarithms of the recorded factors and dividing by the total time gives the same exponent with no ceiling at all, because the pair is never allowed to leave the linear regime. The figure here does not need it, and any attempt to measure this quantity over minutes rather than seconds would.

The check that makes it about pendulums #

Every figure above was produced by a numerical integration, and an unstable integrator produces two diverging trajectories as readily as a chaotic system does. The difference is what the integrator does to a conserved quantity.

The second check is physical rather than numerical, and it is the more important one. A machine that reports a positive exponent for every input it is given has measured its own arithmetic.

The small-amplitude system does something entirely different, and the contrast is the check that makes this about pendulums rather than about arithmetic. Two coupled pendulums at small amplitude have exactly two motions that keep their shape, and every other motion is a superposition of those two — so nearby starts stay nearby forever, and the separation grows linearly at worst. The same apparatus, driven harder, stops having normal modes at all.

One more thing the exponent is not: a statement about how complicated the motion looks. A trajectory can be visually chaotic and have a small exponent, or look tidy for a long stretch and have a large one. What the number measures is the growth of a difference between two nearby states, which is invisible in any single trace — which is why every figure in this essay draws two trajectories or a separation, and never one path on its own.

Where the sensitivity comes from #

The mechanism is visible in a much simpler system, and it is not the nonlinearity itself.

A single pendulum is not chaotic, and its phase portrait shows why. Inside the separatrix the motion swings; outside it, the pendulum goes over the top and rotates; and two starts on the same side stay on the same side. Sensitivity requires trajectories that can be separated by a boundary and then brought back together, and one degree of freedom does not have enough room.

What a single pendulum does not have is anywhere for the stretching to go. Its phase space is two-dimensional and its energy is conserved, so trajectories are confined to closed curves and two nearby states drift apart at worst linearly. The double pendulum’s phase space is four-dimensional; energy fixes one combination and leaves three, which is enough room for a trajectory to be stretched in one direction, squeezed in another, and folded back on itself.

The dimension count is worth doing explicitly, because it is what separates this system from every pendulum on the ladder below. A single pendulum’s state is an angle and a rate: two numbers, one conserved energy, so the trajectory is confined to a curve and there is nowhere for it to go except round. The double pendulum’s state is four numbers and the energy fixes one combination, leaving a three-dimensional surface — the smallest space in which a trajectory can be stretched apart, folded back and prevented from crossing itself.

Stretching and folding is the whole recipe. Stretch alone makes the separation grow and takes the trajectory to infinity. Folding brings it back into a bounded region, so the stretching has to happen again, on a state that has already been stretched. Repeated indefinitely, that is what produces both the exponential separation and the fact that the motion stays inside a finite volume for ever.

The mechanism in its smallest form is a hill between two regions. A trajectory arriving with just enough energy to cross takes an arbitrarily long time near the top, so an arbitrarily small difference in energy makes an arbitrarily large difference in when — and eventually in which side it ends up on. Stretching near an unstable point and folding back is the whole recipe, and a double pendulum has one such point on every swing.

And it needs at least three dimensions. A continuous system confined to a plane cannot be chaotic: trajectories cannot cross, so a bounded two-dimensional flow must either approach a fixed point or wind onto a closed curve, and neither stretches anything for ever. Three is the minimum, and a conservative system needs four because energy conservation removes one. That is exactly the count for the double pendulum — two angles, two rates, one conserved energy — and it is the reason the single pendulum, with two dimensions and one conservation law, cannot do this whatever amplitude it is given.

The exponents come in pairs, and they sum to zero #

There is not one exponent but four, one for each direction in the state space, and their arrangement says something the single largest one hides.

A mechanical system with no friction preserves volume in phase space: a blob of initial conditions may be stretched and folded arbitrarily, and its volume at every later instant is exactly what it started with. A volume grows at the sum of the exponents, so for this pendulum that sum is zero. One direction is stretched at 2.048 per second; another must be squeezed at 2.048 per second; and the remaining two — along the trajectory itself, and across the energy surface — are zero, because a displacement along a trajectory neither grows nor shrinks. The spectrum is $(+\lambda, 0, 0, -\lambda)$, and the pairing is exact rather than approximate.

That is what makes the stretching-and-folding picture quantitative. The blob is not getting bigger; it is getting thinner, at precisely the rate it is getting longer, and folding is what lets an ever-longer, ever-thinner filament stay inside a bounded region. The apparent creation of uncertainty is a redistribution of it: what is lost about one combination of the four numbers is gained about another, and nothing about the total is lost at all. A system that genuinely destroyed information would have a negative sum, which is what friction does and what makes a damped system settle rather than wander.

It also explains why the measurement above is so much easier than it might be. Any two trajectories started nearly together will, with overwhelming probability, have some component of their separation along the stretching direction, and that component grows while every other one shrinks. So the largest exponent is what any careless measurement returns, and getting the rest of the spectrum is the hard problem.

The history that makes the point #

The result was found three times before it was believed. Poincaré met it in the 1880s while competing for a prize on the stability of the solar system, discovered that his prize-winning memoir contained an error, and in correcting it found the tangle of trajectories that a separatrix crossing produces — a figure he declined to draw, calling it too complicated to attempt. Hadamard found exponential divergence on a surface of negative curvature in 1898. Neither result travelled.

It was found again in 1961 by someone re-running a weather model from a printout, entering 0.506 where the machine had been holding 0.506127, and getting a completely different forecast. That is exactly the experiment the figures above perform, with a hundred-millionth of a degree standing in for the truncated decimals — and the reason it landed the third time is that it was found in a system somebody wanted an answer from, rather than in a proof about the stability of the planets.

Why three degrees is not a smaller version of a hundred and twenty #

The third refutation above — that a nonlinearity is not enough — is easy to state and easy to mis-explain. The nonlinearity does not switch on somewhere between 3° and 120°; the sines in the equations of motion are nonlinear at every amplitude, and the small-amplitude case has them too. What changes is how much of the space the regular motion occupies.

At low energy the phase space of this system is almost entirely filled with surfaces that trajectories cannot leave — nested tori, each carrying a motion with two frequencies that repeats for ever. A trajectory confined to such a surface has neighbours that stay neighbours, and the exponent is zero. Chaos exists at that energy too, but only in exceedingly thin layers where those surfaces have broken, near the separatrices, and a trajectory started at random is overwhelmingly unlikely to be in one. The measured exponent of $2\times10^{-4}$ per second is that: not an absence of chaotic regions, but a start that missed them.

Raise the energy and the surfaces break up from the outside in. The thin chaotic layers widen, neighbouring ones merge, and past some energy a single connected chaotic region occupies most of what is available. A trajectory started at 120° is in that region, and it explores it. So the honest statement of what the two curves in the small-amplitude figure show is not “chaotic versus not chaotic” but “a system whose chaotic set has almost no volume, and the same system whose chaotic set has almost all of it”.

Which is why the exponent is quoted at an energy and why it rises with one, and why the phrase mixed phase space is the accurate description of everything in between: at intermediate energies the same apparatus, released twice, will give a large exponent from one start and zero from another, with no error in either measurement.

What the exponent is and is not #

It has units of inverse time, and that is the point. A “sensitivity” with no units is a comparison; a rate is a measurement. Quoting 2.048 per second means the same thing as quoting a doubling time of 0.34 seconds or a lifetime of 0.49 seconds, and every statement about how long a forecast lasts is that number divided into a logarithm.

It is an average, not a rate that holds at every instant. The separation does not grow smoothly by 2.048 per second; it grows in bursts as the trajectory passes through the stretching regions and shrinks in the quiet parts. The exponent is the long-time average of the local rate, and a fit over a short stretch gives a different number.

It says nothing about the long-term behaviour. The pendulum does not fly apart or wind down; it goes on doing what it was doing, at the same energy, for ever, since nothing in the equations dissipates. Chaos is about which state it is in, not about what kind of state.

And none of it involves any loss of reversibility. The equations are unchanged by sending $t \to -t$, so every trajectory run backwards is also a solution, and the exponential separation happens in both directions of time equally. What that means in practice is easy to test and disconcerting to watch: integrate this pendulum forward for twenty seconds and then integrate the result backwards for twenty, and it does not return to where it started. The arithmetic error of the forward run is now an initial condition of the backward one, and it grows at 2.048 per second just as any other difference would. Nothing irreversible has happened, and the calculation is irreversible anyway.

And it is not a licence to treat the system statistically. The trajectory is not random — unlike a walk whose steps really are independent, every point of it is determined by the release — and any average taken over it is an average over a single, particular curve.

A random walk is the contrast worth drawing. Its mean square displacement grows in proportion to time, because each step is independent of the last and errors add incoherently. A chaotic trajectory’s separation grows exponentially, because each step amplifies the accumulated error rather than adding to it. The two look similar over a few steps and diverge completely over many, and confusing them is what makes chaos sound like noise.

Where the practical consequence sits #

There is one more consequence worth naming, and it is the reason forecasts are issued as probabilities. Since the trajectory cannot be predicted past the horizon, what is computed instead is an ensemble: the same model run from many slightly different starts, and the spread of the results reported as the forecast. Before the horizon the ensemble members agree and the forecast is a number; after it they scatter and the forecast is a distribution. The transition between the two is the same straight line on a logarithmic axis that this essay’s second figure draws.

The number for the atmosphere — a fluid whose own stability is a difference of two gradients — is a doubling time of a day or two, and the deflating arithmetic above applies to it unchanged: reducing the initial uncertainty by a factor of a thousand extends the forecast by about ten doubling times, which is a fortnight. The limit is not computing power and it is not the model. It is that the observations would have to improve exponentially to make the forecast improve linearly.

And it changes what a simulation is for. A calculation of this pendulum’s position at t = 60 seconds is worthless as a prediction: the arithmetic error in the sixteenth decimal place has been amplified through fifty decades by then, so the computed trajectory is not the trajectory of the pendulum that was specified. What such a calculation is good for is statistics — how often the lower arm goes over the top, what fraction of the time it spends in each region, what the distribution of speeds is — because those are properties of the region the trajectory explores rather than of the trajectory, and they are reproduced correctly by a computed path that has long since parted company with the true one.

That is a strange and useful fact, and it has a name in the literature: shadowing. A computed trajectory of a chaotic system is usually close to some true trajectory of the same system, just not the one it was asked for. Which is enough for an average and never enough for a forecast.

Where this ladder goes next #

This rung has measured one number on one system. The next asks what that number belongs to: the exponent is not a property of the trajectory but of the region the trajectory explores, and the set of states a bounded chaotic system settles onto has a structure — neither a point nor a curve nor a filled volume, but something with a dimension between them. Measuring that dimension, and showing that it is less than the space it sits in, is what turns “the motion is complicated” into a statement about how much of the space it actually uses.