The equation that only runs forwards, and the walk underneath it

Nearly every equation in physics works equally well run backwards. Reverse the velocities of every particle in a swinging pendulum, a colliding pair of balls, an orbiting planet or a propagating wave, and what results is another perfectly good solution of the same equations. Film any of them backwards and nothing in the film violates a law.

Film a drop of ink spreading in water and run it backwards, and the result is absurd. This is one of a small number of places where the mathematics of physics has a direction built into it, and it is worth being precise about where the direction came from — because underneath the equation is a mechanism that has no direction at all.

The walk #

Start with the mechanism, which is as simple as a mechanism gets.

A particle takes a step left or right, chosen by a fair coin, and repeats. After $N$ steps of length $\ell$ its displacement is the sum of $N$ independent $\pm\ell$ terms. The average displacement is zero, since left and right are equally likely and there is no preference anywhere in the process.

The average of the square is not zero. Squaring the sum gives $N$ terms of $\ell^2$ plus a great many cross terms, and every cross term averages to zero because the steps are independent. So

$$\langle x^2 \rangle = N\ell^2,$$

and the typical distance travelled is $\ell\sqrt{N}$. With steps taken at a constant rate, $N$ is proportional to time, so

$$\text{width} \propto \sqrt{t}.$$

That square root is the whole of diffusion, and it is worth dwelling on because it is so much slower than the intuition it displaces. Travelling at a steady speed covers distance in proportion to time; diffusion covers it in proportion to the square root, so covering ten times the distance takes a hundred times as long.

The consequence is a sharp separation of scales. A molecule in water diffuses a micron in about a millisecond, which is why a cell can move things around inside itself by doing nothing at all — the molecules doing it are the same ones whose speeds a temperature describes. The same molecule needs about three hours to cross a centimetre and about three years to cross a metre. Diffusion is the fastest transport mechanism there is at small scales and hopeless at large ones, and the crossover is entirely a consequence of the square root.

That is why a cell is the size it is, why lungs are subdivided into hundreds of millions of alveoli rather than being one bag, and why stirring a cup of tea is worth doing.

From the walk to the equation #

The step from a walk to a continuum equation is the interesting one, because something is lost in it and the loss is where the arrow of time comes from.

Consider the concentration $c(x,t)$ — the number of walkers per unit length. In one step, each walker at $x$ splits half to the left and half to the right, so the new concentration at $x$ is the average of the old concentrations at its two neighbours. Writing that out and expanding gives, in the limit of small steps taken quickly,

$$\frac{\partial c}{\partial t} = D\,\frac{\partial^2 c}{\partial x^2},$$

with $D = \ell^2/2\tau$ for steps of length $\ell$ taken every $\tau$.

Read that equation as a statement about shape rather than as calculus, and it says something simple: concentration increases wherever the profile is curved upward, and decreases wherever it is curved downward. A peak is curved downward, so it falls. A trough is curved upward, so it fills. A straight-line profile is not curved at all, so it does not change — which is why a steady gradient can persist indefinitely, and why heat flows at a constant rate through a wall with fixed temperatures on its two faces.

The solution for an initial spike is a Gaussian of variance $2Dt$ — spreading, flattening, and conserving its area, which is the statement that nothing is lost. Every curve in the hero figure is that solution evaluated at a different time, and the histogram behind them is the walk run directly, with 900 walkers and no reference to the equation. The measured width of the walk and the width of the curve agree to about a per cent, which is the check the figure exists to make.

Where the irreversibility entered #

Here is the point of the essay. The random walk is perfectly reversible. The diffusion equation is not.

Any particular sequence of coin tosses can be run backwards, and the reversed sequence is exactly as probable as the forward one. Nothing in the mechanism prefers a direction. A film of a single walker run backwards is a film of a perfectly ordinary walker.

The equation, on the other hand, is unambiguous. Running it backwards means integrating with $t$ decreasing, and doing so makes small ripples grow explosively rather than smoothing away — the backwards heat equation is famously ill-posed, and any tiny error in the data blows up without limit. The forward direction is stable and the backward direction is not, and the two are not symmetric in any sense.

So a reversible mechanism has produced an irreversible equation, and the place where the asymmetry entered is identifiable: it entered at the step where individual walkers were replaced by a concentration.

That step discards which walker is which. Once the description is “one third of the walkers are in this bin”, the information needed to run the process backwards — which particular walker went which way at which step — is gone. It has not been destroyed physically; it is still there in the walkers. It has been dropped from the description, and the equation describing what is left has no way to recover it.

This is exactly the argument that gives the second law, stated in the smallest possible example. There are vastly more arrangements of walkers that look spread out than look concentrated, so the spread-out description is where the process goes; and the irreversibility is a property of the coarse description rather than of the underlying dynamics. Diffusion is the second law happening slowly enough to be photographed, and slowly enough that the count of arrangements can be watched increasing rather than merely argued about.

Loschmidt’s objection, answered by the figure #

The tension was noticed immediately and put sharply by Loschmidt to Boltzmann in 1876: if the microscopic laws are reversible, no argument from them can produce an irreversible conclusion. Reverse every velocity and the gas must un-mix.

It must, and it would. The answer is not that the reversal is impossible but that it is fantastically improbable, and the figure gives a way to feel the size of the improbability.

The histogram behind the curves is 900 walkers taking 240 steps each. To make them re-gather, every one of those 216,000 coin tosses would have to come out the way that undoes its predecessor. The probability is $2^{-216{,}000}$. That number, for a picture with fewer than a thousand particles in it and a fraction of a second of history, is already beyond any meaningful comparison — and a drop of ink contains $10^{20}$ molecules.

So the second law is not a prohibition. It is a statement that the exceptions have probabilities of that order, and the difference between “cannot” and “will not in the lifetime of anything” is not a difference any experiment will ever report. It is worth noticing that this is a weaker claim than the one usually made and a considerably more interesting one.

The gradient that is being spent #

There is a second reading of the equation, and it connects diffusion to the rest of thermodynamics rather than merely to probability.

What diffusion consumes is not energy. Energy is conserved throughout: nothing is created and nothing is destroyed, and the area under every curve in these figures is the same. What is consumed is the unevenness, and unevenness is the resource every engine runs on.

That is why the two subjects meet. A heat engine extracts work from a temperature difference and reduces the difference in doing so; diffusion reduces the same difference and extracts nothing. Both processes end in the same place, and the difference between them is whether anything useful was taken on the way. An engine is a way of insisting on payment for something that will happen regardless.

The rate matters too, and it makes the trade concrete. A Carnot engine reaches the ceiling only by running infinitely slowly, and the reason is exactly this equation: transferring heat at a finite rate requires a finite temperature difference across the transfer surface, and that difference is spent unevenness that produced no work. Every real power plant is a compromise between the size of its heat exchangers and the losses in them, and the compromise is a diffusion calculation.

What the continuum description costs #

Replacing walkers by a concentration buys an equation that can be solved and charges for it in three currencies.

The graininess is gone. A concentration is a smooth function, and real diffusing matter comes in molecules. Where the numbers are small the smooth description is simply wrong, and the fluctuations are the phenomenon rather than an error — Brownian motion is what a diffusion equation looks like when there are not enough particles for the average to be smooth, and it is how Avogadro’s number was first measured.

Information is discarded, permanently. As above: this is not an accounting convenience but the source of the equation’s most important property. Anyone who wants the arrow of time in their equations has to throw something away to get it, and the diffusion equation is the cleanest place to see the transaction.

And the tails are wrong. The Gaussian solution is non-zero at every distance, at every time — which says that a molecule released here has a non-zero probability of being a light-year away a second later. That is false, and it is a real defect rather than a quibble: the continuum limit was taken by letting the step size go to zero and the step rate go to infinity, which permits arbitrarily large speeds. The random walk has no such problem, since a walker in $N$ steps cannot be further than $N\ell$ away. The equation is an excellent description of the middle of the distribution and a fictional one about its extremes.

The same equation, four subjects wide #

The derivation used a coin, a limit and no physics, which is why the equation turns up wherever a quantity is conserved and moves by many small independent nudges.

Heat conduction is the same equation with temperature in place of concentration, and the random walk underneath is the transfer of vibrational energy between neighbouring atoms. The number $D$ becomes the thermal diffusivity, and it is why a metal spoon heats along its length in seconds and a wooden one does not.

Momentum diffuses too. Viscosity is the sideways spreading of momentum through a fluid by molecular motion, obeying the same equation with the same square-root law, and the thickness of the boundary layer on a moving surface grows as the square root of the distance along it for exactly that reason.

Charge carriers in a semiconductor diffuse, and the diffusion length — how far a carrier travels before recombining — is one of the two or three numbers that decide how a solar cell or a transistor must be dimensioned.

And prices diffuse, at least in the model that dominates finance: the Black–Scholes equation is the diffusion equation after a change of variables, its square-root law is the reason option values scale with the square root of time to expiry, and the arrow of time in it is the same arrow as the one in the ink drop.

The list is not a set of analogies. It is one theorem — many small independent contributions add to a Gaussian whose width grows as the square root of their number — with four sets of labels attached. The theorem is the same one that gives the shape of the speed distribution in a gas and the reason an average over enough events is smooth.

Where the model stops #

No flow, no forces. The equation describes pure spreading. Add a wind, an electric field on charged particles, or gravity on a suspension, and a drift term appears alongside the diffusion term — the pair together give the equation that describes most transport in nature, and the ratio of the two terms decides whether stirring or waiting is the faster route.

The medium is uniform and unchanging. $D$ was a constant. In real systems it varies with temperature, with concentration, and with position, and the equation with a varying $D$ has different solutions and no simple Gaussian.

The walkers do not interact. Each was assumed to step independently of the others. At high concentration they collide, crowd, and in some systems attract — at which point diffusion can run backwards in the everyday sense, with a mixture spontaneously separating into two phases. That is spinodal decomposition, and it happens because the relevant quantity being flattened is not the concentration but the chemical potential, which can be a decreasing function of concentration.

And nothing here is thermodynamic. The derivation used a coin and a limit. It never mentioned temperature, energy or heat, which is why the same equation describes heat conduction, the spread of a dye, the smoothing of a pressure disturbance in a porous rock, the price of an option, and the blurring of an image. The arrow of time in it comes from the coarse-graining, not from thermodynamics — thermodynamics is another consequence of the same coarse-graining rather than the cause of this one.

The ladder from here #

Later rungs: Fick’s two laws stated properly, and the difference between them. The diffusion equation solved with boundary conditions — a wall, a source, a sink. The Einstein relation, connecting the diffusion constant to the mobility and the temperature, which is the first fluctuation–dissipation relation and the reason Brownian motion measures Avogadro’s number. Diffusion in three dimensions, and why the walk returns to its origin with certainty in one and two dimensions and not in three. Anomalous diffusion, where the width goes as a power of time other than a half. The drift–diffusion equation and the Péclet number. Heat conduction as the same equation with temperature in place of concentration. And the connection to the wave equation, which differs by a single derivative and is completely reversible as a result — one order of derivative separating a world with an arrow of time from one without.