Entropy is a count, and the arrow of time is arithmetic

A film of two billiard balls colliding, played backwards, shows a perfectly legal collision. A film of a glass shattering, played backwards, shows something nobody has ever seen.

Both are made of the same mechanics. Newton’s laws are indifferent to the direction of time — reverse every velocity and every trajectory retraces itself exactly — and so are Maxwell’s equations, and so is almost everything else in fundamental physics. Yet the everyday world has an unmistakable direction, and it is not subtle.

The resolution is not a hidden law. It is counting.

Every arrangement is equally likely #

Toss ten coins. The sequence HHHHHHHHHH has exactly the same probability as HTHTTHTHHT — one in 1,024, both of them. Nothing prefers a mixed sequence.

But “five heads” is not one sequence. It is 252 of them, and “ten heads” is one. That is the entire asymmetry, and there is nothing else to it.

The distinction being drawn is between a microstate — the full specification, which coin is which — and a macrostate — the summary, how many heads. Physics measures macrostates. It measures pressure, temperature, volume, and the number of microstates consistent with any of those measurements is astronomically large.

With four coins the bias toward the middle is a mild preference. All four heads happens once in sixteen, which is unremarkable. The second law does not exist at this scale; nothing about four coins is irreversible.

Now run the same argument for a gas. A litre of air holds about $3 \times 10^{22}$ molecules, and the equivalent of “all heads” is every molecule happening to be in the left half of the room. The number of arrangements favouring the mixed state exceeds the number favouring the sorted one by a factor of $2^{3\times10^{22}}$, a number with more than $10^{22}$ digits.

The air could collect in one half of the room. Nothing forbids it. Waiting for it would take, by an enormous margin, longer than the universe has existed — and the exponent is so large that no refinement of the estimate changes the conclusion by anything meaningful.

The formula on the gravestone #

Boltzmann made the identification exact:

$$S = k \ln W,$$

where $W$ is the number of microstates in the macrostate and $k$ is a constant with units of energy per temperature. Entropy is the logarithm of a count.

The logarithm is not decoration. It is there so that entropy is additive: two independent systems have $W_1 W_2$ arrangements between them, and $\ln(W_1W_2) = \ln W_1 + \ln W_2$. Doubling the gas doubles the entropy, which is what a thermodynamic quantity has to do. Without the logarithm, entropy would multiply, and no amount of nineteenth-century thermodynamics would have recognised it.

That figure shows why the logarithm changes the character of the quantity. In raw counts the distribution is a spike; in logarithms it is a gentle dome. Entropy differences between macrostates are modest numbers even when the underlying count ratios are astronomical, and that is the scale on which thermodynamics was built without anybody knowing the counting was there.

Note also what the constant does. Boltzmann’s constant $k$ is a conversion factor between a pure number — the logarithm of a count, which has no units — and the units of energy over temperature that thermodynamics had already committed to. It is present only because temperature was defined before anyone knew what it measured.

The second law is a probability statement #

Written this way, the second law stops sounding like a law and starts sounding like an observation about large numbers: a system left alone moves to macrostates with more microstates, because there are more of them to move to.

There is no force involved. Nothing pushes a gas toward uniformity. Each molecule follows Newton’s laws with no view about the whole, and the aggregate goes where the counting is, for the same reason a shuffled deck comes out disordered: not because shuffling prefers disorder, but because almost every arrangement is one people would call disordered.

This makes the second law different in kind from the others. Conservation of energy is exact and has no exceptions. The second law is overwhelmingly probable and has exceptions — they are simply too rare to observe at any scale bigger than a few thousand particles. In a very small system, over a very short time, entropy can be observed decreasing, and it has been: fluctuation theorems make quantitative predictions about how often, and experiments on micron-scale beads in optical traps have confirmed them.

That is a genuinely strange status for a law that governs the direction of time.

Where the arrow actually comes from #

Here is the part the counting argument does not settle, and it is worth being honest about it rather than letting the coins imply more than they show.

If the microscopic laws are time-symmetric and high-entropy states are overwhelmingly more numerous, then the same argument that says entropy will be higher tomorrow says it was higher yesterday. Run the reasoning backwards and it makes an absurd prediction: that the past was more disordered than the present.

The counting is not wrong. It is incomplete. It explains why entropy increases toward the future given that it is low now, and it cannot explain why it is low now. That has to be an additional fact about initial conditions — the universe began in an extraordinarily low-entropy state, and everything that has happened since, including every structure that formed and every life that was lived, is the ongoing spending of that initial improbability.

So the arrow of time as experienced is not derived from mechanics plus counting. It is derived from mechanics plus counting plus a boundary condition at the beginning, and the boundary condition is where the mystery has been relocated rather than solved.

The same shape, in a real system #

Coins are a model of the counting and not a model of any physical system. The corresponding curve for something real looks different in detail and identical in structure.

The Maxwell–Boltzmann distribution rises because there are more velocity directions available at higher speed, and falls because high energy is exponentially unlikely. Peak in the middle, tails at both ends, and a sharpness that increases with the number of particles: it is the coin figure with the macrostate labelled by speed instead of by heads.

The correspondence goes all the way down. A gas at equilibrium is not a gas whose molecules have stopped changing. It is one whose distribution has stopped changing while every member of it is reassigned billions of times a second — exactly as a bag of $10^{23}$ coins shaken continuously would sit at half heads forever without any individual coin staying put.

Heat, and why it flows one way #

The abstract counting becomes concrete the moment energy is what is being distributed.

Put a hot body against a cold one. The molecules of the hot body have more energy on average, and the distribution of their speeds is broader. At the contact surface, collisions transfer energy in both directions — fast molecules give energy to slow ones and occasionally the reverse. Nothing in any single collision has a preferred direction.

The net flow is from hot to cold because the number of ways to share a fixed total energy among all the molecules is larger when the sharing is even. Moving a quantum of energy from hot to cold increases the count more than it decreases it; moving it the other way does the opposite. Heat flows down the temperature gradient because that is where the arrangements are.

Temperature itself gets a much better definition out of this. It is not “how much energy is in there” — it is how much the count changes when energy is added:

$$\frac{1}{T} = \frac{\partial S}{\partial E}.$$

A cold body is one whose arrangement count rises sharply when given a little energy; a hot body is one that gains relatively little by it. Energy flows to where it does the most good, counted in arrangements. This definition also permits negative temperatures, in systems with a bounded energy spectrum where adding energy reduces the count — and such systems are hotter than any positive temperature, which the everyday definition of temperature cannot accommodate at all.

It is worth noticing how little this argument used. It never mentioned what the bodies were made of, what forces act in them, or how the energy is stored — only that energy can be shared and that arrangements can be counted. That indifference to mechanism is what the counting argument shares with Gauss’s law and with momentum conservation: each replaces a question about the interior with a question about a total, and each is powerful in direct proportion to how little it needs to know.

The cost, made into an engineering number #

The counting argument has a consequence that can be put on an invoice.

Work is ordered motion: every part moving the same way. Heat is disordered motion: parts moving every way at once. Converting work entirely into heat is easy — friction does it constantly and nobody has to arrange anything. Converting heat entirely into work would mean taking a disordered state and producing an ordered one, which is moving to a macrostate with fewer arrangements, and that is precisely what the counting forbids.

So every engine has a ceiling set by two temperatures and nothing else, and the ceiling is the second law wearing a units label. The same logic makes a perfectly inelastic collision irreversible: the ordered motion of two bodies has become disorder in $10^{23}$ molecules, and there is no arrangement-preserving route back.

Where the model stops #

The coin picture is a good model of entropy and a bad model of a gas, in four specific respects.

Coins are distinguishable; molecules are not. Two identical helium atoms swapped do not make a new microstate, and counting as though they did gives an entropy that is wrong by an amount depending on the system size. That discrepancy — the Gibbs paradox — was noticed long before quantum mechanics and resolved only by it.

Coins have two states; molecules have a continuum. Counting states of a continuous system requires a smallest meaningful cell in phase space to divide by, and classical physics offers none. Planck’s constant supplies it, so even the classical entropy of a gas contains $\hbar$ in it.

Coins do not interact. Real molecules do, and once they do, some arrangements are more likely than others and the flat “every microstate equally likely” assumption needs replacing by the Boltzmann factor. That is the difference between the coin figure and the speed distribution.

Coins have no energy. The figure counts arrangements at fixed total; thermodynamics usually needs the count at fixed energy, which is a constrained count and a harder one.

And a limitation of the picture rather than the model: no drawing can convey the size of the numbers involved. A figure of twenty coins and a real system of $10^{23}$ differ by so many orders of magnitude that the intuition built on the first is quantitatively useless for the second, even when it is qualitatively right. The peak in a real system is not merely sharp — it is sharp enough that the fluctuations are unobservable, and that unobservability is the whole difference between statistics and thermodynamics.

The ladder from here #

Later rungs: entropy as $dQ/T$, the thermodynamic definition, and its equivalence to the counting one. The Gibbs entropy for unequal probabilities. Free energy, which is what a system minimises when it can exchange energy with its surroundings. The Maxwell’s demon problem and its resolution through the entropy of information. Landauer’s principle, and why erasing a bit must dissipate heat. Fluctuation theorems and the measured violations. The entropy of a black hole, which is proportional to area rather than volume and which nobody expected. And the ceiling it places on every engine, which is where the counting argument becomes an engineering constraint.

Boltzmann’s $S = k\ln W$ is carved on his headstone in Vienna. He wrote it down in a form nobody accepted for twenty years, in an argument about whether atoms were real.