Entropy is a count, and the arrow of time is arithmetic

A film of two billiard balls colliding, played backwards, shows a perfectly legal collision. A film of a glass shattering, played backwards, shows something nobody has ever seen.

Both are made of the same mechanics. Newton’s laws are indifferent to the direction of time — reverse every velocity and every trajectory retraces itself exactly — and so are Maxwell’s equations, and so is almost everything else in fundamental physics. Yet the everyday world has an unmistakable direction, and it is not subtle.

The resolution is not a hidden law. It is counting.

Every arrangement is equally likely #

Toss ten coins. The sequence HHHHHHHHHH has exactly the same probability as HTHTTHTHHT — one in 1,024, both of them. Nothing prefers a mixed sequence.

But “five heads” is not one sequence. It is 252 of them, and “ten heads” is one. That is the entire asymmetry, and there is nothing else to it.

The distinction being drawn is between a microstate — the full specification, which coin is which — and a macrostate — the summary, how many heads. Physics measures macrostates. It measures pressure, temperature, volume, and the number of microstates consistent with any of those measurements is astronomically large.

With four coins the bias toward the middle is a mild preference. All four heads happens once in sixteen, which is unremarkable. The second law does not exist at this scale; nothing about four coins is irreversible.

Now run the same argument for a gas. A litre of air holds about $3 \times 10^{22}$ molecules, and the equivalent of “all heads” is every molecule happening to be in the left half of the room. The number of arrangements favouring the mixed state exceeds the number favouring the sorted one by a factor of $2^{3\times10^{22}}$, a number with more than $10^{22}$ digits.

The air could collect in one half of the room. Nothing forbids it. Waiting for it would take, by an enormous margin, longer than the universe has existed — and the exponent is so large that no refinement of the estimate changes the conclusion by anything meaningful.

The formula on the gravestone #

Boltzmann made the identification exact:

$$S = k \ln W,$$

where $W$ is the number of microstates in the macrostate and $k$ is a constant with units of energy per temperature. Entropy is the logarithm of a count.

The logarithm is not decoration. It is there so that entropy is additive: two independent systems have $W_1 W_2$ arrangements between them, and $\ln(W_1W_2) = \ln W_1 + \ln W_2$. Doubling the gas doubles the entropy, which is what a thermodynamic quantity has to do. Without the logarithm, entropy would multiply, and no amount of nineteenth-century thermodynamics would have recognised it.

That figure shows why the logarithm changes the character of the quantity. In raw counts the distribution is a spike; in logarithms it is a gentle dome. Entropy differences between macrostates are modest numbers even when the underlying count ratios are astronomical, and that is the scale on which thermodynamics was built without anybody knowing the counting was there.

Note also what the constant does. Boltzmann’s constant $k$ is a conversion factor between a pure number — the logarithm of a count, which has no units — and the units of energy over temperature that thermodynamics had already committed to. It is present only because temperature was defined before anyone knew what it measured.

The second law is a probability statement #

Written this way, the second law stops sounding like a law and starts sounding like an observation about large numbers: a system left alone moves to macrostates with more microstates, because there are more of them to move to.

There is no force involved. Nothing pushes a gas toward uniformity. Each molecule follows Newton’s laws with no view about the whole, and the aggregate goes where the counting is, for the same reason a shuffled deck comes out disordered: not because shuffling prefers disorder, but because almost every arrangement is one people would call disordered.

This makes the second law different in kind from the others. Conservation of energy is exact and has no exceptions. The second law is overwhelmingly probable and has exceptions — they are simply too rare to observe at any scale bigger than a few thousand particles. In a very small system, over a very short time, entropy can be observed decreasing, and it has been: fluctuation theorems make quantitative predictions about how often, and experiments on micron-scale beads in optical traps have confirmed them.

That is a genuinely strange status for a law that governs the direction of time.

Where the arrow actually comes from #

Here is the part the counting argument does not settle, and it is worth being honest about it rather than letting the coins imply more than they show.

If the microscopic laws are time-symmetric and high-entropy states are overwhelmingly more numerous, then the same argument that says entropy will be higher tomorrow says it was higher yesterday. Run the reasoning backwards and it makes an absurd prediction: that the past was more disordered than the present.

The counting is not wrong. It is incomplete. It explains why entropy increases toward the future given that it is low now, and it cannot explain why it is low now. That has to be an additional fact about initial conditions — the universe began in an extraordinarily low-entropy state, and everything that has happened since, including every structure that formed and every life that was lived, is the ongoing spending of that initial improbability.

So the arrow of time as experienced is not derived from mechanics plus counting. It is derived from mechanics plus counting plus a boundary condition at the beginning, and the boundary condition is where the mystery has been relocated rather than solved.

The same shape, in a real system #

Coins are a model of the counting and not a model of any physical system. The corresponding curve for something real looks different in detail and identical in structure.

The same competition appears in a real gas without any adjustment. The distribution of molecular speeds has a peak in the middle of its range for exactly the reason the coin figure does: the number of ways of having a given speed grows with the speed, because there are more directions available on a larger sphere in velocity space, while the probability of any one of them falls. A growing count multiplied by a falling probability is a peak, and where the peak sits is decided by which factor is winning.

The Maxwell–Boltzmann distribution rises because there are more velocity directions available at higher speed, and falls because high energy is exponentially unlikely. Peak in the middle, tails at both ends, and a sharpness that increases with the number of particles: it is the coin figure with the macrostate labelled by speed instead of by heads.

The correspondence goes all the way down. A gas at equilibrium is not a gas whose molecules have stopped changing. It is one whose distribution has stopped changing while every member of it is reassigned billions of times a second — exactly as a bag of $10^{23}$ coins shaken continuously would sit at half heads forever without any individual coin staying put.

Heat, and why it flows one way #

The abstract counting becomes concrete the moment energy is what is being distributed.

Put a hot body against a cold one. The molecules of the hot body have more energy on average, and the distribution of their speeds is broader. At the contact surface, collisions transfer energy in both directions — fast molecules give energy to slow ones and occasionally the reverse. Nothing in any single collision has a preferred direction.

The net flow is from hot to cold because the number of ways to share a fixed total energy among all the molecules is larger when the sharing is even. Moving a quantum of energy from hot to cold increases the count more than it decreases it; moving it the other way does the opposite. Heat flows down the temperature gradient because that is where the arrangements are.

Temperature itself gets a much better definition out of this. It is not “how much energy is in there” — it is how much the count changes when energy is added:

$$\frac{1}{T} = \frac{\partial S}{\partial E}.$$

A cold body is one whose arrangement count rises sharply when given a little energy; a hot body is one that gains relatively little by it. Energy flows to where it does the most good, counted in arrangements. This definition also permits negative temperatures, in systems with a bounded energy spectrum where adding energy reduces the count — and such systems are hotter than any positive temperature, which the everyday definition of temperature cannot accommodate at all.

It is worth noticing how little this argument used. It never mentioned what the bodies were made of, what forces act in them, or how the energy is stored — only that energy can be shared and that arrangements can be counted. That indifference to mechanism is what the counting argument shares with Gauss’s law and with momentum conservation: each replaces a question about the interior with a question about a total, and each is powerful in direct proportion to how little it needs to know.

The cost, made into an engineering number #

The counting argument has a consequence that can be put on an invoice.

Work is ordered motion: every part moving the same way. Heat is disordered motion: parts moving every way at once. Converting work entirely into heat is easy — friction does it constantly and nobody has to arrange anything. Converting heat entirely into work would mean taking a disordered state and producing an ordered one, which is moving to a macrostate with fewer arrangements, and that is precisely what the counting forbids.

So every engine has a ceiling set by two temperatures and nothing else, and the ceiling is the second law wearing a units label. The same logic makes a perfectly inelastic collision irreversible: the ordered motion of two bodies has become disorder in $10^{23}$ molecules, and there is no arrangement-preserving route back.

A force made of nothing but counting #

The page has said several times that no force is involved — that nothing pushes a gas toward uniformity. That is right, and it has a consequence that sounds like its opposite: some perfectly ordinary forces, the kind measured with a spring balance, are made of nothing but the counting.

A rubber band is the standard example. It is a tangle of long molecules, each free to arrange itself in an enormous number of ways while joining the same two points. Stretch it, and the chains are forced closer to being straight — and there are very few straight arrangements against very many crumpled ones. The count falls, and the resistance felt in the fingers is the system’s overwhelming preference for the states that are numerous.

Nothing is being stretched in the sense a steel spring is stretched. In a metal the restoring force comes from bonds pulled off their equilibrium spacing, and the stored energy is potential energy. In an ideal rubber the internal energy barely changes with length at all; the force is $-T\,\partial S/\partial L$, which is temperature times a slope of a count.

That difference is testable in about ten seconds, and the test is the reason to trust the account. Stretch a rubber band quickly and hold it against the lip: it is warm, because the entropy fell and the energy had to go somewhere. Let it snap back and it cools. And a rubber band hung with a weight contracts when it is heated, where every other material lengthens — because the force is proportional to $T$, so warming pulls harder. Gough noticed the warming in 1805 and Joule measured it in 1859, and both observations are inexplicable without a count.

The two objections Boltzmann never fully answered #

The counting argument met two objections in his own lifetime, both from serious people, and neither is a misunderstanding.

Loschmidt, 1876. If the microscopic laws are reversible, then for every evolution in which entropy rises there is a mirror evolution — every velocity reversed — in which it falls, and the two are equally lawful. So no argument from mechanics alone can produce a preferred direction. It cannot, and Boltzmann eventually agreed: what the counting gives is that the entropy-decreasing initial conditions are astronomically rare, not that they are forbidden.

Zermelo, 1896, using a theorem Poincaré had proved six years earlier: a bounded mechanical system eventually returns arbitrarily close to any state it has been in. So the gas will collect in one half of the room, given long enough, and the second law cannot be exact. Also true. Boltzmann’s answer was to estimate how long, and the recurrence time for a cubic centimetre of air has an exponent with nineteen digits in it — a number for which “longer than the age of the universe” is not an approximation but a grotesque understatement.

Both objections are correct and neither is fatal, and the reason is the status the second law was given above. It is not exact and does not need to be. What the two objections did establish is that the irreversibility cannot come from the mechanics, which is what forces the boundary condition at the beginning into the argument — and that part of the debate was never settled by anyone.

The count that turned out to be information #

Nothing in the counting argument mentions molecules, energy or heat. It mentions arrangements, and how many of them correspond to a description. That is a statement about knowledge, and the fact was noticed late and turned out to matter.

Shannon, working on the capacity of telephone lines in 1948, needed a measure of how much a message tells its recipient, and arrived at the same expression: a sum over probabilities of $p\log p$, differing from Boltzmann’s only in the constant in front. The story that he chose the name entropy on von Neumann’s advice, on the grounds that nobody knows what entropy is so he would always have the advantage in an argument, may be apocryphal. The identity of the formulae is not.

For a long time this was regarded as an analogy — the same mathematics turning up in two subjects, as the same differential equation turns up in half the systems on this site. It is not an analogy. Physical entropy and missing information are the same quantity, and the demonstration is a thermodynamic price tag on a computational operation.

The argument runs through Maxwell’s demon. A demon that could see individual molecules and open a shutter only for the fast ones would separate a gas into hot and cold halves with no work, reducing the count of arrangements and violating the second law. Szilard showed in 1929 that the demon’s measurement seemed to be the loophole, and Landauer in 1961 located the charge exactly: measurement can in principle be performed reversibly and free, but the demon’s memory eventually fills, and erasing a bit is irreversible. Erasure takes two possible states to one, which halves the count of arrangements, and the lost count must appear as heat.

The bill is $kT\ln 2$ per bit — about $2.9\times10^{-21}$ joules at room temperature. It is a very small number and it is not zero, and it is not a matter of engineering: no imaginable technology erases a bit for less. Bennett completed the argument in 1982 by showing that the demon, once required to erase, pays exactly what it gained, to the last joule. The second law survives because information storage is physical.

The prediction was confirmed directly in 2012, with a single colloidal particle in an optical double well acting as one bit, and the heat released on erasure measured against the bound. It sat where Landauer said it would.

How far this is from mattering is worth a number. A processor dissipating a hundred watts while performing something like $10^{11}$ bit operations a second spends around $10^{-9}$ joules per operation, which is a few hundred billion times the Landauer limit. The floor is real, it is thermodynamic rather than technological, and computing is nowhere near it — the entire industry runs at the far end of a margin whose existence was established by counting coins.

Where the model stops #

The coin picture is a good model of entropy and a bad model of a gas, in four specific respects.

Coins are distinguishable; molecules are not. Two identical helium atoms swapped do not make a new microstate, and counting as though they did gives an entropy that is wrong by an amount depending on the system size. That discrepancy — the Gibbs paradox — was noticed long before quantum mechanics and resolved only by it.

Coins have two states; molecules have a continuum. Counting states of a continuous system requires a smallest meaningful cell in phase space to divide by, and classical physics offers none. Planck’s constant supplies it, so even the classical entropy of a gas contains $\hbar$ in it.

Coins do not interact. Real molecules do, and once they do, some arrangements are more likely than others and the flat “every microstate equally likely” assumption needs replacing by the Boltzmann factor. That is the difference between the coin figure and the speed distribution.

Coins have no energy. The figure counts arrangements at fixed total; thermodynamics usually needs the count at fixed energy, which is a constrained count and a harder one.

And a limitation of the picture rather than the model: no drawing can convey the size of the numbers involved. A figure of twenty coins and a real system of $10^{23}$ differ by so many orders of magnitude that the intuition built on the first is quantitatively useless for the second, even when it is qualitatively right. The peak in a real system is not merely sharp — it is sharp enough that the fluctuations are unobservable, and that unobservability is the whole difference between statistics and thermodynamics.

The ladder from here #

Later rungs: entropy as $dQ/T$, the thermodynamic definition, and its equivalence to the counting one. The Gibbs entropy for unequal probabilities. Free energy, which is what a system minimises when it can exchange energy with its surroundings. The Maxwell’s demon problem worked through in detail rather than summarised. Fluctuation theorems and the measured violations. Diffusion, which is the second law happening slowly enough to be photographed. The entropy of a black hole, which is proportional to area rather than volume and which nobody expected. And the ceiling it places on every engine, which is where the counting argument becomes an engineering constraint.

Boltzmann’s $S = k\ln W$ is carved on his headstone in Vienna. He wrote it down in a form nobody accepted for twenty years, in an argument about whether atoms were real.