The bit that has to be paid for

Maxwell imagined a being small enough to see individual molecules, operating a frictionless door between two halves of a gas-filled box. It lets fast molecules through one way and slow ones the other, and after a while one side is hot and the other cold — from which an engine can be run, with no work having been done on the door.

The construction is not a fallacy in the obvious places. The door can be made frictionless in principle, the molecules can be seen in principle, and the sorting genuinely produces a temperature difference. It took eighty years to find where the argument goes wrong, and the answer turned out to be somewhere nobody was looking.

The engine, stripped to one molecule #

Szilard reduced the demon to its smallest possible form in 1929, and the reduction is what made the problem solvable.

Take a box containing exactly one molecule in contact with a heat bath at temperature $T$. Insert a partition down the middle: this costs nothing, and the molecule is now on one side or the other. Find out which side. Attach a load to the partition on the appropriate face and let the molecule push it out isothermally to the end of the box. Remove the partition. The box is back where it started.

During the expansion the molecule does work against the load, drawing the energy from the bath, and the work is

$$W = \int_{V/2}^{V} \frac{kT}{V'}\,dV' = kT\ln 2 .$$

Pressure is a rate of arrival at a wall, and a single molecule bouncing between two walls has a perfectly well-defined average pressure — it is just an average over a long time rather than over many molecules. That is what makes a one-molecule engine a legitimate object rather than a joke: everything thermodynamics says about it is a statement about averages, and averages survive the shrinking.

That is a complete cycle. Heat has been taken from one reservoir and turned entirely into work, which is exactly what Carnot’s ceiling forbids — an engine with one reservoir has an efficiency of zero and this one has an efficiency of one.

With one reservoir, the ordinary ceiling on work is zero. The greatest efficiency available between two reservoirs depends on the ratio of their temperatures, and with only one the ratio is one and the ceiling is nothing at all. Szilard’s engine appears to extract work from exactly that situation, which is why it is a problem rather than a curiosity — it is not inefficient, it is forbidden.

What the measurement did #

The reduction to one molecule makes it obvious where the asymmetry entered. Before the measurement, the molecule could be on either side, and the number of accessible states is twice what it is after. Entropy is a logarithm of a count, so the count halving means the entropy fell by $k\ln 2$.

Nothing was done to the molecule. It was where it was before anybody looked, and it is where it was after. What changed is what is known, and the entropy of a system is a property of a description rather than of a configuration. That is the content of the third refutation above and it is the hardest of the site’s thermodynamic statements to accept, because it seems to make entropy subjective.

It does not, in a way worth stating precisely: the entropy is objective given the description, and the description is a physical fact about what a particular apparatus has recorded. A demon that has recorded which side the molecule is on differs physically from one that has not, and that difference is where the missing entropy went.

The bill, and where it falls due #

Which brings the argument to the place it took eighty years to find. The demon’s memory now holds one bit. To run the cycle again it needs a memory in a known state — an empty notebook — and getting there means erasing what is written.

Erasure maps two states to one. Whatever the memory held, afterwards it holds zero. That is logically irreversible, and a logically irreversible operation on a physical memory must compress the memory’s own state space, which means dumping the difference into the surroundings as heat.

The amount is exactly $kT\ln 2$: precisely the work the expansion delivered. The engine works, extracts $kT\ln 2$, and must spend $kT\ln 2$ to be ready to work again. The books balance to the last term, and the second law is not merely rescued but rescued with no margin at all.

That exactness is the most persuasive feature of the resolution. A bound that merely exceeded the extractable work would look like a patch; a bound that equals it looks like the same quantity counted twice, which is what it is.

Why the cost is at erasure and not at measurement #

The natural objection is that measurement is surely the expensive part, and it took a long time to see that it is not.

A measurement can be made logically reversible: it correlates the memory with the system, and correlating is an operation that has an inverse. Nothing in principle stops it being done arbitrarily gently, with arbitrarily little dissipation, in the same way that a quasi-static compression can be done with arbitrarily little. Bennett’s argument, in 1982, was that the demon can measure for free and cannot forget for free.

Why the cost falls at erasure rather than at measurement is a question about what a memory is. Anything a bit can be stored in — a charge on a capacitor, a domain in a film, a molecule on one side of a partition — is a system with two states separated by a barrier, and the probability of finding it in the wrong one falls exponentially with that barrier over $kT$. Measurement can be done reversibly, because it only correlates two systems. Erasure cannot, because it maps two states onto one, and that is a reduction in the number of arrangements which has to be paid for.

There is a corollary worth stating, because it reverses an intuition. A reversible computer has no fundamental energy cost. Any computation can be rearranged so that no information is discarded — keeping the intermediate results and un-computing them afterwards — and such a machine can in principle run on arbitrarily little energy per operation, at the price of running arbitrarily slowly and of the memory needed to hold the history.

The size of the bound, honestly #

Quoting Landauer’s limit without its context makes it sound like an engineering constraint on computing, and it is not.

At room temperature $kT\ln2$ is $2.87 \times 10^{-21}$ joules. A transistor in a modern processor dissipates on the order of $10^{-17}$ joules per switch — a factor of some three thousand above it — and that figure is dominated by charging and discharging capacitance rather than by anything logical. A processor consuming a hundred watts and performing $10^{18}$ bit-operations per second would be spending $10^{-16}$ joules each, and the Landauer floor for that rate is 3 milliwatts.

So the bound has never been what limits a computer, and saying so is part of quoting it. What it does establish is the shape of the ultimate limit: it scales with temperature, so the only way to make forgetting cheaper is to run cold, and it does not scale with anything about the device, so no material or architecture removes it.

The measurements that have been made #

The whole argument was a century of theory before anybody built one, and three experiments since 2010 have put numbers on it.

A colloidal particle in a double-well optical trap, in 2012, was erased by tilting and merging the wells, and the heat dissipated was measured from the particle’s trajectory. It approached $kT\ln2$ from above as the operation was slowed, and the excess fell as the inverse of the duration, exactly as the finite-time argument requires.

A single-electron box, in 2014, did the same with a charge on an island rather than a bead in a trap, and reached within a few per cent of the bound.

And a Szilard engine has been run. A colloidal particle in a spiral staircase potential was watched, and a barrier inserted whenever it happened to be on the favourable side; the particle climbed the staircase against gravity, powered by nothing but a heat bath and the information gathered by watching it. The energy extracted tracked the information acquired, measured in bits, with the proportionality constant $kT\ln2$.

The scale to keep in mind is $kT$, which is the natural unit of every quantity in this essay. At room temperature it is 4.1 zeptojoules, and Landauer’s bound is $kT\ln 2$ — about 3 zeptojoules per bit erased. A modern transistor switch dissipates something like a million times that, so the bound is nowhere near being the binding constraint on a computer. It is a statement about what is possible rather than about what is built.

None of the three left any room for the alternative resolutions that had been proposed over the previous century — that measurement must cost, that the demon cannot see, that a frictionless door cannot exist. The cost is at erasure, it is where the theory said, and it is the size the theory said.

What the demon has to do with the arrow of time #

The reason this rung sits at the end of the entropy ladder is that it closes a gap the earlier ones left.

The second law is a statement about a total entropy, and the earlier rungs computed that total for gases, for surfaces and for reservoirs. What they did not include was the entropy of anything that knows about the system. Once the observer’s memory is admitted as a physical object with states of its own, the accounting closes, and the demon becomes an ordinary participant subject to the same law rather than an exception to it.

A demon operating at molecular scale is not a small person. It is a physical device of comparable size to what it is sorting, subject to the same random buffeting as everything else there, and its own internal state is a physical configuration with an entropy. That is the observation that resolves the paradox: the demon cannot be idealised out of the thermodynamics, because it is made of the same stuff as the gas.

Sorting by speed, which is the version Maxwell wrote #

Szilard’s version replaces the temperature difference with a position, and it is worth going back to the original because the accounting is the same and the quantities are more familiar.

A still room contains molecules at every speed, so there is nothing to stop a fast one crossing a partition at a moment when a slow one is coming the other way. The sorting the demon performs requires no violation of anything — every individual event it exploits happens constantly and by itself. What it requires is knowing which event is happening, and that is the part with a price.

Each decision the door-keeper makes is a bit: fast or slow, this side or that. Each bit lowers the entropy of the gas by $k\ln 2$ and raises the entropy of the memory’s contents by the same amount, so the total is unchanged and no law is troubled while the memory is being filled. The demon can run for exactly as long as its notebook lasts, producing a genuine temperature difference and genuine work, and it is only when the notebook is full that the bill arrives.

That is a satisfying answer because it says when the exception ends rather than denying that it exists. A finite demon with a finite memory really does beat the second law for a finite time, by an amount exactly equal to what erasing its memory will cost.

The demon’s engine is a Carnot cycle with the temperature difference manufactured rather than supplied. Once the sorting has produced a hot side and a cold side, extracting work is entirely ordinary — the area enclosed by a loop on pressure–volume axes, with no controversy anywhere in it. The whole question is what the manufacture cost, and the answer is that it cost at least what the engine will yield.

Seen that way, blank memory is a thermodynamic resource in the same sense that a cold reservoir is. An engine needs somewhere to put waste heat; a demon needs somewhere to put waste bits, and the two are the same requirement expressed in different units.

The answer that stood for thirty years and was wrong #

The essay has said that the resolution took eighty years and that the cost falls at erasure rather than at measurement. In between there was an answer that was widely accepted, published by two serious people, and wrong in an instructive way.

Brillouin argued in 1951 that the demon must see the molecule, and that seeing costs. The box is full of thermal radiation at the same temperature as the gas, so a photon used to illuminate a molecule must be distinguishable from that background, which means its energy must exceed $kT$. Absorbing it dissipates at least that much, and a short calculation puts the dissipation per observation at about $kT\ln2$ — exactly enough to cancel the engine’s output. Gabor reached a similar conclusion in the same year by a different route.

The arithmetic is correct and the conclusion, as a bound, does not follow. What Brillouin analysed was one scheme: find the molecule by bouncing light off it. He showed that that method costs at least the amount in question. A cost for one method is not a lower bound over all methods, and there is no argument in the paper that every possible way of learning which half a molecule is in must dissipate.

Bennett supplied the counterexample in 1982, and the form of it settled the matter. He described explicit mechanical arrangements — a partition coupled to a memory bit by a linkage, moved arbitrarily slowly — in which the memory ends up correlated with the molecule’s position and no energy is dissipated at all. The operation is logically reversible: from the final state, the initial one can be recovered. Nothing forbids doing it for free.

What cannot be done for free is the step Brillouin’s analysis did not contain, because his demon was implicitly assumed to start fresh each time. A cycle requires the memory to be returned to a known state, and returning two possible states to one is where the irreversibility is.

The episode is worth carrying for its shape rather than its content. A demonstration that one implementation costs something is not a proof that the cost is fundamental, and the difference between the two is exactly the difference between exhibiting an example and quantifying over all of them. Thirty years of a subject rested on the confusion, and it was resolved not by a better argument that measurement must cost but by somebody constructing a measurement that does not.

Why running cold does not help #

The bound is proportional to the temperature, which suggests an obvious escape: run the computer cold, and each erasure costs less. At liquid-helium temperatures $kT\ln2$ is seventy-five times smaller than at room temperature.

The escape does not work, and the reason is a nice application of the rest of thermodynamics to a result from information theory.

Erasing a bit at temperature $T$ dumps $kT\ln2$ of heat into a bath at $T$. That heat does not stay there. The machine is sitting in a room at some ambient temperature $T_0$, and the heat has to be moved from the cold bath to the room — which is refrigeration, and refrigeration against a temperature difference costs work.

The least work a refrigerator can use to move heat $Q$ from $T$ to $T_0$ is $Q(T_0 - T)/T$. Adding that to the erasure itself gives a total of

$$kT\ln 2 + kT\ln 2\,\frac{T_0 - T}{T} = kT_0\ln 2,$$

and the operating temperature has cancelled completely. The thermodynamic cost of erasing a bit, charged to the room the machine is in, is $kT_0\ln 2$ whatever temperature the machine runs at.

That is a satisfying result and it is the expected one, because the ambient temperature is the only temperature in the problem that was not chosen. A designer can pick the machine’s temperature and cannot pick the environment’s, and a bound that could be lowered by a free choice would not be a bound.

In practice the situation is worse than the arithmetic, because a real refrigerator is far from Carnot: a dilution refrigerator delivering a microwatt at ten millikelvin consumes kilowatts, which is four or five orders of magnitude off the ideal. Cryogenic computing is pursued for other reasons entirely — superconducting interconnects that dissipate nothing in transit, faster switching, lower leakage — and never on the grounds that forgetting is cheaper down there.

The general lesson is one this collection keeps meeting. A bound quoted at the temperature of the device is not the bound that matters; the bound that matters is quoted at the temperature of the surroundings, because that is where the waste finally goes and where the accounting has to close.

Where the model stops #

The bound is on the average, not on every instance. At the scale of a few kT, fluctuations are the same size as the quantities, and an individual erasure can cost less than $kT\ln2$ or return energy to the operator. What cannot happen is for the average over many to fall below the bound, and the modern statements are fluctuation theorems that make that precise.

And it is a bound on the ideal, reached only infinitely slowly. Erasing in a finite time costs more, with the excess going as the inverse of the time, so the trade is the same one every part of this subject has: approach the bound and get nothing done. The structure is identical to the ceiling on an engine that produces no power.

The memory is treated as two symmetric states. A real memory has states of unequal energy, unequal volume in state space, and a nonzero probability of flipping on its own, and each of those modifies the bound in a way that has been worked out and is not the clean $kT\ln2$.

And nothing here says what information is. The essay uses “one bit” to mean “a factor of two in a state count”, which is a thermodynamic quantity with units of $k$. That coincides numerically with the information-theoretic bit and is not obviously the same thing; the identification is a hypothesis that has survived every test made of it and is not a theorem.

What the pictures cannot show #

The Szilard figure draws a smooth isotherm, and a one-molecule gas has no smooth pressure. The curve is $\langle p\rangle = kT/V$, an average over many cycles or a long time, and any single expansion is a sequence of impacts with wildly varying intervals. Everything in the essay is an average, and at one molecule the fluctuations are of the same size as the averages.

Nor can any figure show the demon. It appears in the argument as a memory with two states and a rule, and its physical realisation — the thing that has to be built, that has to hold a bit against thermal noise, and that has to be reset — is exactly what the resolution is about and exactly what no drawing here contains.

Where this ladder goes next #

Six rungs of this ladder have argued that entropy is a count: of arrangements, of surface areas, of ways of distributing energy, of ways of mixing. This rung adds the observer’s notebook to the list of things being counted, and the addition is what makes the second law complete rather than a rule with an exception waiting to be exploited.

The habit worth carrying away is the closure test. When a machine appears to beat a conservation law, look for the part of it that has not been returned to its initial state. Szilard’s engine does not close because the memory does not; a ratchet driven by fluctuations does not close because the pawl heats up; a perpetual motion machine of the second kind never closes anywhere. The cycle is the whole of the argument, and cycle means every part, including the parts that only hold information.

With that, the anchor is finished. What remains to be said about the counting of states in this collection belongs to the diffusion that only runs forwards, to the free energy a system at fixed temperature actually minimises, and to the equipartition of energy among a system’s ways of moving — three subjects with their own anchors, all of which take the count for granted and use it.