Illustrated Physics
physics, drawn
Series

Entropy — the series

11 essays on one idea, from the one that introduces it to the one that assumes the rest.
  1. Ways to arrange 10 coins. The number of distinct arrangements giving each number of heads, for 10 coins. Every individual arrangement is equally likely; the middle wins because there are more ways to reach it.

    Entropy is a count, and the arrow of time is arithmetic

    Nothing in mechanics prefers a direction. Entropy is not a force pushing things toward disorder — it is the observation that some outcomes have vastly more ways of happening than others.

    part 1 · thermodynamics
  2. One mass, two entropies. The entropy of a solar mass as ordinary gas, generously counted at ten Boltzmann constants per proton, against the entropy of a solar-mass horizon. The first is 1.19·10⁵⁸ k and the second 1.05·10⁷⁷ k — a factor of 8.83·10¹⁸. This is why a horizon had to be given an entropy: without one, dropping anything at all through it destroys entropy and the second law fails.

    The entropy that lives on a surface

    Throw a cup of tea through a horizon and the entropy of the outside world falls. Either the second law is wrong or the horizon has an entropy of its own — and the only quantity available for it turns out to be its area, in units of a length made from gravity, quantum mechanics and the speed of light together.

    part 2 · thermodynamics
  3. A ratio in the exponent. The Boltzmann factor against the energy of a state measured in units of kT, with the logarithm up the axis so that the straight line is the whole content. Every kT of energy costs a factor of e, so 10, 20, 30 times kT are factors of 10^-4.3, 10^-8.7, 10^-13.0. At 300 K, kT is 25.9 meV, so a barrier of 0.35 eV is 13.5 kT and a factor of 1.3e-6. That is the sense in which a third of an electronvolt is not a small energy: it is small compared with a chemical bond and enormous compared with kT, and it is the second comparison that decides whether anything happens.

    The exponential that decides everything

    Maximising the number of ways a reservoir can arrange what is left after taking E out of it gives one factor, e to the minus E over kT. Its exponent is a ratio, which is why a barrier of a third of an electronvolt — nothing at all by chemical standards — is the difference between instantly and never.

    part 3 · thermodynamics
  4. Ways to arrange 12 coins. The number of distinct arrangements giving each number of heads, for 12 coins. Every individual arrangement is equally likely; the middle wins because there are more ways to reach it.

    What a system actually minimises

    A ball falls to the bottom of a bowl and a gas fills a room, and neither of those is the rule. A system in contact with a large reservoir minimises U − TS, and the minus sign is the reservoir's own entropy written in the system's variables — which is why a rubber band pulls harder when it is heated.

    part 4 · thermodynamics
  5. The entropy of mixing, and the entropy of not mixing. The entropy gained when two ideal gases at the same temperature and pressure are allowed to mix, per particle and in units of Boltzmann's constant, against the proportion of the mixture that is the first gas. The curve has no property of either gas in it — not their masses, not their sizes, not how strongly they interact, since ideal gases do not — and it is largest at 0.500, where it reaches ln 2 = 0.6931. Below it is the same quantity for two samples of the SAME gas, which is zero at every proportion: removing the partition between two halves of a box of nitrogen changes nothing that can be measured, and putting it back recovers the original state. The two results are correct and they do not join up. Make the two gases more and more alike — two isotopes, then two nuclear spin states, then nothing at all — and the upper curve does not descend to meet the lower one; it stays exactly where it is until the two species become identical, and then jumps. What the figure is really about is that the jump is in the counting and not in the gas.

    Mixing what is already mixed

    Let two different gases into each other's halves of a box and the entropy rises by a fixed amount that contains nothing about either gas. Do it with the same gas on both sides and it rises by nothing. Make the gases more and more alike and the answer does not converge — it jumps.

    part 5 · thermodynamics
  6. The work one molecule and one bit are worth. The pressure of a gas of one molecule against its volume, at 300 K, in units of the volume it starts in. The shaded area is the work the molecule does pushing a partition out isothermally, and it is measured here by integrating the drawn curve rather than written down: expanding by 1.5× yields 0.4055 kT against ln 1.5 = 0.4055; expanding by 2× yields 0.6931 kT against ln 2 = 0.6931; expanding by 4× yields 1.3863 kT against ln 4 = 1.3863; expanding by 8× yields 2.0794 kT against ln 8 = 2.0794, agreeing to 2.0e-10. The doubling is the one that matters, because a partition inserted in the middle leaves the molecule on one side or the other, and knowing which is what lets the load be attached to the right face. That single expansion delivers kT·ln2 = 2.87 zeptojoules at 300 K. It looks like work extracted from one temperature, and it is — until the engine is asked to run again, which requires forgetting which side the molecule was on.

    The bit that has to be paid for

    One molecule in a box, a partition, and the knowledge of which side it went — enough, between them, to extract work from a single reservoir, which the second law forbids. The engine is real and the arithmetic is right. What closes the loophole is that the cycle does not finish until the knowledge has been thrown away, and throwing away one bit costs exactly what the expansion delivered.

    part 6 · thermodynamics
  7. Runs that break the second law, and how often. The work done in a process repeated many times, and the same for the process run in reverse with its work reflected, for a free-energy change of 4 kT and a dissipation of 3 kT. The average work exceeds the free-energy change, which is the second law, and individual runs do not have to: the shaded tail is the fraction of runs that do less work than the free energy — trajectories in which the entropy of the universe went down — and it is 11.03% here. The two curves cross exactly at the free-energy change, whatever the dissipation, which is what makes an irreversible measurement able to report an equilibrium quantity.

    The second law, with a probability attached

    Entropy increases, on average. For a small system pulled quickly, individual runs go the other way — and how often is not a matter of taste but an exact number, fixed by a relation with no adjustable constant in it and no requirement that anything be near equilibrium.

    part 7 · thermodynamics
  8. The price of precision never falls below two. For a clock ticking forwards and backwards with a free-energy drop a per step, the product of its squared relative uncertainty and the entropy it has produced, in units of k: exactly a·coth(a/2), whatever the time elapsed. It approaches the bound of 2 only as the driving vanishes and grows as a once the driving is strong; the three clocks compared elsewhere sit at 2.04 (0.5 kT), 2.63 (2.0 kT), 8.01 (8.0 kT). No steady-state process of any design lies below the line at 2 — the thermodynamic uncertainty relation.

    The precision a clock pays for in heat

    Anything that counts — a molecular motor stepping along a filament, a chemical oscillator keeping time in a bacterium, a clock ticking on a wall — has to be driven, because a count that is not driven runs backwards as often as forwards. The heat it gives off buys it regularity, and there is an exact floor under the price: the squared relative uncertainty of the count, times the entropy produced, is never less than two in units of Boltzmann's constant.

    part 8 · thermodynamics
  9. The race to equilibrium that the hotter start wins. How far each system is from equilibrium with the bath — the relative entropy of its distribution from the bath's, in nats, on a logarithmic axis — against time since it was placed in the bath at 0.30, also logarithmic, for a start at 0.50 (warm, dashed) and one at 0.901 (hot, solid). The hot system starts further away: 1.243 against 0.254. Both close in quickly as their left wells equilibrate internally. Then the warm system must wait for probability to leak between the wells, and stalls, while the hot one, whose wells already had the right shares, carries on falling. The hot curve overtakes the warm one at a time of about 6.9, and from then on the hotter start is always nearer equilibrium.

    The hotter start that cools first

    Put two identical systems, one warm and one hot, into the same cold bath, and common sense says the warm one reaches the bath's temperature first, since the hot one has further to go. The claim that hot water sometimes freezes before warm water — the Mpemba effect — has been argued over for sixty years and does not survive careful measurement. But a cleaner version of it is real and has been observed: in systems whose relaxation has a slow route and a fast one, a hotter start can miss the slow route entirely and overtake a cooler start, exponentially. Whether a system cools quickly depends not on how far it is from equilibrium but on how much of the slowest way back it needs.

    part 9 · thermodynamics
  10. A ring of balls that forgets, and then remembers. Kac's ring: 400 balls on 400 sites round a circle, all white at the start, with 52 of the edges between sites marked; each tick every ball moves one site and changes colour if it crosses a marked edge. The solid line is the excess of white over black, as a fraction of all the balls, against ticks; the dashed line is Boltzmann's prediction (1 − 2μ)^t with μ = 0.1, the fraction of edges marked. For the first few dozen ticks the ring follows Boltzmann's decay exactly as an irreversible process would: the colours mix and the excess falls to zero, then wanders by a few hundredths. At tick 400 every ball has gone once round, through all 52 markers — an even number of colour changes — and the ring is all white again. A rule with no randomness in it, run forward, looked irreversible for 400 ticks and then undid itself.

    The ring that forgets and then remembers

    Put a few hundred balls on a ring, all white, and mark a handful of the gaps between them. Each tick, every ball moves one place round and changes colour if it crosses a mark. Nothing is random and everything can be run backwards. Yet the colours mix, half and half, exactly as Boltzmann's equation for a gas says they should, and the entropy rises to its maximum and stays there — until, after one full turn, every ball is white again. Mark Kac built the model in 1956 to show where the arrow of time in Boltzmann's equation comes from, and what it quietly assumes.

    part 10 · thermodynamics
  11. An attraction found by counting. How often two large discs are found at each gap between their surfaces, relative to how often they are found far apart, in a direct simulation of two discs among 270 small particles that pass through each other freely and do not enter the discs (dots), and the prediction from counting the small particles' available area (line); small particles a quarter of the large radius. Nothing in the simulation attracts anything: every move that does not put a small particle inside a large one is accepted. Yet the discs are found touching 2.7 times as often as apart (2.6 predicted), because touching returns the overlap of their exclusion zones to the small particles, and the probability of a separation is proportional to the number of arrangements the small particles have at it. Beyond a gap of twice the small radius the zones no longer overlap and the effect vanishes. Written as a potential, −kT times the logarithm of that ratio, it is an attraction about 1.0 kT deep.

    The attraction that is the small ones' room

    Put large particles into a crowd of small ones that cannot get inside them, and the large ones begin to stick together — though nothing in the mixture attracts anything. Each large particle keeps the small ones' centres out of a thin shell around it. When two large particles touch, their shells overlap, and the overlap is handed back to the small particles as room to move in. More room is more arrangements, more arrangements is more entropy, and the large particles are pushed together by the pressure of the small ones on their outer sides. Fumio Oosawa and Sho Asakura worked it out in 1954. The force is now used to purify DNA, sort colloids onto patterned surfaces and explain why the inside of a cell behaves as though its proteins were stickier than they are.

    part 11 · thermodynamics

All series