Illustrated Physics
physics, drawn
Series

Entropy — the series

7 essays on one idea, from the one that introduces it to the one that assumes the rest.
  1. Ways to arrange 10 coins. The number of distinct arrangements giving each number of heads, for 10 coins. Every individual arrangement is equally likely; the middle wins because there are more ways to reach it.

    Entropy is a count, and the arrow of time is arithmetic

    Nothing in mechanics prefers a direction. Entropy is not a force pushing things toward disorder — it is the observation that some outcomes have vastly more ways of happening than others.

    part 1 · thermodynamics
  2. One mass, two entropies. The entropy of a solar mass as ordinary gas, generously counted at ten Boltzmann constants per proton, against the entropy of a solar-mass horizon. The first is 1.19·10⁵⁸ k and the second 1.05·10⁷⁷ k — a factor of 8.83·10¹⁸. This is why a horizon had to be given an entropy: without one, dropping anything at all through it destroys entropy and the second law fails.

    The entropy that lives on a surface

    Throw a cup of tea through a horizon and the entropy of the outside world falls. Either the second law is wrong or the horizon has an entropy of its own — and the only quantity available for it turns out to be its area, in units of a length made from gravity, quantum mechanics and the speed of light together.

    part 2 · thermodynamics
  3. A ratio in the exponent. The Boltzmann factor against the energy of a state measured in units of kT, with the logarithm up the axis so that the straight line is the whole content. Every kT of energy costs a factor of e, so 10, 20, 30 times kT are factors of 10^-4.3, 10^-8.7, 10^-13.0. At 300 K, kT is 25.9 meV, so a barrier of 0.35 eV is 13.5 kT and a factor of 1.3e-6. That is the sense in which a third of an electronvolt is not a small energy: it is small compared with a chemical bond and enormous compared with kT, and it is the second comparison that decides whether anything happens.

    The exponential that decides everything

    Maximising the number of ways a reservoir can arrange what is left after taking E out of it gives one factor, e to the minus E over kT. Its exponent is a ratio, which is why a barrier of a third of an electronvolt — nothing at all by chemical standards — is the difference between instantly and never.

    part 3 · thermodynamics
  4. Ways to arrange 12 coins. The number of distinct arrangements giving each number of heads, for 12 coins. Every individual arrangement is equally likely; the middle wins because there are more ways to reach it.

    What a system actually minimises

    A ball falls to the bottom of a bowl and a gas fills a room, and neither of those is the rule. A system in contact with a large reservoir minimises U − TS, and the minus sign is the reservoir's own entropy written in the system's variables — which is why a rubber band pulls harder when it is heated.

    part 4 · thermodynamics
  5. The entropy of mixing, and the entropy of not mixing. The entropy gained when two ideal gases at the same temperature and pressure are allowed to mix, per particle and in units of Boltzmann's constant, against the proportion of the mixture that is the first gas. The curve has no property of either gas in it — not their masses, not their sizes, not how strongly they interact, since ideal gases do not — and it is largest at 0.500, where it reaches ln 2 = 0.6931. Below it is the same quantity for two samples of the SAME gas, which is zero at every proportion: removing the partition between two halves of a box of nitrogen changes nothing that can be measured, and putting it back recovers the original state. The two results are correct and they do not join up. Make the two gases more and more alike — two isotopes, then two nuclear spin states, then nothing at all — and the upper curve does not descend to meet the lower one; it stays exactly where it is until the two species become identical, and then jumps. What the figure is really about is that the jump is in the counting and not in the gas.

    Mixing what is already mixed

    Let two different gases into each other's halves of a box and the entropy rises by a fixed amount that contains nothing about either gas. Do it with the same gas on both sides and it rises by nothing. Make the gases more and more alike and the answer does not converge — it jumps.

    part 5 · thermodynamics
  6. The work one molecule and one bit are worth. The pressure of a gas of one molecule against its volume, at 300 K, in units of the volume it starts in. The shaded area is the work the molecule does pushing a partition out isothermally, and it is measured here by integrating the drawn curve rather than written down: expanding by 1.5× yields 0.4055 kT against ln 1.5 = 0.4055; expanding by 2× yields 0.6931 kT against ln 2 = 0.6931; expanding by 4× yields 1.3863 kT against ln 4 = 1.3863; expanding by 8× yields 2.0794 kT against ln 8 = 2.0794, agreeing to 2.0e-10. The doubling is the one that matters, because a partition inserted in the middle leaves the molecule on one side or the other, and knowing which is what lets the load be attached to the right face. That single expansion delivers kT·ln2 = 2.87 zeptojoules at 300 K. It looks like work extracted from one temperature, and it is — until the engine is asked to run again, which requires forgetting which side the molecule was on.

    The bit that has to be paid for

    One molecule in a box, a partition, and the knowledge of which side it went — enough, between them, to extract work from a single reservoir, which the second law forbids. The engine is real and the arithmetic is right. What closes the loophole is that the cycle does not finish until the knowledge has been thrown away, and throwing away one bit costs exactly what the expansion delivered.

    part 6 · thermodynamics
  7. Runs that break the second law, and how often. The work done in a process repeated many times, and the same for the process run in reverse with its work reflected, for a free-energy change of 4 kT and a dissipation of 3 kT. The average work exceeds the free-energy change, which is the second law, and individual runs do not have to: the shaded tail is the fraction of runs that do less work than the free energy — trajectories in which the entropy of the universe went down — and it is 11.03% here. The two curves cross exactly at the free-energy change, whatever the dissipation, which is what makes an irreversible measurement able to report an equilibrium quantity.

    The second law, with a probability attached

    Entropy increases, on average. For a small system pulled quickly, individual runs go the other way — and how often is not a matter of taste but an exact number, fixed by a relation with no adjustable constant in it and no requirement that anything be near equilibrium.

    part 7 · thermodynamics

All series