Mixing what is already mixed

A box is divided in two. On the left, a mole of nitrogen; on the right, a mole of oxygen, at the same temperature and pressure. Remove the partition, wait, and the entropy of the contents has risen by $2R\ln 2$ — about eleven and a half joules per kelvin, for those two moles. Nothing has been heated, no work has been done, and no chemistry has occurred.

Now run the same experiment with nitrogen on both sides. Remove the partition, wait, and the entropy has risen by nothing at all.

Both answers are right. What makes them a puzzle rather than a pair of facts is the sequence between them: make the two gases more and more similar — argon and krypton, then two isotopes of argon, then two nuclear spin states of the same isotope — and the upper answer does not descend toward the lower one. It stays exactly where it is, and then jumps.

What the entropy of mixing is counting #

Entropy is a logarithm of a count, and the count here is a count of arrangements over positions.

Before the partition is removed, each nitrogen molecule is somewhere in the left half and each oxygen molecule somewhere in the right. Afterwards, each molecule of either kind is somewhere in the whole box. Each has twice as many places to be, so the count multiplies by $2^N$ for each species, and the entropy rises by $Nk\ln 2$ per species. Two species, and the total is $2Nk\ln 2$.

Written that way the result is obviously independent of what the gases are. It is a statement about volumes, not about species: each gas has doubled the space available to it. And that immediately explains the identical-gas case, because when both sides hold the same gas, nothing has doubled the space available to anything — the molecules could already be anywhere in the box, and the partition was only a wall they happened not to be crossing.

What actually happens after the partition is removed is diffusion, and in an undisturbed metre-scale box it takes hours. The entropy change does not depend on any of that. It is a difference between two equilibrium states, and the path between them — fast or slow, stirred or still — does not enter the calculation at all. This is the ordinary situation with state functions and it is worth noticing here, because the mechanism is so vivid that it is tempting to think the answer must depend on it.

The work that can be extracted, which is the real measurement #

An entropy difference is not a bookkeeping entry. It is a statement about work, and the statement is testable.

Suppose two semipermeable membranes are available: one that passes nitrogen freely and blocks oxygen, one that does the reverse. Such membranes exist — palladium passes hydrogen and nothing else, and the principle is the same. Slide them slowly through the box in opposite directions and each gas expands against the pressure of its own kind alone, doing work. The total work recoverable, at temperature $T$, is exactly $T$ times the entropy of mixing: about 3.4 kilojoules for the two moles.

Now run the argument with the same gas on both sides. A membrane that passes nitrogen and blocks nitrogen does not exist, and could not, and the impossibility is not a technological one. So no work can be extracted, and the entropy of mixing must be zero — not by convention, but on pain of a machine that generates work from a partition being lifted.

The distinguishability that matters is the existence of a process that can tell the two apart. That is what turns the question from a philosophical one into a physical one, and it is why the answer is a step rather than a slope: either a membrane can exist or it cannot, and there is no partial membrane for two things that differ by nothing.

That ceiling is the one every such extraction runs into. Whatever the mechanism, the work obtainable from a difference between two reservoirs — of temperature in the engine case, of composition here — is bounded by the entropy that would otherwise have been generated. A device recovering more than that would run a cycle whose net effect is heat turned entirely into work, which is the second law stated as a prohibition rather than as a count.

The paradox, stated properly #

The version of this that circulates as “Gibbs’s paradox” is usually put as a puzzle about continuity: mixing entropy is a discontinuous function of the difference between the gases, and physics is not supposed to work that way. That framing is worth taking apart, because the discontinuity is in the wrong place to be alarming.

Consider the sequence of experiments. Nitrogen and oxygen: full mixing entropy, and a membrane exists. Argon-36 and argon-40: full mixing entropy, and a membrane exists in the form of a centrifuge or a mass spectrometer, which is a membrane with extra steps. Ortho- and para-hydrogen, which differ only in the relative orientation of two nuclear spins: full mixing entropy, and separation is possible with a magnetic catalyst. Two samples of the same molecule in the same state: zero, and no separation is possible by any means whatever.

At no point in that sequence do the two species become slightly different. They differ in some property or they do not, and the property either supports a separation process or it does not. The apparent continuity — argon-36 and argon-40 being “nearly the same” — is a statement about their chemistry and their masses, neither of which enters the mixing entropy at all. The function is discontinuous in a variable that does not exist.

What is continuous, and is worth noticing, is the difficulty. Separating nitrogen from oxygen is routine; separating argon isotopes is expensive; separating nuclear spin states requires a catalyst and patience. The work recoverable is the same in every case, and the fraction of it that any real apparatus recovers falls steeply as the species become harder to tell apart. The thermodynamics does not interpolate and the engineering does, and conflating the two is where the sense of paradox comes from.

The counting that had to be repaired #

There is a second, deeper problem hidden in the same box, and it appears even with one gas.

Count the arrangements of $N$ molecules in a volume $V$ as though each molecule carried a label. The number of positions available to the set is proportional to $V^N$, so the entropy is $Nk\ln V$ plus terms in the temperature. Now double everything: two identical boxes, joined. The entropy should double, because entropy is extensive — a quantity that scales with the size of the sample, like mass or volume.

It does not.

Gibbs saw that the repair is to divide the count by $N!$, the number of ways of permuting the particles among themselves — that is, to declare that two arrangements differing only in which molecule sits where are the same arrangement. Using Stirling’s approximation, $\ln N! \approx N\ln N - N$, the division removes exactly the offending $\ln N$ and leaves an entropy per particle that does not depend on the sample size.

This is not a mathematical convenience. It is an assertion about the world, and it has consequences that were measured: the absolute entropy of a gas enters its vapour pressure over the corresponding solid, and the Sackur–Tetrode expression — which contains the division — matches the measurements, while the undivided version does not.

What makes the episode remarkable is the chronology. Gibbs made the correction in 1875 on the grounds that without it thermodynamics gave nonsense, with no justification available from classical mechanics, where particles unquestionably have trajectories and are therefore in principle followable and labelled. The justification arrived fifty years later, from quantum mechanics, where identical particles are not merely hard to tell apart but do not have separate identities at all.

The same indistinguishability, in a different subject #

The connection is not an analogy but a shared premise. In both cases the physics depends on how alternatives are counted, and in both the rule is that alternatives which no measurement can separate are one alternative. In the box, that rule removes the $N!$ and makes entropy extensive. At the slits, it makes amplitudes add before they are squared rather than after. It is the same statement about identity applied to two different sums.

The connection runs further than that. Insisting that particles are genuinely identical, and asking what states a collection of them may occupy, produces the two quantum statistics — and the rule that no two fermions may share a state is what a mixing entropy would have looked like if it had been discovered from the quantum side first.

What the division is worth, as a number #

The size of the correction is worth putting on a scale, because “an entropy that is not extensive” sounds like a formal complaint rather than a quantitative failure.

For a mole of gas, $\ln N!$ is about $N \ln N - N$ with $N = 6\times10^{23}$, so the correction to the entropy is roughly $Nk(\ln N - 1)$, which is about 440 joules per kelvin — comparable to the whole entropy of the gas, which for argon at room conditions is about 155 joules per kelvin per mole. The undivided count does not give a slightly wrong answer; it gives an answer three times too large and growing with the sample.

That is why the Sackur–Tetrode expression is a genuine test rather than a formality. It gives the absolute entropy of a monatomic gas in terms of its mass, temperature and pressure and nothing adjustable, and it contains Planck’s constant — a volume per particle has to be measured against something, and the something turns out to be $h^3$. Argon’s measured entropy at 298 K and one atmosphere is 154.8 joules per kelvin per mole; the expression gives 154.7. The agreement tests the division, the constant, and the whole apparatus of counting states at once.

The work nobody comes close to #

The essay says that the thermodynamics does not interpolate and the engineering does. That is worth quantifying, because the gap between the two is not a factor of a few.

Uranium as it is mined is 0.72 per cent uranium-235 and the rest uranium-238. Two isotopes of one element, chemically indistinguishable, differing by three neutrons. The mixing entropy of that composition is 0.043 in units of Boltzmann’s constant per atom, so the minimum work required to unmix it at room temperature is about a hundred joules per mole of uranium — a fraction of a watt-hour per kilogram.

The actual figures are elsewhere. Gaseous diffusion, the process that produced the world’s enriched uranium for four decades, consumed around two and a half thousand kilowatt-hours per unit of separative work; the centrifuge plants that replaced it use around fifty. Converting either to energy per kilogram of feed puts the practical cost four or five orders of magnitude above the thermodynamic floor.

The reason is the same one that makes the separation hard in the first place. Both processes work on a property in which the two isotopes barely differ — a mass ratio of 1.0086 in the hexafluoride — so each stage achieves a separation factor barely above one, and thousands of stages have to be cascaded, each recompressing and recirculating almost all of the material it handles. Essentially all of the energy goes into moving gas about, and essentially none of it into the separation.

That is the honest content of the discontinuity. The recoverable work does not depend on how similar the species are; the fraction of it that a real process recovers depends on almost nothing else. Separating nitrogen from oxygen, where the difference is chemical, is done at a large fraction of the ideal cost; separating two isotopes, where the difference is a per cent of a mass, is done at a hundred-thousandth of it.

And the pattern generalises past uranium. Deuterium is concentrated by exploiting a chemical isotope effect rather than a mass difference, at a few hundred times the ideal cost; carbon-13 by cryogenic distillation of carbon monoxide, similarly. In every case the thermodynamic bound is a small number that says the task is allowed, and the engineering cost is a large one that says how the allowance is spent.

Whose entropy is it #

There is a reading of this subject that complicates the clean statement above, and it is worth having because the essay’s own remark about ortho- and para-hydrogen already contains it.

Jaynes put it in the form of a thought experiment. Suppose two gases are identical in every property that anybody has yet discovered, and that later someone finds a property in which they differ and builds a membrane that exploits it. Before the discovery, an experimenter computes zero mixing entropy, predicts that no work can be extracted, and is right about every experiment available to him. After the discovery, an experimenter with a membrane extracts work and computes a mixing entropy that is not zero, and is right about every experiment available to her.

Neither was mistaken. The entropy is a property of a macrostate, and a macrostate is specified by the variables the experimenter chooses to control and to measure. Widening that set — by discovering a new property, or by acquiring a new instrument — changes what counts as one state and what counts as two, and therefore changes the entropy.

That is not a licence for the entropy to be whatever anybody wants. Given a stated set of controllable variables the answer is definite, and the second experimenter cannot extract work with the first experimenter’s apparatus. What it does say is that “distinguishable” in this subject means distinguishable by the processes available, and that the phrase carries a reference to the apparatus that is usually left implicit.

The essay’s ortho- and para-hydrogen case is exactly that reference made explicit. The two forms interconvert slowly, so a sample behaves as two species over an hour and as one over a year; the table entry depends on which, and every table says which. There is no fact of the matter that is independent of the timescale.

The position this leaves is narrower than the one the essay states and compatible with it. Quantum mechanics does supply an objective floor: two helium-4 atoms are not merely indistinguishable in practice but have no separate identities at all, so no undiscovered property can distinguish them and the zero is permanent. Between that floor and full chemical difference, though, the entropy of mixing is a statement about what a given apparatus can address — and the discontinuity is real for any fixed apparatus while sitting in a different place for different ones.

Where the model stops #

The gases must be ideal. The result contains no interaction between the species because there is none. Real gases have a heat of mixing as well, so the entropy change is not the whole story — mixing ethanol and water warms the mixture and shrinks it, and neither effect appears anywhere above.

What the ideality assumption means in practice is that each gas behaves as though the other were absent. The molecules’ speeds are distributed by the Maxwell–Boltzmann law and nothing about that distribution depends on the presence of the second species, which is precisely what makes the mixing entropy a sum of two independent expansions rather than something that has to be computed for the pair.

The temperature and pressure must match. If the two halves start at different pressures there is a further entropy of equalisation, which does depend on the ratio, and the mixing term is only one piece of the answer.

The step is a limit, not a discontinuity in nature. Two isotopes really are distinguishable and really do give the full mixing entropy; two nuclear spin states are distinguishable in principle, and whether the entropy is realised depends on whether anything in the experiment couples to spin on the timescale of the measurement. In practice a spin-mixing entropy that no process can access is a number in a table rather than a quantity that shows up in a calorimeter, and the honest statement is that the entropy depends on which degrees of freedom the apparatus can address.

What decides that last point in practice is a rate. Whether a process connecting two states is fast on the timescale of the experiment determines whether the states count as connected at all, and a degree of freedom frozen out by an exponential is one the entropy does not include. That is why the “absolute” entropy of a substance depends on which processes are considered available — the number is not a property of the substance alone but of the substance and the patience of the person measuring it.

What the pictures cannot show #

The mixing curve is drawn as a smooth function of composition, and the composition of a real sample is itself a fluctuating quantity — the number of nitrogen molecules in the left half after mixing is not exactly half, and the curve is a statement about the mean.

The two lines in that figure are drawn together to be compared, and that arrangement invites the eye to look for a path between them. There is none: they are two different answers to two different questions, and no continuous deformation of the apparatus takes one into the other.

And nothing here shows the mechanical reality underneath. Entropy is a count of arrangements; pressure is momentum arriving at a wall. Both are properties of the same molecules and neither is visible in the other’s picture, which is the usual situation in thermodynamics and the reason the subject needed statistical mechanics to join its two halves together.

The membrane, taken seriously #

The semipermeable membrane in the work argument is doing more than illustrating; it is the operational definition of the whole subject, and it is worth pressing on.

Separating what is mixed costs exactly what mixing it released, and a membrane that passes species A and blocks species B must couple to some property in which A and B differ — a size, a shape, a charge, a mass, a magnetic moment. If the two species differ in no property at all, no such coupling exists, and the impossibility of the membrane is not an engineering limitation but a statement that there is nothing for it to act on.

That is why the entropy of mixing is not a matter of what an experimenter happens to be able to do. It would be uncomfortable if the entropy of a box depended on whether anybody had yet invented a separation technique, and it does not: what enters is whether the species differ, not whether anybody has exploited the difference. The mixing entropy of two isotopes was the same before mass spectrometry as after.

There is a genuine subtlety underneath that, and it concerns which differences are available on the timescale of the experiment. Ortho- and para-hydrogen interconvert slowly without a catalyst — they differ in whether two identical particles may share a state — so a sample kept for an hour behaves as a mixture of two species and a sample kept for a year behaves as one. The entropy in a table depends on which convention is being used, and every table says which — a rare case of a thermodynamic quantity carrying an explicit statement of what was held fixed.

Where the ladder goes next #

This ladder has run from entropy as a count, through the exponential that makes the count sharp, to a case where the count depends on a question about identity rather than about states. The next rungs are the two quantum statistics proper — what changes when the indistinguishability is built in from the start rather than corrected for afterwards — and the information-theoretic reading, in which the entropy of mixing is the number of yes-or-no questions needed to specify which species each molecule is, and the identical-gas answer is zero because the question has no content.

The habit worth carrying away is about the shape of the answer. A quantity that jumps rather than sliding is a signal that something discrete is being counted, and the useful question is what. Here it is not the molecules and not the states but the cases: whether two situations are one situation. Whenever a physical answer refuses to interpolate, look for a count of alternatives underneath it — and expect the discontinuity to sit where two alternatives become one.