Illustrated Physics
physics, drawn
Astrophysics

The estimate that misses by a hundred and twenty

Every argument about the Planck scale is an argument about consistency rather than about data, with one exception. The zero-point energy of the quantum fields gravitates, dimensional analysis at the Planck cutoff says how much, and what is measured is 10¹²¹ times smaller. It is the largest disagreement between an estimate and a measurement anywhere in physics, and lowering the cutoff does not rescue it.
16 min read 5 figures Where the model stopsApproximations that lie

Assumes: The length no experiment can resolve · Where every model runs out at once

The length no experiment can resolve ends with a statement that sounds like a resignation and is meant as a description: every claim about physics at the Planck scale is a claim about consistency rather than about data, because no measurement of any kind reaches within fifteen decades of it.

There is one exception, and it is not a marginal one. The zero-point energy of the quantum fields is a real energy, energy gravitates, and how much energy the vacuum has is therefore an astronomical observable. So the scale makes a prediction that can be checked.

It is wrong by 10¹²¹.

The one prediction the Planck scale makes. The energy density the vacuum should have, from summing the zero-point energy of a field's modes up to a cutoff, against where that cutoff is put — thirty decades of cutoff energy and a hundred and thirty of density, both logarithmic. The horizontal line is what is measured: 5.34e-10 joules per cubic metre, the dark energy that accounts for sixty-nine per cent of the universe. Cutting the sum off at the Planck energy — which is where dimensional analysis says every description available runs out — overshoots it by 10^121. That is the largest disagreement between an estimate and a measurement anywhere in physics, and the slope of the line is why it cannot be argued away: the density goes as the fourth power of the cutoff, so cutting off at the electroweak scale still overshoots by 10^54 and cutting off at one electronvolt — below which no physics is in doubt at all — still overshoots by 10^8. The cutoff that would give the right answer is 8.0e-3 electronvolts, which is a wavelength of about a tenth of a millimetre and corresponds to no known physics whatever.
Fig. 1 The energy density the vacuum should have, from summing the zero-point energy of a field’s modes up to a cutoff, against where the cutoff is put — thirty decades of cutoff and a hundred and thirty of density. The horizontal line is what is measured. Cutting the sum off at the Planck energy overshoots by 10¹²¹; cutting it off at one electronvolt still overshoots by 10⁸. The cutoff that would work is eight millielectronvolts.

The estimate, which is one integral

Every mode of a quantum field has a ground-state energy of 12ℏω\tfrac{1}{2}\hbar\omega, and it has that energy whether or not anything is in the mode — which is the same irreducible motion a confined particle cannot be rid of, applied to a field with infinitely many modes instead of to one coordinate. Counting those modes up to a wavenumber cutoff kmaxk_\text{max} and summing gives an energy density

ρ∼ℏc kmax416π2\rho \sim \frac{\hbar c\,k_\text{max}^4}{16\pi^2}

which diverges as the cutoff is removed, and which for any cutoff at all is a number. It is the same count and the same divergence the blackbody spectrum ran into, with one difference that changes everything: there the energy per mode was wrong and Planck’s factor cured it, and here the factor is already in place and it is the ground state energy that will not go away.

This is not an optional piece of bookkeeping. The same zero-point energy, differenced between two configurations, is the attraction between two uncharged plates in vacuum — the Casimir force, measured to a few per cent, with no adjustable parameter. Take away the zero-point energy and that measurement has no explanation. So the vacuum’s energy is not a formal artefact; its differences are observed directly, and only its absolute value is the problem.

The absolute value is what gravitates. General relativity has no zero of energy to discard — every form of energy gravitates, which is the statement the equivalence principle is and the most heavily tested part of the theory: a constant energy density with an accompanying negative pressure is a cosmological constant, it curves spacetime, and its magnitude is measurable from how the expansion accelerates. That measurement gives 5.3×10−105.3\times10^{-10} joules per cubic metre, which is about four hydrogen atoms’ rest energy in a cubic metre, and which is the line on the figure. Set beside the Planck density — the Planck mass in a Planck volume, which is what the fourth-power estimate returns — it is the one Planck quantity anybody has ever been in a position to compare with an observation.

Why the cutoff cannot be blamed

The obvious response is that the Planck cutoff was an absurd place to stop the sum, and the figure exists to close that response off.

The density goes as the fourth power of the cutoff. So dropping the cutoff by a decade buys four decades of density, and buying a hundred and twenty decades costs thirty decades of cutoff. Thirty decades below the Planck energy is eight millielectronvolts — which is not a scale of any known physics, and is a wavelength of about a tenth of a millimetre.

Every intermediate choice fails by a large factor. The electroweak scale, 246 GeV, where the Higgs field’s own potential lives and which is tested at colliders: too large by 10⁵⁴. The energy the LHC reaches: 10⁶³. One electronvolt, below which there is no physics anybody doubts at all — atoms, chemistry, ordinary matter: still too large by 10⁸.

So the problem is not located at the Planck scale. It exists in the region of physics that is completely understood, and the Planck cutoff only makes it worse. That is the single most important thing to know about it, and it is why the problem cannot be deferred to a future theory of gravity.

How the measurement was made, and why it was believed

The observation that turned the constant from a possibility into a number is worth a section, because a result that overturns an estimate by a hundred and twenty decades has to be very solid indeed.

Type Ia supernovae are exploding white dwarfs and they are nearly standard in brightness — not identically so, but their brightness correlates with how fast they fade, which is measurable, so a corrected brightness can be recovered. A standard brightness gives a distance; a spectrum gives a redshift; and the relation between distance and redshift over many billions of years is the expansion history.

What two teams found in 1998 is that distant supernovae are fainter than a decelerating universe predicts — further away than they should be for their redshift. The expansion has been speeding up for the last few billion years, which requires something with negative pressure, and a constant vacuum energy is the simplest thing that has it.

Two things made it stick. First, the two teams were competing and used different samples and different analyses. Second, and more decisively, the conclusion is now overdetermined by measurements that share nothing with it: the angular scale of the fluctuations in the microwave background fixes the total density, the clustering of galaxies fixes the matter density, and the difference between them is the vacuum’s — with no supernova in the argument. Three independent routes to one number is what moved it from a candidate to a parameter.

It is worth noticing what that did to the problem. Before 1998, a theorist could hope the constant was exactly zero and look for the symmetry that made it so. Afterwards the target is a number that is neither zero nor natural, which is a much harder thing to explain and which is why the problem is often dated to 1998 rather than to 1967.

What a cancellation would have to do

The standard hope is that the various contributions cancel. It is worth seeing what that requires.

Four contributions, and what they must add up to. Four separately known contributions to the energy density of the vacuum, on a logarithmic scale spanning a hundred and thirty decades, with the measured total at the bottom. None of them is speculative. QCD quark condensate is 10^45 times the measured density; the electroweak vacuum is 10^56 times the measured density; zero point to the LHC's reach is 10^63 times the measured density; zero point to the Planck energy is 10^123 times the measured density. The quark condensate is the least arguable: it is required by the physics of the strong interaction, its size is fixed by measured hadron masses, and it alone is 45 decades too large. So the problem is not that a speculative high-energy estimate disagrees with observation; it is that contributions from physics nobody doubts have to cancel, against each other and against whatever else is there, to more than 45 decimal places — and to 123 if the sum runs to the Planck scale. Nothing in any theory requires such a cancellation, and no symmetry has been found that would produce one.
Fig. 2 Four separately known contributions, on a logarithmic scale spanning a hundred and thirty decades, with the measured total at the bottom. The quark condensate of the strong interaction is 10⁴⁵ times too large; the electroweak vacuum 10⁵⁶; the zero-point sum to the Planck energy 10¹²³. Each is required by physics that has been tested, and they have to sum, with whatever else is there, to the line at the bottom.

The condensate is the case that settles the argument. The vacuum of quantum chromodynamics is not empty: it contains a quark condensate and a gluon condensate whose existence is required by the observed masses of hadrons and by the pattern of chiral symmetry breaking, and whose magnitude is fixed by measurements. Its energy density alone exceeds the measured total by forty-five decades.

That contribution is not speculative, not optional, and not a high-energy extrapolation. It is the vacuum of the strong interaction, the same vacuum protons sit in, and it has to be cancelled to forty-five decimal places by something.

No symmetry is known that would do it. Supersymmetry cancels the bosonic and fermionic zero-point contributions exactly — which is one of the reasons it was attractive — but only if it is unbroken, and it is manifestly broken at the scales tested, which leaves a residue of order the breaking scale to the fourth power: still fifty decades too large. Adjusting a constant by hand to cancel the rest is possible and is what is done, but it is an adjustment to more decimal places than any other number in physics, and nothing explains why it should come out just above zero rather than just below or exactly at it.

The three things that are known about the answer

For a problem with no solution it is worth being precise about what has been established, because it is more than nothing.

It is not zero. Before 1998 the honest position was that the constant might be exactly zero, and a vanishing constant is a far more inviting target: exact cancellations are the sort of thing a symmetry does. The supernova measurements removed that hope. A constant that is zero needs a symmetry; a constant that is 10−12010^{-120} needs a symmetry that is broken by exactly the right tiny amount — and a symmetry handing over a conservation law is the only mechanism physics has for making a quantity exactly anything.

It does not change, to the precision available. The equation of state of the dark energy — the ratio of its pressure to its energy density — is measured to be −1-1 within a few per cent, which is what a constant vacuum energy gives and which is different from what a slowly rolling field would give. Every attempt to make the constant dynamical, so that its smallness is a consequence of the universe being old, is constrained by that measurement.

And it cannot be much larger. A vacuum density a hundred times the measured one would have begun accelerating the expansion much earlier, structure would not have had time to form, and there would be no galaxies — an application of the criterion that decides whether a cloud collapses or disperses to the whole universe. That is an anthropic bound, it is real, and its existence is why a selection argument is taken seriously here when it would not be elsewhere — the bound and the measurement are within a factor of a few of each other, which is the only place in physics where that is true.

The second strangeness

The first problem is the size. There is a second, and it is independent.

A constant beside two that dilute, crossing now. The energy densities of radiation, matter and the vacuum, in units of today's critical density, against the size of the universe relative to today — both logarithmic, from a millionth of the present size to a hundred times it. Radiation dilutes as the fourth power of the expansion because its wavelengths stretch as well as its density falling; matter as the third; the vacuum not at all, which is what a vacuum energy means. So there are two crossings and both are located here by bisection: radiation gives way to matter at a redshift of 3,369, and matter gives way to the vacuum at a redshift of 0.31 — which is to say a few billion years ago, out of fourteen. That is the second strangeness. The first is that the vacuum density is a hundred and twenty decades below its estimate; the second is that a quantity which does not change happens to be comparable with two that fall by decades, at this moment rather than at any other. A hundredfold change in it would put the crossing far in the past or far in the future, and either way nothing would be here to notice.
Fig. 3 The energy densities of radiation, matter and the vacuum against the size of the universe. Radiation dilutes as the fourth power, matter as the third, and the vacuum not at all. Both crossings are located by bisection: radiation gives way to matter at a redshift of 3,369, and matter gives way to the vacuum at a redshift of 0.31 — which is a few billion years ago, out of fourteen.

A vacuum energy density is constant. Matter’s density falls as the cube of the expansion and radiation’s as the fourth power. So the three curves cross, and there is no reason for any of the crossings to be anywhere in particular.

The matter–vacuum crossing is at a redshift of about a third. The universe became vacuum-dominated a few billion years ago and will be increasingly so for ever, and the era in which the two are comparable is a small fraction of its history. A constant has turned out to be within a factor of two of a quantity that has fallen by more than a hundred decades since the beginning, and to be so now.

A hundredfold larger and the crossing would be at a redshift of five, before most galaxies formed. A hundredfold smaller and it would be in the distant future and unobservable. Either way, the coincidence would not be there to notice.

That is the coincidence problem, it is separate from the magnitude problem, and it is the one that makes a selection argument tempting: if a constant this small is necessary for observers to exist, and if a constant much smaller is no more likely than one this size, then observers find themselves at the crossing because that is when observers can be. Whether that is an explanation or a description of the difficulty is not agreed, and it depends on whether there is any ensemble for the selection to act on.

The units, and the one that is not tiny. The five Planck quantities and something familiar beside each. the Planck length: 1.616e-35 m; the Planck time: 5.391e-44 s; the Planck mass: 2.176e-8 kg; the Planck energy: 1.956e+9 J; the Planck temperature: 1.417e+32 K. Four of them are absurd — a length twenty decades below a proton, a temperature thirty-two decades above the Sun's centre. The mass is not: at 21.8 nanograms it is a visible speck, and the Planck energy is the chemical energy of about 57 litres of petrol — a tankful, computed here rather than repeated. That is not a coincidence to be explained away; it is the statement that gravity is weak. A Planck mass is the mass at which a particle's Compton wavelength and its Schwarzschild radius coincide, and gravity being weak means that happens at a mass enormously larger than any particle's.
Fig. 4 For scale: the five Planck quantities with something familiar beside each, from the Planck-unit argument. Four of them are absurd and the mass is twenty-two micrograms. The vacuum energy density this essay is about is the Planck density — the Planck mass in a Planck volume — and it is 10¹²¹ times what is observed, which makes it the one Planck quantity anybody has ever been in a position to check.

The Planck density is the least intuitive member of that family and it is the one the estimate returns. It is the Planck mass in the cube of the Planck length — twenty-two micrograms in a volume of 10−10510^{-105} cubic metres — and the reason the combination appears at all is the crossing below.

The two lengths every mass has. The Compton wavelength and the Schwarzschild radius of the same mass, against mass, on logarithmic axes. One falls and the other rises, so they cross exactly once — here at 1.539·10⁻⁸ kg and 2.286·10⁻³⁵ m, found by bisecting the difference rather than by writing down √(ħG/c³). The conventional Planck values are 2.176·10⁻⁸ kg and 1.616·10⁻³⁵ m; the crossing sits a factor of 1.414 away from them, which is exactly √2 and is the factor of two in the Schwarzschild radius coming through a square root. That is the whole precision this argument has, and it is worth saying, because a number written to four figures invites a reader to believe the definition is doing more work than it is. Nothing in physics is known at that length.
Fig. 5 And where that density comes from: the two lengths every mass carries, one falling with the mass and one rising, crossing exactly once. The scale in the hero figure’s cutoff is the energy at that crossing, and the density is that energy in the volume the crossing’s length encloses. Nothing in the construction is a choice — there is one combination of the three constants with the dimensions of a length, and it is this one.

The same shape of failure, one level down

It is worth setting this failure beside the one it most resembles, because the comparison is what makes the vacuum problem unusual rather than merely large.

The Higgs boson’s mass has the same structural difficulty. Quantum corrections to it grow with the cutoff — as the square rather than the fourth power — so a cutoff at the Planck energy requires a cancellation to about thirty-four decimal places to leave the 125 GeV that is measured. That is the hierarchy problem, it is the main motivation for supersymmetry and for models in which the true scale of gravity is much lower, and it is taken extremely seriously.

The vacuum energy’s cancellation is to a hundred and twenty places rather than thirty-four, and it has attracted far fewer proposed solutions. The reason is instructive: the Higgs mass is a parameter of a quantum field theory, where there are known mechanisms — symmetries, compositeness — for protecting a mass from corrections. The vacuum energy is a parameter that couples to gravity, and there is no quantum field theory of gravity within which to look for such a mechanism.

So the two problems are the same shape and different subjects, and the one with the smaller number is the one with the tools. That is a fair summary of why one has a literature of proposed solutions and the other has a literature of impossibility arguments.

A cutoff is not a calculation

The zero-point sum is not a calculation, it is a cutoff regularisation. In a proper treatment the divergence is absorbed into a renormalised constant, and the renormalised value is not predicted by anything — it is an input, fixed by measurement, exactly as an electron’s mass is. That is the technically correct statement and it is also a restatement of the problem rather than a resolution of it: the constant is then a free parameter that happens to be 10−12010^{-120} of the natural scale, and no other renormalised parameter in physics is so far from its natural value.

The estimate is for one massless field. A real calculation sums over every field in the standard model with signs — bosons positive, fermions negative — and the sum is not obviously dominated by any one term. It is also not obviously smaller, and nothing arranges the signs to cancel.

The measured quantity is a fitted parameter of a cosmological model. What is observed is the expansion history, the microwave background and the distribution of galaxies; the dark-energy density is what those require within the standard model of cosmology. A different model of gravity on large scales would attribute the same observations to something else, and several have been proposed. None of them removes the estimate’s problem, which exists whether or not the vacuum energy is what is accelerating anything.

The Casimir comparison is about differences, not totals. The measured force confirms that the zero-point energy changes when the boundary conditions do, which is all any experiment can show — a force that lives where the model is not has no access to the absolute value. So the vacuum’s energy is established as real and its magnitude is established by nothing.

And the anthropic bound is a bound on one parameter with the others held fixed. Varying several at once opens regions of parameter space where structure forms with quite different constants, which weakens the argument considerably, and how much is a matter of active dispute.

A curve drawn against a quantity that does not exist

The first figure draws a density against a cutoff as though the cutoff were a physical quantity, and it is not. There is no wavenumber at which the modes stop; the cutoff is a placeholder for wherever the description used stops being valid, and what a correct theory would do there is exactly what is not known. A line drawn against it is a family of guesses rather than a prediction.

The contributions figure draws four bars and cannot show their signs. Fermionic zero-point energies are negative, the Higgs potential’s minimum can be either sign depending on the convention chosen for its zero, and the condensates contribute with a sign that requires care. What has to cancel is a signed sum, and a figure of magnitudes shows the difficulty of the cancellation while hiding the possibility that it might be less arbitrary than it looks.

And the history figure draws three smooth curves and hides that one of them is not measured the way the others are. Matter’s density is measured many ways; radiation’s is measured from the microwave background’s temperature; the vacuum’s is inferred from the expansion’s acceleration, which is a second derivative of a distance measurement and is therefore the most delicate of the three. The line’s flatness is an assumption being tested rather than an observation.

Still open: everything, and one thing worth saying about how

This is the point at which the honest answer is that nobody knows, and the useful thing to add is what kind of not-knowing it is.

The problem has a shape that is unusual. Most open problems in physics are places where a calculation cannot be done or a measurement cannot be made. This one is a place where the calculation can be done, gives an answer, and the answer is wrong by more than any other estimate in the subject — using only physics that is separately confirmed. That is not a gap; it is a contradiction, and it has stood since Zel’dovich pointed it out in 1967 with no serious progress.

The candidate resolutions divide by what they give up. A symmetry that enforces the cancellation gives up nothing and has not been found. A dynamical field that relaxes towards zero gives up the constancy, and is constrained by the equation-of-state measurement. A selection effect across a landscape of vacua gives up the idea that the constant has an explanation at all, and requires an ensemble whose existence is not testable. A modification of how gravity responds to a vacuum energy gives up general relativity’s equivalence of all forms of energy, which is the most heavily tested statement in the theory.

Each of those is a large price, and the fact that the cheapest available option is still expensive is the actual content of the problem.

The habit worth carrying away is the one the hero figure’s slope is about. When an estimate is wrong, check how the error depends on the assumption it is blamed on. The vacuum energy is routinely dismissed as an artefact of a high cutoff, and a fourth power settles that in one line: the assumption cannot be adjusted far enough, and the failure survives every retreat to physics that is certain. An error that gets smaller as the questionable assumption is weakened is an artefact. One that does not is a result.

Part 3 of 4

This essay is one argument about Planck scale. The others:

Two terms, and the distance neither can beat. The smallest distance a probe of a given momentum can resolve, as the sum of two terms. The falling one is the uncertainty relation: more momentum, shorter wavelength, finer resolution. The rising one is gravity: the probe's own energy curves the region it is probing, and past a point it makes a horizon larger than the thing being looked at. With the gravitational term at 1× the Planck area, the least resolvable distance is 2.286e-35 m — each located by scanning the drawn curve over four hundred thousand momenta rather than by substituting into a formula. There is no momentum at which the resolution is better than that, so the ordinary procedure for measuring a distance has a floor, and the floor is the Planck length up to a factor of order one. The length no experiment can resolve Part 2 — Measuring a small distance needs a short wavelength, a short wavelength needs a large energy, and a large energy in a small region makes a horizon. Past a point, pushing harder makes the probe bigger — and the distance where that turns round is the Planck length. How large the hidden dimensions would have to be. The size extra dimensions would need, for gravity's true scale to be at a TeV rather than at 10¹⁹ GeV, against how many of them there are — a logarithmic axis of metres, for three choices of the true scale. The relation is the one that makes the arrangement work: the Planck mass observed in four dimensions is M_*^(2+n)Rⁿ, so gravity is weak because its field spreads into a volume nothing else can enter. Solved for R at a true scale of a TeV: 1 dimension needs 2.9e+13 m, 2 dimensions needs 2.4e-3 m, 3 dimensions needs 1.0e-8 m, 4 dimensions needs 2.2e-11 m, 5 dimensions needs 5.4e-13 m, 6 dimensions needs 4.5e-14 m. The two horizontal lines are where experiment has been. One extra dimension would have to be of order a hundred astronomical units, which would have wrecked the orbits of the planets and is excluded absolutely. Two would have to be of order a millimetre — which is what made the arrangement famous, because a millimetre is a distance a laboratory can test, and torsion balances have since verified the inverse-square law down to fifty-two micrometres and excluded it. Three or more sit below a nanometre, where no gravitational measurement reaches, and are untouched. The scale that may not be where it looks Part 4 — Every Planck number assumes gravity is four-dimensional all the way down. If it is not — if the field spreads into dimensions compact enough to have escaped notice — the true scale where gravity becomes strong could be at a TeV, and the whole remoteness of the Planck scale would be an artefact of where the field lines go. It is the one part of the subject an experiment can address, and the experiments have addressed it.

Where this goes next

Two terms, and the distance neither can beat. The smallest distance a probe of a given momentum can resolve, as the sum of two terms. The falling one is the uncertainty relation: more momentum, shorter wavelength, finer resolution. The rising one is gravity: the probe's own energy curves the region it is probing, and past a point it makes a horizon larger than the thing being looked at. With the gravitational term at 1× the Planck area, the least resolvable distance is 2.286e-35 m — each located by scanning the drawn curve over four hundred thousand momenta rather than by substituting into a formula. There is no momentum at which the resolution is better than that, so the ordinary procedure for measuring a distance has a floor, and the floor is the Planck length up to a factor of order one. The length no experiment can resolve Measuring a small distance needs a short wavelength, a short wavelength needs a large energy, and a large energy in a small region makes a horizon. Past a point, pushing harder makes the probe bigger — and the distance where that turns round is the Planck length. The two lengths every mass has. The Compton wavelength and the Schwarzschild radius of the same mass, against mass, on logarithmic axes. One falls and the other rises, so they cross exactly once — here at 1.539·10⁻⁸ kg and 2.286·10⁻³⁵ m, found by bisecting the difference rather than by writing down √(ħG/c³). The conventional Planck values are 2.176·10⁻⁸ kg and 1.616·10⁻³⁵ m; the crossing sits a factor of 1.414 away from them, which is exactly √2 and is the factor of two in the Schwarzschild radius coming through a square root. That is the whole precision this argument has, and it is worth saying, because a number written to four figures invites a reader to believe the definition is doing more work than it is. Nothing in physics is known at that length. Where every model runs out at once Every mass carries two lengths — one below which quantum mechanics will not let it be located, one below which gravity will not let anything escape. One falls with mass and the other rises, so they cross exactly once, at a length nothing in physics has ever probed. The only part of the field that can pull. A dielectric slab part-way into a parallel-plate capacitor. Everywhere except at the slab's edge the field is perpendicular to the plates and therefore perpendicular to the direction the slab can move, so it exerts no force along that direction at all: inside the parallel-plate model, which is uniform between the plates and zero outside them, nothing pulls the slab anywhere. The force lives in the bowed lines drawn at the edge, where the field leaks past the end of the dielectric and acquires a component along the plates. Those lines are what the model throws away as a small correction near the boundary, and they are the entire mechanism. The energy method sidesteps the drawing altogether: differentiate the total energy with respect to the insertion and the answer is 1.328e-3 newtons, inward, without ever asking where on the slab the force is applied. The force that lives where the model is not A slab of glass held at the mouth of a charged capacitor is pulled in. Inside the parallel-plate model there is no force at all — the field is perpendicular to the slab's motion everywhere — and the same model's energy nevertheless gives the pull exactly right. The mechanism is entirely in the part of the field the model throws away.

The objects named here

The third axis, after the field and the reading path: the things themselves, and every essay that touches each one.

BlackbodyCosmological constantDimensional analysisEnergyFine tuningPlanck scaleQuantum gravityThe second lawVacuum energyZero-point energy