The Second Law of Thermodynamics and Entropy

Beginner

Some things happen by themselves in only one direction.

Step 1: heat goes downhill. Leave a hot mug of cocoa on the table and it cools until it matches the room; the room never pushes heat back into the mug to make it hotter. Energy passes from the faster-jiggling molecules of the hot object to the slower ones of the cold object until both jiggle the same on average.

Step 2: things spread out. Spray perfume in one corner and soon the whole room smells of it. The molecules wander about at random, and they never gather back into the bottle by themselves.

Step 3: it is a matter of counting. Toss ten coins. Getting all ten heads can happen in only one way, but getting five heads and five tails can happen in 252 different ways. Molecules are like trillions of coins: tidy arrangements (all in one corner) are rare, spread-out ones are overwhelmingly common, so random motion lands in them and stays there.

Step 4: entropy. Physicists measure how spread out energy and matter are with a quantity called entropy. For a system left completely alone, entropy can only grow or stay the same. This is the second law of thermodynamics, and it gives time a direction: a film of a mug warming up by itself looks wrong.

The same law is why no engine can turn all its heat into work, and why a fridge needs a motor.

Check yourself: you drop an ice cube into warm lemonade. Which way does heat flow? Could it ever reverse by itself? (From the lemonade into the ice; no.)

In the animation, a wall keeps 20 molecules in the left half of a box. When it is removed they spread out until the bars show about 10 on each side, and they practically never all crowd back to the left.

Intermediate

Two statements of one law

Energy conservation alone would let a hot mug grow hotter by drawing heat from the cooler room. That never happens. The second law of thermodynamics picks out which energy-conserving processes really occur, in two classic statements:

• Clausius: heat never flows by itself from a colder body to a hotter one.
• Kelvin–Planck: no engine working in a cycle can take heat from a single reservoir and turn all of it into work.

The two are equivalent. If heat could leak by itself from cold to hot, an engine's waste heat $Q_c$ could leak straight back to the hot reservoir, and heat from that reservoir alone would have become work; a perfect engine driving a refrigerator would break Clausius in turn. A refrigerator does move heat from cold to hot, but not by itself: a motor supplies work.

Entropy: heat divided by temperature

Entropy turns the law into arithmetic. When a body at a constant absolute temperature $T$ receives heat $Q$ reversibly (slowly, without friction or temperature jumps), its entropy $S$ changes by
$\Delta S=\frac{Q}{T}\text{(reversible heat, }T\text{ in K)}$
in joules per kelvin (J/K). Heat leaving a body is a negative $Q$. A reservoir is a body so large that its temperature does not change. The entropy of a reservoir changes by $Q/T$ however the heat arrives.

The second law then says that in an isolated system, which exchanges neither heat nor work with its surroundings, the total entropy never decreases:
$\Delta S_{\mathrm{total}}\ge 0\text{(isolated system)}$
The equal sign holds only for idealised reversible processes. Friction, mixing and heat crossing a temperature gap all give $\Delta S_{\mathrm{total}}>0$, so they cannot run backwards by themselves.

When heat $Q$ passes from a body at $T_h$ to one at $T_c$, the hot body loses $Q/T_h$ and the cold one gains $Q/T_c$:
$\Delta S=\frac{Q}{T_c}-\frac{Q}{T_h}>0\text{(heat }Q\text{ from hot to cold)}$
This is positive exactly when $T_c<T_h$, because a joule carries more entropy at a lower temperature: the Clausius statement in symbols.

Entropy as a count of arrangements

Why must the total grow? The answer is counting. A macrostate is what you see from outside, such as how many molecules are in each half of a box; a microstate is the full microscopic detail, such as which molecules are in which half. The number $W$ of microstates that look like one macrostate is its multiplicity, and
$S=k\ln W$
where $k=1.38\times 10^{-23}J/K$ is the Boltzmann constant.

With $n_L$ of the $N$ molecules on the left and $n_R=N-n_L$ on the right, the number of ways to choose which molecules are on the left is
$W=\frac{N!}{n_L!n_R!}$
Random motion visits every microstate equally often, so the gas spends nearly all its time near the macrostate with the largest $W$.

Worked example: the gas in the animation

The box holds $N=20$ molecules, all on the left behind a wall: one way only, so $W=1$ and $S=k\ln 1=0$. This $S$ counts only the left-or-right part of the entropy; the rest does not change, because the temperature does not.

At $t=1.8\mathrm{s}$ the wall disappears. One molecule across gives $W=20$; five across give $W=20!/(15!5!)=15504$ and $S=9.6k$; ten on each side give
$W=\frac{20!}{10!10!}=184756,S=k\ln 184756\approx 12.1k$
These are the values on the $W$ and $S$ readouts when the blue and green bars show 10 and 10. Once the gas is mixed, $n_L$ drifts between 7 and 12, so $S$ stays between 11.3 k and 12.1 k, never near zero.

Nothing pushes the gas: the gold $v$ arrows show three molecules flying straight and bouncing off the walls. Each molecule is equally likely to be in either half, so all $2^{20}=1048576$ assignments are equally likely, and all-on-the-left has probability $(1/2{)}^{20}\approx 10^{-6}$, the middle line of the table. For 2 molecules it would be 1 in 4; for $10^{23}$, zero for all practical purposes. Even ten and ten occurs only $184756/1048576\approx 18\%$ of the time, so the bars keep jiggling.

Worked example: a hotplate heating water

A hotplate at 600 K passes 1200 J of heat to a large pan of water at 300 K. Both keep their temperatures, so
$\Delta S_{\mathrm{plate}}=-\frac{1200}{600}=-2J/K,\Delta S_{\mathrm{water}}=+\frac{1200}{300}=+4J/K$
The total rises by 2 J/K. The reverse transfer would give $-2J/K$ and is forbidden: the Clausius statement in numbers.

The 2 J/K measures lost work. An ideal engine between the same temperatures could have turned $1-300/600=50\%$ of the heat, 600 J, into work and rejected the other 600 J to the water, which would then gain only $600/300=2J/K$, exactly what the hotplate lost. Leaking directly throws away $T_c\Delta S_{\mathrm{total}}=300\cdot 2=600\mathrm{J}$ of possible work. Each joule per kelvin created beside surroundings at $T$ wastes $T$ joules of work.

Heat pumps and refrigerators

A heat pump uses work $W_{\mathrm{in}}$ to take heat $Q_c$ from a cold place and deliver $Q_h=Q_c+W_{\mathrm{in}}$ to a warm one: a narrow heat arrow in from the cold side, a work arrow from the motor, a broad heat arrow out into the house. The warm side must gain at least the entropy the cold side loses, $Q_h/T_h\ge Q_c/T_c$, which caps the coefficient of performance:
$\mathrm{COP}=\frac{Q_h}{W_{\mathrm{in}}}\le \frac{T_h}{T_h-T_c}\text{(heat pump)}$
$\mathrm{COP}=\frac{Q_c}{W_{\mathrm{in}}}\le \frac{T_c}{T_h-T_c}\text{(refrigerator)}$

For a house at 20 °C and outdoor air at 0 °C, $T_h=293.15\mathrm{K}$ and $T_h-T_c=20\mathrm{K}$: at most $293.15/20\approx 14.7$ joules of heat per joule of electricity. A refrigerator across the same gap could remove at most $273.15/20\approx 13.7$ joules per joule. Real machines reach 3 to 4, which is still three to four times the heat an electric heater gives. On a −20 °C day the limit falls to $293.15/40\approx 7.3$.

Common mistakes

• Entropy can fall locally. A kilogram of water freezing at 0 °C loses $334000/273.15\approx 1223J/K$, so the freezer must release at least $293.15\cdot 1223\approx 358\mathrm{kJ}$ into the kitchen at 20 °C, at least 24 kJ of it from its motor.
• Temperatures must be in kelvins: dividing 1200 J by 27 instead of 300 for water at 27 °C gives 44 J/K instead of 4 J/K.
• The body that gives heat loses entropy: in the hotplate example the total is $-2+4=+2J/K$, not 6 J/K.
• A COP above 1 creates no energy: the ideal heat pump delivers 14.7 J per joule of work by moving 13.7 J that were already outdoors.
• Entropy is not visible untidiness: shuffling a sorted 52-card deck adds at most $k\ln 52!\approx 2\times 10^{-21}J/K$, while warming the 100 g deck by 1 K adds about 0.4 J/K.

Where you meet it

Power stations are heat engines bound by the second law: coal and nuclear plants turn only 33–45 % of their heat into electricity. Fridges, air conditioners and heat pumps run the same cycle backwards. The Earth absorbs sunlight emitted at about 5800 K and returns the same energy to space as infrared at about 255 K, exporting some 20 times more entropy than it receives; that outflow drives weather and life.

The Carnot efficiency comes from First Law of Thermodynamics and Heat Engines, and the speed distribution of Kinetic Theory of Gases is the one with the greatest entropy at a given energy. Counting ways to share energy with $S=k\ln W$ gave the first quantum formula, in Blackbody Radiation and the Quantum.

Advanced

From Carnot's postulate to entropy

This section builds entropy from a single experimental fact, so that $S$ is not merely defined but shown to exist. The fact is Carnot's postulate in its modern form: no cycle can take heat at one temperature and turn it into work with no other change. The lesson First Law of Thermodynamics and Heat Engines drew two consequences from it. All reversible engines working between the same two temperatures have the same efficiency, and that common efficiency defines the absolute temperature $T$. A reversible engine that takes heat $Q_h$ at $T_h$ and rejects $Q_c$ at $T_c$ therefore obeys
$\frac{Q_h}{T_h}=\frac{Q_c}{T_c}$
Read this as a conservation rule: a reversible engine passes the quantity $Q/T$ from the hot reservoir to the cold one unchanged. Entropy is that quantity, made into a property of the working substance.

Legend

$p$	Pressure of the gas.	$\mathrm{Pa}$
$V$	Volume of the gas.	${\mathrm{m}}^3$
$T_1$	Temperature at the hot end of the strip.	$\mathrm{K}$
$T_2$	Temperature at the cold end of the strip.	$\mathrm{K}$
$\delta Q_1$	Heat taken in along the hot end, at $T_1$.	$\mathrm{J}$
$\delta Q_2$	Heat given out along the cold end, at $T_2$.	$\mathrm{J}$
$dS$	Step in entropy between the two adiabats that bound the strip: $\delta Q_1/T_1$, which equals $\delta Q_2/T_2$.	$J/K$

Figure 1 extends the rule from a two-temperature engine to any reversible cycle, drawn as an arbitrary closed loop on the $p$–$V$ diagram. The trick is to slice the loop with adiabats, the curves along which a gas changes state without exchanging heat; the dashed curves are a few of a dense family. Look at the purple strip between two neighbouring adiabats, drawn far wider than it really needs to be. Its long sides are adiabats, so no heat crosses them. Its short ends are tiny pieces of the loop: the hot end at the upper left, at temperature $T_1$, and the cold end at the lower right, at $T_2$. Each end is so short that its temperature hardly changes along it, so it acts as an isotherm.

Two adiabats and two isotherms make a Carnot cycle, so the strip is a small Carnot engine: it takes in $\delta Q_1$ at $T_1$, gives out $\delta Q_2$ at $T_2$, and by the rule above
$\frac{\delta Q_1}{T_1}=\frac{\delta Q_2}{T_2}=dS$
This common value, marked $dS$ inside the strip, depends only on which two adiabats bound the strip, not on where the loop happens to cross them.

Now add up all the strips. Together they trace the loop: each inner adiabat is run once in each direction by the two strips that share it, so those legs cancel, and the ends join up into the outline of the loop. Counting heat as positive when it enters the gas, every strip contributes $\delta Q_1/T_1-\delta Q_2/T_2=0$, so for the whole loop
$\oint \frac{\delta Q_{\mathrm{rev}}}{T}=0$

A quantity whose integral round every closed path vanishes has the same integral along every path between two given states. That integral is therefore the change of a function of state, the entropy $S$, with
$dS=\frac{\delta Q_{\mathrm{rev}}}{T}$
and the adiabats are the curves of constant $S$. Heat itself is not a function of state, because the heat a gas absorbs between two states depends on the path. Dividing by $T$ repairs this: $1/T$ is the integrating factor that turns $\delta Q_{\mathrm{rev}}$ into the exact differential $dS$.

What changes when a process is irreversible? An engine with friction, or with heat leaking across a finite temperature difference, delivers less work than a reversible engine from the same $Q_h$. By energy conservation it then rejects more heat, so $Q_h/T_h-Q_c/T_c<0$. For a general cycle, imagine every heat $\delta Q$ the system receives being supplied, at the temperature $T$ where it enters, by a small reversible engine fed from one large reservoir at $T_0$. That engine draws $T_0\delta Q/T$ from the reservoir. Over a cycle everything returns to its starting state, so the net heat $T_0\oint \delta Q/T$ drawn from that single reservoir has all become work. Carnot's postulate forbids this to be positive, which gives the Clausius inequality
$\oint \frac{\delta Q}{T}\le 0$
Here $T$ is the temperature of the reservoir that supplies each $\delta Q$, since a system out of equilibrium need not have one temperature of its own; equality holds only for a reversible cycle.

Apply the inequality to a cycle that goes from state a to state b by any process and comes back reversibly. The reversible return contributes $-(S_b-S_a)$, so
$\Delta S=S_b-S_a\ge {\int }_a^b\frac{\delta Q}{T}$
An isolated system takes in no heat at all, so for it $\Delta S\ge 0$: its entropy can grow or stay the same, never fall. That is the second law in its most general form.

The Carnot limits as an entropy balance

Every limit on engines, heat pumps and refrigerators follows from one balance. Over a full cycle the working substance returns to its starting state, so its own entropy change is zero. Only the reservoirs' entropy changes, and the second law requires their total change to be zero or positive.

Legend

$S$	Entropy, along the horizontal axis.	$J/K$
$T$	Absolute temperature, along the vertical axis.	$\mathrm{K}$
$T_h$	Temperature of the hot reservoir.	$\mathrm{K}$
$T_c$	Temperature of the cold reservoir.	$\mathrm{K}$
$Q_h$	Heat taken from the hot reservoir: the area of the tall rectangle, of height $T_h$.	$\mathrm{J}$
$Q_c$	Heat given to the cold reservoir: the area of the lower rectangle, blue and purple together.	$\mathrm{J}$
$Q_h/T_h$	Entropy lost by the hot reservoir: the width of the tall rectangle.	$J/K$
$Q_c/T_c$	Entropy gained by the cold reservoir: the width of the lower rectangle.	$J/K$
$\Delta S$	Entropy created in total, $\Delta S_{\mathrm{total}}=Q_c/T_c-Q_h/T_h$: the width of the purple strip that sticks out.	$J/K$
$T_c\Delta S$	Area of the purple strip: the extra heat rejected, which is the work lost when the surroundings are at $T_c$.	$\mathrm{J}$

Figure 2 keeps this account on a $T$–$S$ diagram, temperature up and entropy across. The point of this choice is that heat exchanged at a fixed temperature $T$ moves an entropy $Q/T$, so it can be drawn as a rectangle of height $T$ and width $Q/T$, whose area is $Q$: heats become areas. The engine takes $Q_h$ from the hot reservoir at $T_h$, rejects $Q_c$ to the cold reservoir at $T_c$, and delivers the work $W_{\mathrm{out}}=Q_h-Q_c$.

Now read the figure. The tall rectangle has height $T_h$ and width $Q_h/T_h$, the entropy the hot reservoir loses, so its area is $Q_h$. The lower rectangle, blue and purple together, has height $T_c$ and width $Q_c/T_c$, the entropy the cold reservoir gains, so its area is $Q_c$. The total entropy change is
$\Delta S_{\mathrm{total}}=\frac{Q_c}{T_c}-\frac{Q_h}{T_h}\ge 0$
so the lower rectangle must be at least as wide as the tall one. A reversible engine makes the two exactly as wide, up to the dashed line. The real engine drawn overshoots: the purple strip that sticks out has width $\Delta S$, the entropy created, and area $T_c\Delta S$, the heat dumped into the cold reservoir beyond what a reversible engine would dump.

The work is what is left of $Q_h$ after $Q_c$ has been paid out. For a reversible engine it is the part of the tall rectangle above the level $T_c$, the fraction $1-T_c/T_h$ of $Q_h$, and no engine can do better. The engine drawn, with $T_h=2T_c$, also loses the purple area: its surplus width is half of $Q_h/T_h$, so the lost area $T_c\Delta S$ is a quarter of $Q_h$, and it delivers half the Carnot work. In symbols, $Q_c/T_c\ge Q_h/T_h$ means $Q_c\ge Q_h{T}_c/T_h$, hence
$\eta =\frac{W_{\mathrm{out}}}{Q_h}=1-\frac{Q_c}{Q_h}\le 1-\frac{T_c}{T_h}$

Run the cycle backwards and the work $W_{\mathrm{in}}$ pumps the heat $Q_c$ out of the cold side and delivers $Q_h=Q_c+W_{\mathrm{in}}$ to the hot side. Now the hot side gains entropy and the cold side loses it, and the same balance, $Q_h/T_h\ge Q_c/T_c$, caps the coefficients of performance of a heat pump and of a refrigerator:
${\mathrm{COP}}_{\mathrm{heat}}=\frac{Q_h}{W_{\mathrm{in}}}\le \frac{T_h}{T_h-T_c}$
${\mathrm{COP}}_{\mathrm{cool}}=\frac{Q_c}{W_{\mathrm{in}}}\le \frac{T_c}{T_h-T_c}$

Irreversibility therefore has a price in work. In a device that keeps running, the entropy created cannot pile up inside it; it must leave as heat into the surroundings at $T_0$, and that heat, $T_0\Delta S_{\mathrm{total}}$, is energy a reversible process would have delivered as work:
$W_{\mathrm{lost}}=T_0\Delta S_{\mathrm{total}}$
In Figure 2 the cold reservoir plays the surroundings, $T_0=T_c$, and the lost work is exactly the purple area. Engineers use this rule to find the waste in a power station: combustion, and the large drop in temperature from the flame to the boiler water, create most of the entropy, so they cost most of the work.

Free expansion: entropy without heat

Entropy can grow with no heat at all, and this case is the classic trap. The gas in the animation expands into a vacuum. It pushes against nothing, so it does no work, and no heat crosses the walls, so $\Delta U=0$. The energy of an ideal gas depends only on its temperature, so the temperature stays the same. Yet nobody ever sees the gas gather itself back into one half: the process is irreversible, and $S$ must rise. Writing $\Delta S=\int \delta Q/T=0$ is the mistake, because that integral gives $\Delta S$ only along a reversible path, and a free expansion is not one.

Legend

$p$	Pressure of the gas.	$\mathrm{Pa}$
$V$	Volume of the gas.	${\mathrm{m}}^3$
$V_1$	Volume in state 1, before the expansion.	${\mathrm{m}}^3$
$V_2$	Volume in state 2, after it.	${\mathrm{m}}^3$
$T$	Temperature of the gas, constant along the curve, which is the isotherm $p=nRT/V$.	$\mathrm{K}$
$dV$	Width of one thin strip: a small step in volume.	${\mathrm{m}}^3$
$pdV$	Area of the strip: the work done in that step, which equals the heat $\delta Q$ taken in.	$\mathrm{J}$
$Q$	Heat taken in over the whole path: the area under the curve, $nRT\ln (V_2/V_1)$.	$\mathrm{J}$

Because $S$ is a function of state, its change can be computed along any reversible path that joins the same two end states. Figure 3 draws the convenient one: a slow isothermal expansion against a piston, for an ideal gas at temperature $T$. The curve is the isotherm $p=nRT/V$, from state 1 at volume $V_1$ to state 2 at $V_2$. The free expansion itself cannot be drawn on this diagram: between its end states the gas is churning, out of equilibrium, and has no single pressure.

Look at the thin strip of width $dV$. Over that step the pressure is $p$, so the gas does the work $pdV$, the area of the strip. The temperature, and with it $U$, stays fixed, so by the first law the gas takes in exactly that much heat, $\delta Q=pdV$; the whole area under the curve is the total heat $Q$. Each strip adds $\delta Q/T=pdV/T$ to the entropy. Substituting $p=nRT/V$ makes $T$ cancel:
$\Delta S={\int }_{V_1}^{V_2}\frac{pdV}{T}={\int }_{V_1}^{V_2}\frac{nRdV}{V}=nR\ln \left(\frac{V_2}{V_1}\right)$
With $nR=Nk$, where $N$ is the number of molecules and $k$ is Boltzmann's constant,
$\Delta S=nR\ln \left(\frac{V_2}{V_1}\right)=Nk\ln \left(\frac{V_2}{V_1}\right)\text{(ideal gas, same }T\text{)}$
The free expansion has exactly this entropy change, although it took in no heat. Doubling the volume adds $k\ln 2$ per molecule: $R\ln 2\approx 5.76J/K$ for a mole, and $20k\ln 2\approx 1.9\times 10^{-22}J/K$ for the 20 molecules in the animation's box.

The same recipe handles a change of temperature. Along a reversible path the first law gives $\delta Q_{\mathrm{rev}}=dU+pdV=n{C}_{V}dT+nRTdV/V$, where $C_V$ is the molar heat capacity at constant volume. Dividing by $T$,
$dS=n{C}_V\frac{dT}{T}+nR\frac{dV}{V}$
so an ideal gas with constant $C_V$ has $S=n{C}_V\ln T+nR\ln V+\mathrm{const}$. For a liquid, whose volume hardly changes, only the first term matters: a mass $m$ with specific heat capacity $c$, heated from $T_1$ to $T_2$, gains $mc\ln (T_2/T_1)$. Warming 1 kg of water from 20 °C to 80 °C adds $4180\ln (353.15/293.15)\approx 778J/K$, with the temperatures in kelvin.

Counting microstates: the graph and the histogram

Thermodynamics says how much entropy changes; counting says why. A macrostate is what we observe, here how many molecules are in each half of the box. A microstate is the full detail: which molecule is where. Boltzmann's relation ties entropy to $W$, the number of microstates that look like the same macrostate:
$S=k\ln W$
Doubling the volume doubles the number of places open to each molecule, so $W$ grows by a factor $2^N$ and $\Delta S=k\ln 2^N=Nk\ln 2$, exactly the thermodynamic result. Requiring the two to agree is what fixes Boltzmann's constant, $k=R/N_A$.

Counting also explains why heat flows from hot to cold, the Clausius statement of the second law. At fixed volume $1/T=\partial S/\partial U$, so one joule changes $\ln W$ by $1/(kT)$. A joule leaving a hot body therefore lowers its $\ln W$ by less than the same joule raises the $\ln W$ of a cold body. The flow increases the total number of microstates, which is why it is overwhelmingly what happens.

Legend

$n_L$	Number of molecules in the left half of the box.
$W$	Number of microstates of the macrostate with $n_L$ molecules on the left: $N!/(n_L!n_R!)$, with $n_R=N-n_L$.
$\sigma$	Width of the peak: the standard deviation $\sqrt{N}/2$ of $n_L$. The arrow ends at $10+\sigma$.
$S/k$	Entropy of the macrostate in units of $k$, equal to $\ln W$.
$N\ln 2$	Value of $S/k$ for all $2^N$ microstates together: the dashed line.
$\frac{1}{2}\ln (\pi N/2)$	Gap between the top of the dome and $N\ln 2$, from Stirling's formula.

Figure 4 unpacks the two panels of the animation for $N=20$ molecules. Both parts of the figure share the horizontal axis $n_L$, the number of molecules in the left half. Start with the histogram at the bottom. The bar over each $n_L$ has height $W=N!/(n_L!n_R!)$, the number of ways to choose which molecules are on the left. It is 1 at both ends, since all-left and all-right can each happen in only one way, and it is largest, 184 756, at $n_L=10$. The peak is binomial, with standard deviation $\sigma =\sqrt{N}/2\approx 2.2$; the arrow runs from the centre to $10+\sigma$. Relative to the mean $N/2$ that width is $1/\sqrt{N}$, which is why the animation's blue dot stays near the top of the histogram yet keeps jiggling, between about 7 and 12.

The upper part plots the entropy of each of those macrostates, $S/k=\ln W$. Taking the logarithm flattens the sharp peak into a dome: zero at both ends, broad and flat at the top. The animation's $S(t)/k$ graph reads this dome as $n_L$ changes with time: zero until the wall is removed at 1.8 s, a climb lasting about 3 s, then a plateau near 12 that never comes back down.

Why does the plateau sit below the dashed line $N\ln 2\approx 13.9$? The dashed line is $\ln 2^N$, which counts all $2^N$ microstates of the gas, every split included; the top of the dome counts only those of the single likeliest split. The bracket in the figure marks the gap, and Stirling's formula, $\ln N!\approx N\ln N-N+\frac{1}{2}\ln (2\pi N)$, gives its size:
$\ln W_{\max }\approx N\ln 2-\frac{1}{2}\ln \left(\frac{\pi N}{2}\right)$
For $N=20$ this is $13.86-1.72=12.14$, against the exact 12.13. The gap grows only like $\ln N$, whereas $N\ln 2$ grows like $N$: for $10^{23}$ molecules the correction is about 27 against $7\times 10^{22}$, utterly negligible.

The dome also gives probabilities. Every microstate is equally likely, so a macrostate with $W$ microstates has probability $W/2^N=e^{-(Nk\ln 2-S)/k}$. Each $k$ of entropy below $Nk\ln 2$, the entropy of all $2^N$ microstates together, costs a factor $e$; the all-left state, with $S=0$, has probability $2^{-N}$. For the $2.7\times 10^{19}$ molecules in a cubic centimetre of air the relative width $1/\sqrt{N}$ is $2\times 10^{-10}$: the histogram becomes a spike, and the gas is never seen far from an even split.

Can thermal jiggling be harvested?

If molecules never stop jiggling, why not harvest that motion? Picture a tiny toothed wheel held by a spring-loaded pawl that lets it turn only forwards, on an axle that also carries vanes standing in a gas. Molecules strike the vanes from both sides, and the pawl seems to keep only the forward kicks. It fails. The pawl is just as small, so at the same temperature it is shaken too: now and then it lifts just as the wheel is kicked backwards, and on average the slips exactly cancel the steps. With the vanes hotter than the pawl the wheel does turn, but then the device is a heat engine working between two temperatures, bound by $1-T_c/T_h$.

Maxwell's demon fails more subtly. It is a being that opens a trapdoor between the two halves only for fast molecules, sorting hot from cold without doing work. To decide, it must measure and record each molecule's speed, and its memory fills up. To keep going it must erase old records, and erasing one bit at temperature $T$ costs at least $kT\ln 2$ of work, $2.9\times 10^{-21}\mathrm{J}$ at 300 K. This is Landauer's principle; the entropy the erasure creates is at least the entropy the sorting removed. Information is physical, and this cost has been measured with single colloidal beads.

Worked problem: three ways to double a volume

One mole of ideal gas at 300 K, in contact with a large reservoir at 300 K, doubles its volume. The gas always ends in the same state, at the same temperature with twice the volume, so its entropy change is always $\Delta S_{\mathrm{gas}}=R\ln 2=5.763J/K$. Since $\Delta U=0$, the heat drawn from the reservoir equals the work the gas does, and the reservoir loses the entropy $Q/T$. Compare three methods.

• Free expansion into vacuum, as in the animation: no work and no heat, so $\Delta S_{\mathrm{total}}=5.763J/K$.
• Sudden expansion against a constant outside pressure equal to the final pressure $p_1/2$, where $p_1$ and $V_1$ are the starting pressure and volume: the volume grows by $V_1$, so the work is $(p_1/2)V_1=\frac{1}{2}RT=1247\mathrm{J}$, the reservoir loses $1247/300=4.157J/K$, and $\Delta S_{\mathrm{total}}=5.763-4.157=1.606J/K$.
• Slow, reversible isothermal expansion: the work is $RT\ln 2=1729\mathrm{J}$, the reservoir loses $1729/300=5.763J/K$, and $\Delta S_{\mathrm{total}}=0$.

The work missing, compared with the reversible method, is $T_0\Delta S_{\mathrm{total}}$ each time: $300\cdot 5.763=1729\mathrm{J}$, all of it, for the free expansion, and $300\cdot 1.606=482\mathrm{J}$, exactly $1729-1247$, for the sudden one. The gas's own entropy change is the same in all three; the entropy change of everything together is not, and it measures the waste. For the 20 molecules of the animation the lost work at 300 K is $20kT\ln 2\approx 5.7\times 10^{-20}\mathrm{J}$.

Limits of validity and the arrow of time

The second law is statistical: effectively certain for very many particles, only a strong tendency for a few. In the box, all 20 molecules back on the left has probability $2^{-20}$ at any moment. Fluctuation theorems make this precise: over a short time, histories that destroy an entropy $\Delta S$ are $e^{\Delta S/k}$ times rarer than those that create it. Beads a few micrometres across, held in optical traps, show such entropy-consuming stretches; for everyday processes $\Delta S/k$ is $10^{20}$ or more, and they never happen.

Thermodynamic entropy belongs to equilibrium states. For a gas that is still spreading, the value depends on how coarse the description is, here halves only. Counting also needs quantum mechanics: phase-space cells have a size set by Planck's constant, and identical molecules must not be counted as different, or mixing two samples of the same gas would seem to create entropy (the Gibbs paradox). As $T\to 0$, $S$ tends to a constant, zero for a perfect crystal: the third law. Gravity reverses the everyday intuition: a self-gravitating gas raises its entropy by clumping into stars.

Finally, the arrow of time. Apart from a tiny asymmetry in some weak decays, the laws governing the collisions work equally well backwards: reverse every velocity and the gas would gather back on the left. Irreversibility comes from the starting point, a state of very low entropy from which almost every trajectory leads to higher $S$. Our world's starting point goes back to the early universe, whose matter was spread almost perfectly evenly, a very low-entropy state for gravitating matter.

Where this leads

In Blackbody Radiation and the Quantum, Planck derived his radiation law in 1900 by counting the ways to share energy quanta among oscillators with $S=k\ln W$; the formula is carved on Boltzmann's tombstone in Vienna. The latent heats of Phase Changes and Latent Heat are entropy changes in disguise: $L/T$ is 1223 J/K for each kilogram of ice that melts. A heat flow $P$ through a wall, as in Conduction, Convection and Radiation, creates entropy at the rate $P(1/T_c-1/T_h)$. Black holes, treated in Gravity as Curved Spacetime, carry an entropy proportional to the area of their horizon. And information in bits, $-\sum p_i{\log }_2{p}_i$, is the general entropy $S=-k\sum p_i\ln p_i$ in other units.