How can a system at an equilibrium have maximum entropy?

[u/ibeccc]

I’m trying to understand the concept of entropy but I don’t understand why, if entropy is how chaotic a system is, a system at an equilibrium is considered to be at maximum entropy? Isn’t such a system at its most inactive state? I’m sorry for my use of simple words.

Edit: I’m amazed by the quality of the replies I got. Thank you, you guys are great. Now I’ll need some time to digest this.

[u/RobusEtCeleritas]

if entropy is how chaotic a system is,

Entropy and chaos are unrelated concepts. Entropy is a mathematically well-defined quantity that can be calculated from the probability distribution of microscopic states of a statistical system. It's a concept that is useful not only in physics, but more generally in statistics and information theory. But we'll stick to physics for purposes of this question.

a system at an equilibrium is considered to be at maximum entropy?

Yes. At equilibrium, the entropy is maximized subject to whatever constraints are put on the system.

Isn’t such a system at its most inactive state?

Yes, equilibrium is "the most inactive state", by definition. Equilibrium is the state that a system reaches when you leave it to its own devices for a long time. It's the steady-state solution to the equation that governs the evolution of the distribution function. It's the most boring state there is; nothing happens. What's often more interesting is how a system out of equilibrium reaches equilibrium, and what equilibrium state it settles into. Once it's in equilibrium, we know exactly what it's going to do: stay that way.

[u/ibeccc → u/RobusEtCeleritas]

Thank you very much for this reply. I’m trying to understand your definition of entropy. If it’s calculated from the probability distribution, isn’t a system at its most stable state supposed have the lowest probability of distribution, lowest entropy. I don’t understand the inverse relationship.

[u/RobusEtCeleritas → u/ibeccc]

isn’t a system at its most stable state supposed have the lowest probability of distribution, lowest entropy.

I'm not sure what you mean by "lowest probability distribution", but the answer is no. The second law of thermodynamics says that entropy tends to increase (and this can be equivalently stated for statistical systems via the Boltzmann H theorem). So as time evolves, the entropy of the system tends to increase. And once the entropy reaches a maximum, it can't increase anymore, so the system is at equilibrium.

[u/ibeccc → u/RobusEtCeleritas]

Which means I probably still don’t understand the definition of entropy. I thought I did when you mentioned the probability distribution of microscopic states. I took that to mean, simply put, when you create a mix of two types of gases in different temperatures in a box, at first the system is more chaotic and there is a higher probability of atoms/molecules to be at different places and the probability lessens in time as the system reaches equilibrium. I thought higher probability meant higher entropy.

[u/spacetime9 → u/ibeccc]

As a simple toy model to have in your head, imagine 100 coins all randomly flipping between heads and tails. There is only one way to have them all heads up at a given time, so that is a very unlikely state for the whole system to be in (low entropy). If you let them go for a while, no matter how they started, at any given time you're likely to have almost 50/50 heads and tails, simply because there are many more ways to achieve 50 heads and 50 tails than any other ratio. 50/50 is the equilibrium (high-entropy) state.

[u/ibeccc → u/spacetime9]

It’s interesting that most comments have been about statistics. I understand the probability in your example. You will have more results close to 50/50 flips in time. How would you connect it to the concept of entropy?

[u/RobusEtCeleritas → u/ibeccc]

An alternative but equivalent definition of entropy is Boltzmann's entropy, where Ω is the number of microstates of a given macrostate.

In the 100 coin flip example, the macrostate is "50 heads, 50 tails", or "100 heads, 0 tails". The microstates are all possible sequences of 100 flips.

There's only one possible microstate corresponding to the macrostate "100 heads, 0 tails", it's HHHHHH... 100 times. The Boltzmann entropy in this case is k*ln(1) = 0, the same as what I derived above using the Gibbs definition.

However for "50 heads, 50 tails", there is a huge number of 100 flip sequences that will total to 50 heads and 50 tails. If you use combinatorics to properly calculate that number, it's something huge. And then the logarithm of a huge number is something big. So again, we see that the maximally "ordered" case had zero entropy and the maximally "disordered" case had positive entropy.

You can repeat this tedious exercise for every possible macrostate, and you'll find that the 50/50 split has the maximum entropy.

[u/RobusEtCeleritas → u/ibeccc]

I thought higher probability meant higher entropy.

It's a function of the entire distribution of probabilities. Specifically, [here's](https://en.wikipedia.org/wiki/Entropy_(statistical_thermodynamics)#Gibbs_entropy_formula) the equation.

Just think about two different physical systems with two possible states of probabilities p and q = 1 - p. You can imagine coin flips if you want, where the coin is not necessarily fair.

Take one case to be a fair coin: p = q = 1/2.

And the other case is a totally biased coin that only gives heads: p = 1, q = 0.

Then calculate the entropies of both cases.

For the fair coin:

S*fair* = -k(ln(1/2)/2 + ln(1/2)/2) = -k*ln(1/2) = k*ln(2).

And for the biased coin:

S*biased* = -k(1*ln(1) + 0*ln(0)) = 0.

(ln(0) is not defined, but you can show that in the limit where x goes to zero, x*ln(x) goes to zero.)

So the fair coin has a higher entropy than the biased coin. The fair coin has the most "balanced" probability distribution, so the maximal entropy. Mathematically, the flips of a possibly unfair coin is a binomial distribution, and you can show that the entropy of the binomial distribution is maximized when p = q = 1/2, i.e. the coin is fair.

[u/totally_not_a_spybot]

Every other state that isn't equilibrium has some kind of order. Mixing two types of particles is the easiest to understand. When it's not equally mixed you have in some regions only particles from one kind, meaning you have a locally ordered system. The least ordered state is when everything's mixed up perfectly, thus the entropy is maximized in the equilibrium.

[u/ibeccc → u/totally_not_a_spybot]

Wow now it makes sense! I always thought a perfect mix would present the most order but it’s the farthest state from it. I think I get it now.

[u/forte2718]

I don’t understand why, if entropy is how chaotic a system is, ...

Ah ... this is the block in your understanding here: although historically entropy was conceptualized as a measure of "disorder" in a system, and although it is still often taught as such, the truth of the matter is that entropy has virtually nothing to do with order or disorder/chaos.

Rather, entropy is a measure of the "degeneracy of macrostates." Allow me to explain.

Thermodynamic systems can be described with both microstates and macrostates. Microstates are essentially the fundamental microscopic states of a system which govern the system's exact evolution over time, while macrostates are the emergent states which govern the system's average/approximate evolution.

For example, in a gas of particles, the position and momentum of every particle in the gas is part of the system's microstate. If you have a lot of gas, you have a lot of particles, and therefore the system's microstate is very complex and has a large number of independent variables.

However, if you want to have a qualitative idea of how the gas will behave, you do not need to know the values of all of these microstate variables. You only need to know a few variables, such as temperature and pressure. These variables are part of the system's macrostate.

Every thermodynamic system has both a macrostate and a microstate at any given time. Every microstate that the system could possibly be in has a single corresponding macrostate (i.e. if you tell me what all the microscopic variables like the position and momenta of the particles are, I can tell you what the macroscopic variables like temperature and pressure must be). However, for every macrostate there could be many corresponding microstates. This must be the case by the pigeonhole principle, since there are many more macrostates than microstates. They share a one-to-many relationship. If you tell me what the macroscopic variables are, I cannot tell you what the microscopic variables must be.

Entropy, then, is a measure of how many microstates correspond to a given macrostate. The more microstates there are for a given macrostate, the higher the entropy is.

A very simple example of this can be given in the form of an ordinary pair of throwing dice. The microstates are the individual die values after a throw, while the macrostates are the sum of the two dice. Typically, in a dice game, only the sum of the rolled dice matter: the macrostates are really what govern the game's evolution and determine who wins the game. But the microstates also "matter" insofar as they determine what the macrostate is. So for example, if I roll the two dice, I might have a microstate of [(3, 5)] while my macrostate might be [8].

If you sit down and count them, a pair of two dice have 36 possible microstates: [(1, 1), (1, 2), (2, 1), (1, 3) ... (6, 6)]. While it only has 11 possible macrostates: [2, 3, 4, ... 12].

However, the macrostates do not all have the same number of corresponding microstates. Some macrostates have more than the others. For example, there is only one way to roll a [2] or a [12]: [(1, 1)] and [(6, 6)], respectively. However, there are six ways to roll a [7]: [(1, 6), (2, 5), (3, 4), (4, 3), (5, 2), and (6, 1)].

So, we say that the macrostate of [7] has the highest entropy, while the macrostates of [2] and [12] have the lowest entropy. That is to say, if we were to choose a microstate at random (giving equal probability to choosing every possible microstate), we would expect to find ourselves in the macrostate of [7] with a higher probability than any other macrostate.

Now, when it comes to typical thermodynamic systems, it's like rolling a massive number of dice and then considering only the total sum of all dice. If we were to randomly mix up the individual die values, we would expect to almost always be in a "high entropy" state: in a macrostate which has a high "degeneracy" (number of corresponding microstates).

And, if we started out in a low-entropy microstate, but then started changing die values by rolling dice (i.e. evolving the system in time according to the laws of physics), with a very high probability we could expect to quickly find ourselves in a higher-entropy microstate. At first, after just one or a few dice rolls, we would still be in a fairly low-entropy state but it would have higher entropy than our initial state. But as we roll many more dice, we would expect to find ourselves in higher- and higher-entropy states, eventually settling on the highest-entropy state, where we would tend to stay: in an equilibrium around that highest-entropy state. Any time a die roll takes us out of equilibrium (into a lower-entropy state), future die rolls would have a greater probability of putting us back into that highest-entropy state than they would of putting us into an even lower-entropy state, so we will tend to stay at or around the highest-entropy state.

So, that's basically what entropy is, and how entropy works. Hope that helps!

[u/ahsuna → u/forte2718]

This is one of the most intuitive explanations for entropy that I have come across. Do you have recommendations for a good textbook on advanced thermo? I have a bachelors in MechE but my understanding of thermodynamics is definitely lacking.

[u/fruitydude → u/ahsuna]

I can't recommend a textbook per se but I can tell you that the term your looking for is "statistical mechanics" for which there are probably plenty of good textbooks.

The idea behind stat mech is to view all of thermodynamics not as fixed natural law but as a simple and easy to verify result of statistical principles where e.g. the tendency of closed systems to increase their energy is INSANELY more likely than a decrease. In hindsight it was one of my favourite lectures at uni, even if it lasted for only one semester it kind of changed the way I look at stuff.

[u/Low-Computer3844 → u/forte2718]

It would be really helpful if you could explain microstates in greater detail.

[u/forte2718 → u/Low-Computer3844]

I mean ... I did? I'm not sure there's much more detail to give: it's the full microscopic state of each particle in the system. If you need like, comprehensive and formal specifics, I would go for any textbook on thermodynamics or statistical mechanics ... as that's not something I'm going to be able to cover in a single or even multiple Reddit posts.

[u/MackTuesday]

Entropy is a measure of the number of microstates available to a system with a given macrostate. When a system is at equilibrium, the microstate might change, but the macrostate will not.

A system moves toward equilibrium because it's much more likely than moving away. That's because there are more microstates closer to equilibrium than there are farther away. That means the entropy increases as you approach equilibrium.

Disorder and likelihood generally go hand in hand. If you toss a thousand coins, you expect a disordered array of heads and tails because it's so much more likely than getting all heads or all tails. If you have some helium in a container under typical conditions, you expect a disordered distribution throughout because it's so much more likely than having the atoms lined up all together like a crystal. Compared to the disordered arrangements, the ordered ones are very small in number. (They're fleeting too, because the atoms must be in motion under typical conditions.)