Tags

So, this week’s topic is a big one.  It touches on how our communications systems work, why statistical mechanics makes sense, and even the ultimate fate of the universe.  The topic is entropy.  When I use the word “entropy”,  those of you out there who know of the term will have very different ideas of what I am talking about depending on what your field is.  All of these meanings, however, directly relate to each other.  So, in order to understand the relationship between the three types of entropy and how they relate to the physical world, I must first explain a few background items.

### Information, Counting, and Microstates

A 2 particle, 4 microstate system

When we look at a physical system, it is often composed of many little parts, each able to move through their configuration spaces along any of their
degrees of freedom. The configuration space is simply the collection of ways to describe the outcome of an event (For example, flipping a coin has a configuration space of 3 points: heads, tails, and edge).   Each point in this configuration space is called a microstate, and the collection of all of the parts and their microstates is called the system’s macrostate.  An example would be a pair of point particles, each capable of existing in one of two positions along a one-dimensional space.  Each of the two possible positions for each particle is a microstate (giving us a total of 4 microstates for the system).  The whole configuration of the two particles is the macrostate of the system. Note that we obtain the total number of microstates by multiplying the set of states for each particle, not adding them. I guess I should have used more than 2 particles with 2 states in my example. 😉

### Boltzmann Entropy

So, if we want to take a look at “microstatiness” of a system, what we want is the entropy. In this physical case, we are looking at the “Boltzmann Entropy”.  This value tells us something about the number of microstates, and the probability of falling into any particular microstate.  If we take the number of positions in the system’s configuration space Ω, then the total entropy of a system S is (apologies for the math)

S = kBln(Ω)
(Where kB is Boltzmann’s constant)

This is cool, because it allows us to add entropies, rather than multiply them when we combine two systems, because ln(AB) = ln(A)+ln(B).  Thus, we can look at any system composed of parts able to exist in different microstates, and come up with a term to describe the freedom of the system itself to jump around from microstate to microstate.  The entropy is a kind of window into the black box of the system.

### Von Neumann Entropy

If you extend the Boltzmann entropy to quantum systems, you get Von Neumann Entropy.  This encompasses the ability of quantum systems to become entangled, and thus reduce the ability for the system to flow from one microstate to another.  This type of entropy really isn’t much different, conceptually, from Boltzmann entropy, other than its inclusion of quantum features.