REVELATIONS:

Entropy and Information Explained    

With A New Mathematics of Relationships

 

 

By A. Thomas Rogovsky, Ben Calabria, Ruth Calabria & Peter Calabria.

© by Ruth Calabria, June 5, 2009. Contact: a.thomas.rogovsky@matrix-evolutions.com

 

 

For nearly two centuries the notion of entropy has intrigued and confused the mind of man. Stand in your backyard in Vermont in your underwear in the middle of winter and you will find your body rapidly losing heat to the cold surroundings. And if you stay out there long enough, your body will cool all the way down to the temperature of the cold air as you freeze to death

 

In a less dramatic example of heat flow and thermal equilibration, if you lay a penny just heated with a match atop an unheated penny, the heated penny will cool down and the cooler penny warm up until, if insulated, both pennies will reach a temperature between their hot and cool starting points.          

 

Experience consistently tells us that heat flows from a hot body to a cold body rather than vice versa. But why does it do that? The second law of thermodynamics says it’s because the entropy increases. But what is entropy? These are not easy questions to answer. To quote a prominent 19^th Century developer of this science: “Any method involving the notion of entropy, the very existence of which depends on the second law of thermodynamics, will doubtless seem to many far-fetched, and may repel beginners as obscure and difficult of comprehension.” Not too long after J. Willard Gibbs enunciated these difficulties understanding entropy in 1873, the noted Austrian physicist, Ludwig Boltzmann, tried to clarify entropy with the following equation for it.   

 

 (1)                                   

 

With the k term a constant, Boltzmann’s equation says that entropy, S, is a measure of the Ω (omega) possible states the energy in a system would spread itself over. The scientists of Boltzmann’s day, though, were not taken with the notion of a physical quantity like entropy being a function of probable states. As science folklore goes, Boltzmann was so deeply affected by their rejection of his ideas that he hung himself in 1906.

 

A few years after his suicide, his equation was accepted and enshrined on his tombstone in Vienna. But entropy yet remains a strange, almost mystical, concept, difficult to understand intuitively for the numerous scientists and engineers who have been exposed to it over the last 100 years. As a professor at Rensselaer Polytechnic Institute told this writer in a graduate course in thermodynamics a number of years back, “Don’t worry about entropy. Nobody under the age of 40 understands it.” And perhaps few over the age of 40, at least not well enough to explain it to those under the age of 40.  

 

The problem with entropy lies, though, not with some inherent complexity of it but with the faulty theory used to explain it. This is not to criticize Boltzmann’s genius for the breakthroughs he made in statistical mechanics over a hundred years ago. But he and his followers have been hampered in their efforts by the linear mathematics that science has had available to explain physical nature with. A strikingly clear picture of entropy emerges when entropy is explained with a new non-linear matrix mathematics we have developed. We call it REVELATIONS because it reveals so much about nature that is otherwise impossible to understand. This includes not just physical nature but also human nature, which is ultimately based on information, a concept closely akin mathematically to entropy.    

 

The foundation of standard mathematics is a count of individual units. In REVELATIONS what is counted most basically are dyadic relationships between two units of their distinction or sameness. It is easiest to present this new mathematics by weaving it into the clear explanation of entropy and the second law of thermodynamics it provides.       

 

A realistic thermodynamic system has on the order of a billion, trillion, trillion molecules. To make sense out of thermodynamic systems, we need to analyze them microscopically by starting with simple systems that have just a few molecules in them. One of the systems we will look at has N=3 gas molecules in it with K=12 units of energy distributed over those molecules so as to cause them to move about in a container colliding with each other. And another simple N=3 molecule system we shall look at has K=84 energy units as will make its molecules move faster and collide more often.  

                                                                 

                      

Figure 2. Two N=3 Molecule Systems, one with K=12 Energy Units, the Other with K=84 Energy Units.

 

 

Eventually we will bring these six molecules together in one container to mix and collide with each other in order to explain thermal equilibration, entropy and the second law. For now, though, we will focus just on the K=12, N=3 system. Assumed is that the K=12 discrete energy units in it are distributed over its N=3 molecules randomly. One of the way its K=12 energy units might distribute by chance is with one molecule getting x₁=7 energy units, another x₂=4 energy units and the last, x₃=1 energy unit, the (7, 4, 1) configuration of the system.

 

 

 

Figure 3. The K=12, N=3 System in the (7, 4, 1) Configuration.

 

 

The K=12 energy units are kinetic energy or energy of motion. This omits the rotation and vibration energy of molecules, a simplification that does not affect the analysis. The kinetic energy of a molecule of mass, m, and velocity, v, is (1/2)mv². We assume all molecules to have a unit mass of m=1 so that the x₁=7, x₂=4 and x₃=1 energy levels come about solely from variations in the velocity of the molecules. The (7, 4, 1) configuration is one way that the K=12 energy units distribute over the N=3 molecules. There are, in all, nineteen configurations for the system, nineteen ways that the K=12 energy units can be distributed over the N=3 molecules as listed below.   

 

 

                                                                             Table 4. The Configurations of the K=12, N=3 System.

 

 

The K=12, N=3, system is in one of these configurations at any moment in time. The configuration of the system changes over time because of collisions of the gas molecules during which energy units are transferred from one molecule to another and change the configuration. Some of the configurations will come about from these molecular collisions more often than others. The relative frequency of a configuration over time, how often it appears, also understandable as its probability of existence at any moment in time, is a function of the number of ways a configuration can be made as is calculated from the combinatorial formula shown below.

(5)                                                                                

 

 

This combinatorial assumes that the energy units are distinguishable from each other, a departure from Boltzmann statistical mechanics, which is based on the opposite, ad hoc, assumption that energy units are indistinguishable. What is developed here, hence, is a statistical mechanics based on a different starting assumption about the distinguishability of energy units.

 

In the above combinatorial, K is the number of energy units, N, the number of molecules and x_i, the number of energy units on the i^th molecule, i=1,2,…N. The n_j term is the number of molecules having x_i energy units, j=1,2,….K. An example computation of W is for the (10, 1, 1) configuration that has x₁=10, x₂=1, x₃=1, K=12, N=3, n₁=2, (2 molecules that have x_i=1 energy unit), n₁₀=1, (1 molecule that has x_i=10 energy units), and the other n_j=0, (0 molecules with the other possible energy levels.) Hence, the W number of ways for the (10, 1, 1) configuration is  

(6)                                                                              

 

 

To get some sense of what this W=396 ways of making the (10, 1, 1) configuration with K=12 distinguishable energy units means, think of K=12 balls distinguishable by color distributed into N=3 baskets. 

 

Figure 7. K=12 Distinguishable Balls to be Placed in N=3 Baskets.

 

 

Two ways these K=12 balls can be arranged in N=3 colored baskets are:

 

Figure 8. Two Ways that K=12 Distinguishable Balls can be Placed in N=3 Baskets .

 

 

It is easy to see how W=396 different (10, 1, 1) arrangements could be generated from these K=12 different colored balls. We shall talk about energy unit distinction, which is different than color distinction, later. For now we note that the W=396 ways may also be referred to as the W=396 states of the (10, 1, 1) configuration and we list the W number of ways for all the configurations of the K=12, N=3 system below.  

 

 

Table 9. The W Number of Ways of the Configurations of the K=12, N=3 System.

 

 

The K=12, N=3 thermodynamic system exists at any moment as one of these configurations, for example, (7, 4, 1). The configurations change from repeated molecular collision and the transfers of energy the collisions bring about. Note below the full set of all N^K=532,441 states of all the configurations as they appear on average over time and collectively constitute the K=12, N=3 thermodynamic system. The W symbol placed in the set is meant to indicate that there is a W number of each configuration, the sum of which collectively constitute the thermodynamic system.

 

(10)                           [W: (12, 0, 0); (11, 1, 0); (10, 2, 0); (10, 1, 1); (9, 3, 0); (9, 2, 1); (8, 4, 0); (8, 3, 1); (8, 2, 2);

                                  (7, 5, 0); (7, 4, 1); (7, 3, 2); (6, 6, 0); (6, 5, 1); (6, 4, 2); (6, 3, 3); (5, 5, 2), (5, 4, 3); (4, 4, 4)]

 

A configuration of N molecules bearing K energy units like (7, 4, 1) is a set of molecular energies that can be described in shorthand by the μ (mu), average molecular energy.  

 

(11)                                                                                              

 

For (7, 4, 1), this mean or arithmetic average is μ=K/N=12/3=4 energy units pre molecule.  The μ average succinctly and simply describes the N energies in the set with just the one value of the average energy. The set of configurations in Line 10 has N^K=531,441 configurations of various W number. Does it also have an average configuration of all N^K=531,441 configurations that can succinctly and simply describe the thermodynamic system that the configurations collectively comprise?  To answer that, we need to look at a property of a configuration like (7, 4, 1) that describes its distribution of energy, namely its σ² variance, a commonly used statistical measure of distribution that is the square of the familiar σ standard deviation.  

 

(12)                                                                                 

 

Every configuration has a σ² variance that specifies its distribution. For the N=3, K=12, μ=4, (7, 4, 1) configuration, the variance is σ²=(7²+4²+1²)/3−4²=6. The σ² variance measures of the energy distribution in the configurations of the N=3, K=12 system are listed below. 

 

 

Table 13. The σ² Variances of the Configurations of the K=12, N=3 System

 

 

To obtain an average configuration of the N^K=531,441 configurations in the configuration set of a thermodynamic system we must find an average of the σ² variances of all these configurations.

 

(8)                           [W: (12, 0, 0); (11, 1, 0); (10, 2, 0); (10, 1, 1); (9, 3, 0); (9, 2, 1); (8, 4, 0); (8, 3, 1); (8, 2, 2);

                                (7, 5, 0); (7, 4, 1); (7, 3, 2); (6, 6, 0); (6, 5, 1); (6, 4, 2); (6, 3, 3); (5, 5, 2), (5, 4, 3); (4, 4, 4)]

 

The average variance must be simply the sum of the σ² variances of the N^K=531,441 configuration states divided by N^K=531,441, which is also computable as the W/N^K weighted average of the σ² of the configurations as shown below.  

 

 

Table 14. The K=12, N=3 System σ² Variance as the W/N^K Weighted Average of its Configuration σ²

 

 

Because the set of configuration states collectively comprise the thermodynamic system, the σ² average variance of the configurations is the variance of the thermodynamic system. We can also obtain the σ² system variance in a faster way as the σ² variance of a random multinomial distribution, which is what a thermodynamic system is from the perspective of its random energy distribution. This derives from a general expression for the σ² variance of a multinomial distribution, (see Wikipedia.) 

 

(15)                                                                               σ² = P_iK(1–P_i)

 

In the above P_i is the probability that any one of K entities will be distributed to the i^th of N containers. For random or equiprobable distribution, as the thermodynamic distribution of energy units to molecules is, the P_i probabilities are all the same as P_i=1/N. For example, the probability that any one of the K=12 energy units in a thermodynamic system will be randomly distributed to any one of N=3 molecules in a system is P_i= 1/3=1/N. This simplifies the variance formula of Eq12 with 1/N substituted for P_i to obtain σ² for a random multinomial distribution as  

 

(16)                                                              

 

This determines the σ² variance of the of K=12, N=3, system as σ²=K(N−1)/N²=12(2)/3² =24/9=2.667, the very same value arrived at by the longer method of weighted averaging in Table14. Now we note that the σ²=2.667 variance of the thermodynamic system as obtained either from Eq16 or Table 14 is the very same value as the σ²=2.667 of the (6, 4, 2) configuration computed in Table 14. This (6, 4, 2) configuration is, hence, the average configuration of the set of configurations on the basis of its having the same σ² energy distribution measure as the thermodynamic system as a whole. The (6, 4, 2) average configuration, hence, succinctly and simply represents the entire thermodynamic system in all of its important characteristics, its K=12 number of energy units, its N=3 number of molecules and its σ² distribution of energy.   

 

To repeat for emphasis, much as the μ arithmetic average of a number set represents all the numbers in the number set in a simple and succinct way, so the average configuration or avcon simply and succinctly represents all the configurations of a thermodynamic system or the thermodynamic system as a whole. This is a new non-equilibrium way to represent a thermodynamic system as presents a perfectly clear picture of it. That the average configuration or avcon is it is a valid representation of a thermodynamic system is proven by its having the Maxwell Boltzmann energy distribution shape of realistic thermodynamic systems that have on the order of a billion, trillion, trillion molecules.

                                 

 

Figure 17. The Maxwell-Boltzmann Energy Distribution Curve.

 

 

The horizontal axis in the Maxwell-Boltzmann energy distribution represents the energy of a gas molecule and the vertical axis is the number of molecules that have that energy. To see if the (6, 4, 2) average configuration or avcon of the N=3, K=12, system has the Maxwell-Boltzmann distribution, we plot the number of energy units per molecule for the (6, 4, 2) repcon versus the number of molecules that have that number of energy units. We see in the N=3 molecule (6, 4, 2) avcon that represents the K=12, N=3, system that 1 molecule has 2 energy units, 1 molecule has 4 energy units and 1 molecule has 6 energy units. Plotting these points (and omitting the zero molecule, 3 and 5 energy levels, as data points in the plot because they don’t contribute to the energy distribution), we obtain.   

 

 

Figure 18. The Energy Distribution Curve of the Average Configuration or Avcon of the K=12, N = 3 System

 

 

Clearly this is not the Maxwell-Boltzmann distribution, so perhaps our using the avcon to represent the thermodynamic system was incorrect. But, as we shall see momentarily, the problem lies not in an incorrect analysis but in there being too few molecules in the K=12 energy unit, N=3 molecule, system to develop the Maxwell-Boltzmann curve. It is not strange that we need more molecules to generate the Maxwell-Boltzmann curve for recall that the realistic curve of it in Figure 16 is for a system with a billion, trillion, trillion molecules. This becomes clear when we consider a system that has a larger N number of molecules in it than N=3.

 

Eq16 obtains the σ² variance of the K=40, N=15 system as σ²=K(N−1)/N² =40(14)/15² =2.489. A Microsoft Excel program searches through all the configurations of the K=40, N=15 system and finds four configurations that have those that have the σ²=2.489 variance including (0, 1, 1, 1, 2, 2, 2, 3, 3, 3, 3, 4, 4, 5, 6), which is an avcon of the system on that basis. A plot of its number of energy units per molecule vs. the number of molecules with that energy is shown below.

 

 

Figure 19. The Energy Distribution Curve of the Repcon of the K=40, N = 15 System.

 

This curve bears a primitive resemblance to the Maxwell-Boltzmann curve of Figure 17. Next let us look at a system with more molecules in it, the K=145, N=30 system. From Eq16 we see that it has a σ²= K(N−1)/N² =145(29)/30² =4.672. There are nine configurations for this system that have σ²=4.672 as include (0, 0, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 6, 7, 7, 8, 8, 9, 10), an avcon on that basis. A plot of its number of energy units per molecule vs. the number of molecules is shown below.

 

 

Figure 20. The Energy Distribution Curve of the Repcon of the K=145, N = 30 System.

 

 

Albeit not smooth, this looks quite like the Maxwell-Boltzmann distribution of Figure 17. The progressively better resemblance of the avcon curves of increasing N and K to the Maxwell-Boltzmann of Figure 17 suggests that if we continued to increase the N and K values, we would obtain a progressively better fit to the realistic Maxwell Boltzmann of Figure 17. We need not just conjecture that or try to demonstrate it by generating curves for increasingly larger N and K systems, for it is a relatively simple matter to conclusively show analytically that the avcon has the Maxwell-Boltzmann distribution. We do that by showing that the avcon σ² variance has the same form as the σ² variance of a realistic Maxwell-Boltzmann distribution, which will show them to be the same distribution.

 

For very large N molecule systems that approach realistic thermodynamic systems, in Eq16, we see (1/N)à0 with the σ² variance of the system and avcon becoming    

 

(21)                                                                         

 

From accepted physico-chemical theory, (see Wikipedia), the σ² variance of the realistic Maxwell Boltzmann distribution is 

 

(22)                                                                            

 

In the above, k is Boltzmann’s constant, T, the Kelvin temperature, and m, the molecular mass. For systems with molecular mass assumed to be a unit mass, m=1, as we have assumed,

 

(23)                                                                         

 

The Kelvin temperature, T, of realistic thermodynamic systems is characterized in accepted physico-chemical theory as   

 

(24)                                                                           

 

This relationship is noted on http://hyperphysics.phy-astr.gsu.edu/Hbase/kinetic/molke.html#c1, the Hyperphysics blog, where it is made clear that it is the applicable μ energy average for the case of random distribution as generates the Maxwell-Boltzmann curve: “Note that the average kinetic energy for molecules, μ=(3/2)kT, is not the same as the average energy for purely random energies under the Boltzmann distribution, which is μ=kT.” Eq24 determines Eq23 as

 

(25)                                                                          σ² = μ

 

In the above being Eq21 as derived from our analysis of the avcon, this shows conclusively that the avcons do have the realistic Maxwell-Boltzmann distribution and, hence, are correct representations of thermodynamic systems.

 

The ramifications of this are significant for in obtaining such a clear picture of the origin of the Maxwell Boltzmann curve from the weighted average of the σ² variances of the configurations with energy units assumed distinguishable, which is direct contradiction to Boltzmann’s assumption of indistinguishable energy units, the foregoing casts doubt on the entirely of Boltzmann statistical mechanics, all of which is based on the indistinguishable energy units assumption.   

 

Of course, it would be possible to save the day for Boltzmann by generating the Maxwell-Boltzmann distribution by weighting the σ² variances of the configurations with the combinatorial expression used for indistinguishable energy units. Forethought tells us that is not likely to succeed because indistinguishable energy units are in firm logical contradiction to distinguishable energy units, so it would be odd, indeed, if both of these mutually contradictory assumptions produced correct results. We can try it, though, to make sure. The combinatorial for the number of ways with distributed entities indistinguishable is ω, (omega),

(26)                                                                                     

 

 

 

For the N=3, (10, 1, 1), configuration, which has n₁=2 and n₁₀ =1, the ω number of ways is

(27)                                                                      

 

 

 

The ω number of ways for all the configurations of the K=12, N=3 system are listed below.