After many years of effort trying to understand entropy, with its ubiquitous nature but confusing explanations, I believe I am making some personal headway. In February 2013, I was contacted by a new friend I had met at the MathEd Forum at Fields Institute in Toronto, and asked to reconstruct some of the models developed by Dr Yakovenko and his students. David Talbot has a shared interest in persistent and ubiquitous probability distributions, and we had met a few times informally to discuss them. My interest is principally in connection with the distribution of wealth in a sustainable ModEco economy which I call the PMM. His interest is primarily in the distribution he refers to as "Burke's Wave", in reference to his uncle's passion for it, a class of distributions that includes the Gaussian or normal distribution. We had discovered that our interests overlapped when marveling at the simplicity of Yakovenko's capital exchange models, and the variety of distributions of wealth it could produce, and the similarity to real-world distributions. Yakovenko's models are able to produce Burke's Wave, my ModEco-based distribution (similar to the Maxwell-Boltzmann distribution of energy in an ideal gas), as well as the more commonly found power law distributions of wealth. So, early in 2013 David asked me if I could replicate those models. Due to our shared interest, I somewhat reluctantly set aside my other projects for a while, only to discover that a deeper study of Yakovenko's capital exchange models contained a missing key to my search for understanding of the phenomenon of entropy.
There have been many books, and many thousands of journal articles, written on the topic of entropy, in both of its well-known forms: thermodynamic entropy and informational entropy. Both topics are obtuse. There have been many, many mathematical articles written about distributions that are consistent with maximal entropy. Most are close to incomprehensible due to the very abstract reasoning and very exotic mathematical knowledge required to properly critique them.
So I, with my new-found insight, and with the arrogance of a man who "knows just enough to be dangerous" (Don't I wish!), have decided, somewhat arrogantly, to add to that confusing, obtuse and mountainous body of work by developing my own theory of entropy. :-)
On this page, and on the companion page called "Entropy in ABMs", I plan to make available the results of my musings on the topic.
Three files - one application. This is the EiLab application software. This zip file contains three files, two of which are dummy stubs, but which must be placed in the same directory as the other.
- EiLab.exe is the executable.
- EiLab.cnt is the content file for the help system, which is empty.
- EiLab.hlp is the help system, which is empty.
Place the zip file in a directory of your choice and unzip it there. I suggest 'My Documents/OrrerySW/EiLab/. Double-click on the EiLab.exe executable, and it will start.
This is a paper I presented at MABS 2013 in which I hypothesize that the Maximum Power Principle (MPP) of H.T.Odum and the Maximum Entropy Production Principle (MEPP), two proposals for the fourth law of thermodynamics, are two sides of one phenomenon, and that they are both operational in economies, and that they are both operational in agent-based models of economies. My work since then has been primarily an exploration of these hypotheses.
This is a formal description of Model I of EiLab using the ODD protocol. Model I is the model I used to start my exploration of entropy in ABMs that are closed (as opposed to open). You can consider this to be user documentation.
This unpublished ( and, as yet, unfinished) paper defines entropic index, and studies its behaviour in Model I of EiLab. I really like the chart on page 22 in which I can finally see the nature of the 'arrow of time' as driven by entropy.
Version 1.39 (Build 39), Alpha release, April 2017.
Capital Exchange Models A through H (Dragulescu and Yakovenko, 2000) are active, except for the bank model. These are grouped together and called the "Wealth Study", or WStudy. Model I (MEP study) is complete, and Model J (MEPP Study) is there but not really tested. These have a different behaviour and are grouped together and called the "Entropy Study", or EStudy. I have plans to write three or four papers expanding on the nature of entropic indices when applied to ABMs before I get into the MEPP study. Time will tell how that works out.
To download a software package with the most current release of EiLab, click here (2.5Mb).
This download package also includes a selection of documents. Unfortunately, at this point, I don't have user documentation, but the ODD model description included above will help you understand the workings of the model.
System Requirements: Tested on Windows 95, Windows XP, Windows 7. Not tested on Windows 8. Will not work on Windows Vista. Requires screen resolution of 1024 x 768 or higher.
Installation: This package of software does not install itself. Copy it to a suitable directory. I suggest you make a new directory in your "My Documents" directory called "OrrerySW\EiLab\" and place it there. Then use WinZip or a similar utility to decompress it in place. There are three files associated with the software, only one of which is functional, but the three are needed in place. They are EiLab.exe (the executable), EiLab.cnt (contents for the disabled help system), and EiLab.hlp (the disabled help system). Copy the exe file and place a short cut on your desk top.
Start-up and operation: First, read the paper by Dragulescu and Yakovenko. When done, double click on the EiLab.exe file, or on the desk-top icon, click once on the splash screen, and close the welcome screen. Use the Initiation Wizard (IWiz button on the toolbar) to choose a model and set parameters. There are buttons for causing the model to advance by 1 tick (1Tic), in slow mode (SloMo), in fast mode (Go) or to stop (Stop). There are also toolbar buttons to toggle the display areas: main (T/M), left (L), right (R), status (S) or notes (N). Finally, you should know that a wide variety of data output files can be generated using the menu command "Excel Export".
NOTE: EiLab is a work in progress!
Ok. I made that phrase up. :-)
I cannot claim to have expertise in entropy, but I do have opinions. If entropy in a closed or open system is S(t), then let Smax be the maximum amount of entropy that could be achieved by the system. Then we can define the entropic index I(t) as S(t) / Smax. Since the entropy of a closed system always rises towards a maximum determined by the invariant characteristics of the system, I(t) for an isolated system is always in the interval [0, 1]. We can also define the grade of the system as G(t) = 1 - I(t). For more details, see the documents available on page 'Entropy in ABMs'.
Wealth Study (WStudy)
The Wealth Study (or WStudy) - Based on the "capital exchange" model descriptions in the paper "Statistical Mechanics of Money" by Dragulescu and Yakovenko (2000), a modified version of seven out of eight of their models are implemented as models A through H in this application. Read their paper (you can download it and many others from Yakovenko's site or directly from archive). It will explain the models. There are some really cool insights into the nature of entropy hidden in these models.
WStudy - Model A
Model A - Is a variation on the first most basic capital exchange model described by Dragulescu and Yakovenko. This is a closed system. That is, it is closed with respect to money. A thousand agents each have $1000. They are randomly paired each tick, and, after the flip of a coin, the loser pays the winner $1.
- After 24,000 ticks, the majority of agents are very poor, and a small number are filthy rich, as seen in the wealth distribution histogram that looks like a line graph in the upper right area. The disparity in wealth is NOT due to greed, or skill, or experience, or luck. It is due to the "hidden hand" of entropy production.
- At the bottom you see the entropic index climbing close to 1 and staying there.
- To the left you see a distribution of entropic indices taken over the last 243 ticks. Note that, while the entropic index stays close to S-max, it actually has a probability distribution with a mode LESS THAN S-max and occasional perturbations well below S-max.
Entropy Study (EStudy)
The "Entropy Study" (or EStudy) - Having watched the fascinating demonstrations of rising entropy in the models and demonstrations found on Yakovenko's website, and spurred on by my desire to understand sustainable economics, I decided to implement a similar facility in my version of his models. The insight gained from that exercise, lead me to revise his models yet again, and study the production of entropy in more detail. The EStudy has two representative models, in this version: Models I and J.
Below you see the IWiz panel (Initiation Wizard) for setting the parameters for Model I.
- K is the number of allowed levels of wealth (allowed bins in the wealth distribution histogram).
- A is the number of economic agents. No agents come into or leave the economy.
- W is the total wealth in the closed system. No wealth (money) flows in or out.
Model I is extremely simple and extremely constrained. Both the number of agents (A) and the total money (W) are conserved in this model. And the allowed denominations of wealth are fixed. For example, if K=4 then the allowed wealth denominations are $1, $2, $3 and $4.
EStudy - Model I
Model I of the EStudy - Below you see that extremely simple capital exchange model, having four allowed levels of capital, and only eight agents. It is described in a formal ODD model description available in the download with the software. It is also is examined in detail in my unpublished paper in which I define an "Entropic Index" for use in agent-based models (ABMs). In the paper, I examine a dynamic system that has exactly 13 possible states in its state space, and compute changes in entropic index as the system moves from state to state. These documents are still in draft form.
I am having difficulty finishing some of my combinatorial mathematical formulae, which are noted as missing. If you know the missing formulae, and you would like to help me finish the paper and make it publishable, that would be GREAT. I think the paper is, nevertheless, still readable at this point. The really cool results are on pages 17 and following. The source of the "arrow of time" becomes clear, and the connection between entropy gradients and probability gradients in phase spaces also becomes clear.
In looking at the above picture, note that in the main panel, there is a histogram showing the number of agents (eight) who have in their possession amounts of $1, $2, $3 and $4. Time goes forward in discrete-time ticks. At each tick of the model, two agents are chosen randomly, and a coin is flipped, then the loser pays the winner $1 if he/she can. If the loser only has $1, or if the winner already has $4, no exchange is made on that tick of the model. These are arbitrary boundary conditions on the model. Such attempted transactions are disallowed. Below the main panel is a "Status" panel which can show a variety of debug and technical information. A variety of similar technical panels are also hidden to the right of the main panel. Below the status panel is a "Notes" panel in which, in this picture, is shown a line graph of a time series of measurements of the entropic index of the system. It hovers more-or-less near 1.0, but random fluctuations often take it far from its equilibrium position. To the left you see a distribution of entropic indices taken over the last many ticks. Note how the entropic indices seem to form a discrete spectrum of values, rather than a continuum of values. Compare this with the similar distribution chart for the more complex Model A system (above).
Model I - Probability of Visitation vs Configuration Visited
In this picture to the right, you see some detail for Model I. Using the "L" button on the toolbar, the distribution of entropic indices (in the panel on the Left of the EiLab screen) has been replaced with this distribution of counts by possible configuration. States are listed, left to right, in the order first visited in this run. 12 of the 13 states in the state space (or phase space) are here represented, one having not been visited yet. Note that the state that corresponds to thermodynamic equilibrium, in some sense, i.e. (2,2,2,2), is visited most often, about 33% of the time. This is the state with the highest entropic index.
Note, as described above, during some ticks, no exchange is made due to arbitrary boundary conditions (e.g. a loser cannot give away his last dollar). This results in "disallowed" transactions 39% of the time.
Note also that there is some hidden symmetry in this graph. For example, the configuration (1,2,5,0) and the configuration (0,5,2,1) both appear, and both have the same entropic index associated with them, and they are visited approximately the same number of times.
Model I - All Phase Spaces for given K and A
For such a simple model, one can use MS Excel to analyse expected behaviour. In the draft paper in which I define entropic index for ABMs, I describe in detail how I analysed Model I to compare actual behaviour with expected behaviour. Below is one of my diagrams. With four bins and eight agents, there are only 165 possible states. But conservation laws (of agents and money) partition that into 25 distinct phase spaces, each represented by a column of points. Two of these phase spaces are degenerate, consisting of only a single configuration (e.g. (8,0,0,0) ) in which all exchanges are disallowed.
I like this diagram because it has a (somewhat degenerate) fractal pattern. The implied enclosing curve recurs again and again within the pattern. As in nature, there is an upper and lower scale beyond which it is no longer a self-similar pattern. More complex models (e.g. more bins, more agents) increase the detail within the diagram. As mentioned above, each distinct phase space is represented by a vertical column of dots. For example, the seven dots in the middle column represent the 13 states of Model I as described and configured above. In that space there are:
- Six pairs of configurations, each pair sharing a common entropic index, which account for six dots [e.g.the pair (2,1,4,1) and (1,4,1,2)]; and
- The configuration (2,2,2,2), which is unique, having the highest entropic index.
Send comments and questions to Garvin H Boyle at email: [email protected]ogers.com
Last updated: February 2017