Shannon, in his seminal work (reprinted in [1]), unleashed a genie who refuses to return to the bottle. Shannon’s original work is a paradigm of simplicity. Shannon’s theorem provided an elegant mathematical formulation to the problem, how to design complex communication networks, at a most auspicious time. Telecommunications networks were growing geometrically during the decades that followed the Second World War. America was being transformed into the Global Village: a place where your fingers could do the walking to anywhere in the world.
The section under review consists of the following papers:
The Wider Scope of Information Theory, by Donald M. MacKay.
Information Theory in Psychology, by George A. Miller.
Entropy and the Measure of Information, by Peter Elias.
The Entropy Function in Complex Systems, by Elliott W. Montroll.
Entropy, Probability, and Communication, by Myron Tribus.
These writers have addressed entropy from a perspective that, albeit interesting, useful, and often elegant, fails to do justice to the elegance of Shannon’s original equation. Shannon, a master craftsman in the application of Occam’s Razor, removed the unnecessary material and left the pearl. I recommend that anyone interested in information theory and especially its evolution and application read the papers being reviewed. They are well written and, given the technical nature of the material, interesting. I was especially fascinated by the application of “entropy” to other fields of human knowledge. I found Montroll’s application of entropy as a measure of the Sears Roebuck Catalog to be both fascinating and quite unexpected.
Miller perhaps sums up the world’s attitude to Shannon’s classic theorem by stating that “. . . ideas first introduced . . . through information theory have now become foundational assumptions that everyone takes for granted, their historical origins now irrelevant to their present role.” I believe this quote sums up Shannon’s place in history.
