8th Black Board Day (BBD8)
Last Sunday (April 28th, 2013) was the 8th Black board day (BBD), which is a small informal workshop I organize every year. It started 8 years ago on my hero Kurt Gödel‘s 100th birthday. This year, I found out that April 30th (1916) is Claud Shannon‘s birthday so I decided the theme would be his information theory.
Andrew Tan: Holographic entanglement entropy
Andrew wanted to connect how space-time structure can be derived from holographic entanglement entropy, and furthermore to link it to graphical models such as the restricted Boltzmann machine. He gave overviews of quantum mechanics (deterministic linear dynamics of the quantum states), density matrix, von Neumann entropy, and entanglement entropy (entropy of a reduced density matrix, where we assume partial observation and marginalization over the rest). Then, he talked about the asymptotic behaviors of entropy for the ground state and critical regime, and introduced a parameterized form of Hamiltonian that gives rise to a specific dependence structure in space-time, and sketched what the dimension of boundary and area of the dependence structure are. Unfortunately, we did not have enough time to finish what he wanted to tell us (see Swingle 2012 for details).
Jonathan Pillow: Information Schminformation
Information theory is widely applied to neuroscience and sometimes to machine learning. Jonathan sympathized with Shannon’s note (1956) called “the bandwagon”, criticized the possible abuse/overselling of information theory. First, Jonathan focused on the derivation of a “universal” rate-distortion theory based on the “information bottleneck principle”. Then, he continued with his recent ideas in optimal neural codes under different Bayesian distortion functions. He showed a multiple-choice exam example where maximizing mutual information can be worse, and a linear neural coding example for different cost functions.
- C. E. Shannon. A Mathematical Theory of Communication. Bell System Technical Journal 27 (3): 379–423. 1948
- E. T. Jaynes. Information Theory and Statistical Mechanics. Physical Review Online Archive (Prola), Vol. 106, No. 4. (15 May 1957), pp. 620-630
- Brian Swingle. Entanglement renormalization and holography. Physical Review D, Vol. 86 (Sep 2012), 065007
- MIT Open CourseWear lectures: Statistical Mechanics I: Statistical Mechanics of Particles, Statistical Mechanics II: Statistical Physics of Fields (recommended by Andrew Tan)
- C. E. Shannon. The Bandwagon. IRE Transactions on Information Theory, 1956
- N. Tishby, F. Pereira, W. Bialek. The Information Bottleneck Method. In Proceedings of the 37-th Annual Allerton Conference on Communication, Control and Computing (1999), pp. 368-377