John Baez

Categorical Semantics of Entropy, CUNY Grad Center, May 11, 2022

Lisbon Webinar on Mathematics, Physics and Machine Learning, June 16, 2022

Shannon Entropy from Category Theory

Shannon entropy is a powerful concept. But what properties single out Shannon entropy as special? Instead of focusing on the entropy of a probability measure on a finite set, it can help to focus on the "information loss", or change in entropy, associated with a measure-preserving function. Shannon entropy then gives the only concept of information loss that is functorial, convex-linear and continuous. This is joint work with Tom Leinster and Tobias Fritz.

You can see the slides or watch a video:

On this video, after my talk you can see Tae-Danae Bradley's talk Entropy and operads. For a video of another version of my talk go here.

For more, read these:

• John Baez, Tobias Fritz and Tom Leinster, A characterization of entropy in terms of information loss, 2011.
• Tom Leinster, An operadic introduction to entropy, 2011.
• John Baez and Tobias Fritz, A Bayesian characterization of relative entropy, 2014.
• Tom Leinster, A short characterization of relative entropy, 2017.
• Nicolas Gagné and Prakash Panangaden, A categorical characterization of relative entropy on standard Borel spaces, 2017.
• Tom Leinster, Entropy and Diversity: the Axiomatic Approach, 2020.
• Arthur Parzygnat, A functorial characterization of von Neumann entropy, 2020.
• Arthur Parzygnat, Towards a functorial description of quantum relative entropy, 2021.
• Tai-Danae Bradley, Entropy as a topological operad derivation, 2021.

home