Uploaded February 2019 | Updated September 2026, 2 hours ago
Entropy is often taught as a measure of how disordered or how mixed up a system is, but this definition never really sat right with me. How is "disorder" defined and why is one way of arranging things any more disordered than another? It wasn't until much later in my physics career that I learned a much better way of thinking of entropy: in terms of information.
In statistical mechanics, we try to substitute exact descriptions of systems for descriptions based on probabilities and average states. However, in doing so, we sacrifice information about the precise state of the system, so we introduce entropy to account for this ignored information. With this in mind, we can derive a relatively simple expression for the entropy corresponding to the large-scale behaviors of a system. Once we do that, we can apply it to a simple system to make sure that our definition works as it is supposed to.
As it turns out, this definition of entropy has more applications than just to thermodynamics. It is often used to study quantum entanglement and when applied to black holes, it sparked the search for a quantum theory of gravity.
Check out Higgsino Physics's video on entropy: youtube.com/watch?v=RJ7H6bKbp24
Entropy is often taught as a measure of how disordered or how mixed up a system is, but this definition never really sat right with me. How is "disorder" defined and why is one way of arranging things any more disordered than another? It wasn't until much later in my physics career that I learned a much better way of thinking of entropy: in terms of information.
In statistical mechanics, we try to substitute exact descriptions of systems for descriptions based on probabilities and average states. However, in doing so, we sacrifice information about the precise state of the system, so we introduce entropy to account for this ignored information. With this in mind, we can derive a relatively simple expression for the entropy corresponding to the large-scale behaviors of a system. Once we do that, we can apply it to a simple system to make sure that our definition works as it is supposed to.
As it turns out, this definition of entropy has more applications than just to thermodynamics. It is often used to study quantum entanglement and when applied to black holes, it sparked the search for a quantum theory of gravity.
Check out Higgsino Physics's video on entropy: youtube.com/watch?v=RJ7H6bKbp24
