Art of the Problem | What is a bit? (Information Theory) @ArtOfTheProblem | Uploaded 11 years ago | Updated 11 hours ago
How can we quantify/measure an information source? We introduce the ideas of Nyquist & Hartley using a simple game involving yes/no questions. It's important to realize all of this happened before Claude Shannon arrived on the scene. However, this measure applies only when communication involves random sequences...


References:
Hartley - Transmission of Information
http://www3.alcatel-lucent.com/bstj/vol07-1928/articles/bstj7-3-535.pdf

Nyquist - Certain Factors Affecting Telegraph Speed
http://www3.alcatel-lucent.com/bstj/vol03-1924/articles/bstj3-2-324.pdf