Given the importance of information theory to some intelligent design arguments I thought it might be nice to have a toolkit of some basic functions related to the sorts of calculations associated with information theory, regardless of which side of the debate one is on.
What would those functions consist of?
Over in the In Slight Defense of Granville Sewell thread there’s a discussion going on between myself and keiths over the entropy of a pair of dice. A tempest in a teapot I’m sure, with any disagreement probably revolving around his use of the Boltzmann equation and my use of the Shannon measure.
Perhaps we can have some fun and learn something as well. Here’s a programming challenge:
Write the code necessary to calculate the entropy of a pair of standard six-sided dice.
What functions should we write to do this? Please explain your code. Extra credit for writing tests to validate the code. Let me know if you have any questions.