Just a note that I have put up a new post at Panda’s Thumb in response to a post by Eric Holloway at the Discovery Institute’s new blog Mind Matters. Holloway declares that critics have totally failed to refute William Dembski’s use of Complex Specified Information to diagnose Design. At PT, I argue in detail that this is an exactly backwards reading of the outcome of the argument.
Commenters can post there, or here — I will try to keep track of both.
There has been a discussion of Holloway’s argument by Holloway and others at Uncommon Descent as well (links in the PT post). gpuccio also comments there trying to get someone to call my attention to an argument about Complex Functional Information that gpuccio made in the discussion of that earlier. I will try to post a response on that here soon, separate from this thread.
Who gives a damn what you believe?!
The truth??? From you??? As you see it???
It’s a good one… You actually made me smile…
Watching ID proponents in apologetics videos is one of the best ways to learn what the ID movement is attempting to do. The brief video I linked to above is the most expositive of all that I’ve watched. Stephen Meyer identifies the crucial components of his argument in about two minutes. The following link take you 60 seconds into the video, when Meyer is just getting into the meaty part.
It takes only 18 seconds for Meyer to go from Lyell’s uniformitarianism, which he is coopting (with unacknowledged modification), to his main point: “Information always comes from an intelligence.” As you can see in the screenshot attached to this comment, the emphasis on always comes from the video. You will hear in the video that the “interviewer,” who says hardly anything, jumps in at this point to verbally emphasize always. Note that Meyer later shifts from an intelligence to a mind. I haven’t checked this closely, but I believe that that you can put mind in place of intelligence throughout ID writings without changing the meaning.
Back to your comment… The primary use of information in ID rhetoric is to implicate minds. Above, Mung stated a point that I’ve been making for years, namely, that the technical arguments of ID proponents can be stated strictly in terms of probability. I have demonstrated that the arguments are actually simpler and clearer when “-log” is not prepended to “p.” What I have said, and Mung has not said, is that the “information” talk is gratuitous.
I predict that the principal ID activists — the ones overtly affiliated with the Discovery Institute — will never acknowledge that log improbability is not itself information. They know at this point that I’m right on the matter. But they will never do anything to undercut the “information” rhetoric, in which the ID movement has invested hugely.
I don’t much object to that part. It fits with what I mean by “information.”
But there’s the real problem. The word “information” has many different meanings. The information argument from ID depends on equivocation between different meanings. It is all sleight of hand, much like so many other apologetics arguments.
Continuing my last comment, and returning to Eric Holloway and the Walter Bradley Center for Natural and Artificial Intelligence…
To make their “Information always comes from an intelligence [mind]” argument work as they want, ID activists must argue also that a machine cannot be an intelligence. Any seeming intelligence on the part of a machine must ultimately come from its designer. Machines cannot create information. That, as a practical matter, is the meaning of “conservation of information in search.”
That’s fine, as long as you’re coherent and honest. Disguising creationist arguments from improbability by applying the logarithm function is dishonest, and has in fact led to incoherence in ID claims.
Agreed.
If you want to do that soon, then I should make a priority of wrapping up “Evo-Info 4: Meaningless Meaning” (title tentative). I think you’ll want to see what I have to say before proceeding. There’s a fair amount that we’ve never discussed. For instance, the one relevant theorem of Marks et al., “algorithmic specified complexity is rare,” was derived by someone else, about 25 years ago. It’s essentially an application of Markov’s inequality. 😉
Moved a post to guano.
That seems to fit with the way I have seen the word information used, such as the enormous emphasis on creating new information, whereas simple copying and modifying what already exists doesn’t count. Is “creating” a thing that only persons can do? Is something only new when it is the result of a creative process? It really helps understanding the arguments to pay attention to these things.
Fisher information?
Yeah, people have yapped for decades about this stuff without clear definitions. Everybody is supposed to “just know” the meanings of terms like create and new information, but I honestly don’t know them. I can think of many possible meanings. And the fact that there are so many possible meanings is precisely the problem.
It seems that where Bob Marks would say that he created something, I would say that I found a solution to a previously unsolved problem. I did not adopt my language in opposition to ID. Never in my life have thought of what I did in terms of creation. It was always discovery. (I say that as someone whose first master’s thesis was a collection of 40 poems, along with a poetics. It felt to me as though the words I needed were “out there,” to be discovered.) Funny thing is, my discovery of something is the same irrespective of whether I’m actually the first to have discovered it. Something I came up with in the research for my second master’s thesis turned out to have been developed by Alan Turing for use in deciphering Enigma-enciphered messages. Did I create old information instead of new information? 😉
Tom (or anyone): Does the same apply to the phrase “Law of Conservation of Information”?
I see how one might apply that name to Levin’s mathematical result regarding Kolmogorov complexity and the related definition of mutual information. For a different meaning, there is the conservation of quantum information in fundamental physics.
But is that phrase used anywhere in consensus biology, and in particular in evolutionary models?
Definitely.
Conservation of information might make sense in terms of algorithmic information theory. But it makes no sense if we are talking about Shannon information, where we can generate as much new information as we want just because we decide to communicate.
Years ago Peter Medawar used the term in his book The Limits of Science. His argument was that if you applied a 1-1 transformation to information, none would be lost. Which is fairly obvious, since you can always apply the inverse of the transformation and get back to where you started. (Think lossless compression).
One other person used it: in a paper on algorithmics of search in 1996. Guy by the name of Tom English. I’ll leave him to tell you about that.
Neil Rickert,
I see the same issue on the evolution side. The question is what mechanism can consistently generate symbols that translate to a diverse set of functions. We know a computer programmer can. The diversity of life requires this.
I’m not seeing that. It’s the ID folk who try to describe everything in terms of symbols. As far as I can tell, evolutionary biologists are concerned with traits, behaviors (behavioral traits), adaptation and the like. Yes, they may mention DNA, but they see it more as templates than as symbols.
Neil Rickert,
This is what Joshua is doing with his cancer can create information hypothesis.
I don’t think i have this book by Quastler. I think I do have the one from 1953:
Essays on the Use of Information Theory in Biology
Oh yes, I noticed. 🙂
Gave me a new term to google. Heh.
https://arxiv.org/pdf/1602.05063.pdf
He is trying to answer the ID folk on there own terms. I doubt that his own biology research is done in terms of symbols.
Yes. I kept trying to get them to understand that H apples to “the larger expression” (the distribution as a whole).
Joshua Swamidass has this thing where he likes to claim that entropy = information and that appears to be based on his claim that -log p = H. It appeared to me that he meant -log pi = H and simply was typing in haste.
I tried to show why -log pi is not equal to H.
I always fall back on what are the probability distributions. That may or may not be the right question to ask. I’ve just now started reading up on Kolmogorov Complexity and AIT.
Well, that’s a tough one. My interest primarily lies in seeking clarity so that the sides are not talking past each other and so that a layperson such as myself can understand just what is being asserted. Information in biology has a long history that pre-dates ID.
An example is in trying to get people to distinguish between what is said to be a measure of information and what is actually being measured. Some people think that Shannon gave us a mathematical definition of information. I’m not convinced that he did.
Over at PS I argued that “surprisal” was unsatisfactory because I am no more surprised when snake eyes is rolled than I am when a seven is rolled because I know the underlying distribution. 🙂
ETA: Oh, and I prefer the term “entropy” to “information” because that’s what Shannon called it and that’s what it’s called in the books. So yes, I do think we can dispense with a lot of the “information” talk.
The only “reason why” is the definition
which plainly is not 
Or are you trying to explain Shannon’s rationale for defining entropy as he did, in the context of communication? My recollection is that Shannon gave a clear explanation in his original paper. However, I don’t recall whether there’s a brief passage that would be appropriate for copy-and-paste.
When asking about the logic of Dembski’s argument, we can omit all of it, and just talk about the probability of getting this good an adaptation (in some fitness-related sense of goodness).
The most embarrassing of all my errors. I put a lot of work into figuring out exactly how I went wrong, and into correcting my error. It was a character-building experience. I learned that there is dignity in admitting to mistakes, precisely because the process is painful. Pretending that you do not make mistakes is undignified.
A key point is that the search is on the part of the entity selecting an algorithm. The selected algorithm merely samples. A sampler begins with no information [EDIT: about unsampled points], and does not gain information about unsampled points by processing sampled points. Hence a sampling process is itself utterly uninformed, and my talk about “conservation of information” was at best misleading. See my blog post “Sampling Bias Is Not Information.”
I think his Ph.D. is in bioinformatics. If that’s right, then he has done a lot in terms of symbols. But that doesn’t mean that the referents of the symbols are themselves symbols.
Neil Rickert,
His field is computational biology. All his work is done with symbols 🙂
Dated 2017. I hope that’s a sign that the “pointwise” usage is catching on. I think it goes some way to prevent confusion that clearly is a problem. The “oh, we know what me mean, so the term doesn’t matter” response flies in the face of experience.
The tool used in investigation is not itself the object of investigation. 🙂
The source that Bruce linked to, up the thread, is my favorite. I presently have it open in a tab: “Algorithmic Information Theory.” It makes important links between classical (statistical) information, defined on distributions, and algorithmic information, defined on strings of symbols. (Note that some care is required in extending algorithmic complexity to discrete objects that are not themselves binary strings, and that Marks, Dembski, and Ewert do not take the care that they should in defining algorithmic specified complexity.)
I see danger is with the use of the term ‘symbol’ because it implies semantic information to me. But that’s not what biology is using; it uses Shannon information as far as I know.
So when using the term ‘symbol’, one has to be very careful to limit any implications of semantics to clearly specified mathematics and science which are not in need of humans to determine meaning.
For example, if one speaks of a genetic code, then it is important to recognize this is just a short form for certain facts of biochemistry along with certain cell mechanisms, and is not for meant to imply code in the sense that semantics is used for human language.
Switching topics: You say “generate symbols that translate to a diverse set of functions”. I note that functional information as originally introduced by Szostak for biochemistry is the opposite of what you wrote, at least in the following sense: Shannon info can remove the redundancies of a a genetic sequence treated as a meaningless string of symbols, but alone it is insufficient to deal with biological redundancies resulting from from the fact that “different sequences can encode RNA and protein molecules with different structures but similar functions.”
To deal with such redundancies, Hazen proposes to use the -log2 of “the probability that a random sequence will encode a molecule with greater than any given degree of function.” To make the approach apply to a specific situation, one has to specify the function of interest and then empirically estimate the number of sequences which has that function (the example function “bind ATP’ is used in his paper).
So one could say that functional information translates a diverse set of symbols into a single biological function.
Go on then. I double dog dare you not to post any more posts with that word.
Of course Shannon gave a mathematical definition of (mutual) information. The mistake is in thinking that such a definition would tell us what information “really is.” There’s a long history of appeals to Mathematical Truth in creationism, i.e., to establish what is true in physical reality. That reflects a sad misunderstanding of how math works. (I know that Neil is with me on this.)
The key point, I think, is that there is no physical measurement of probability. The reason that we have probability measures in mathematics is that Kolmogorov adapted measure theory to probability theory. The “measure” in measure theory is an analogy. Thinking in terms of physical measurement will help you in acquiring an understanding of the math — early in the process, anyway. But when a probability measure or an information measure is part of a scientific model, there’s no implication that probability or information is itself physically measurable.
Oh, no, not that!
BruceS,
If you tried to apply the function of your skin cells dividing to a diverse set of symbols you would struggle.
Many biologists do their work with symbols. But their work is not about alleged symbols in biological systems.
https://www.informatics.indiana.edu/rocha/publications/pattee/
As Martin so wisely put it I’m aware of his work.
Although admittedly not to any great level of detail.
However, I believe nothing in his work contradicts the point I was trying to make.
Am I missing something? If I am. and you have a quote to prove it, it would be great if you can provide a context for that quote in your words, and not just a cite. If you do, I am willing to ignore up to three uses of that word which must not be named.
No doubt. But I would struggle with trying to explain just about anything biological.
On the other hand, if you asked any biologist with the relevant expertise, I am sure she or he could provide an explanation involving biochemistry and mechanisms. And if you wanted to push the concept of function versus mis-function or lack of function, then a philosopher of biology could complete any missing details of the conceptual analysis to naturalize the norms involved without regression to an intelligent observer.
That’s just the Platonist in you, as I posted at PS. But the right way out of the cave is math, not dictionarianism.
You remind me of KeithS, who I assume is lost on the hiking trails of UK (soon just England), searching for his TSZ redeemer.
Why do I draw that comparison with KS? Because you seem to share his theistic view of concepts as things which exist and are explainable from some God’s-eye view. (I’m stealing and possibly misunderstanding Neil’s critique).
Of course, in your case, I believe the theistic part is fine by you.
Not sure you understood my comment. 🙂
Let me try it this way. Was Shannon seeking to create a mathematical definition of information or was he trying to create a “measure” of “an amount of information” where what information is remains as vague and undefined after Shannon as it was before Shannon?
You could be right that I do not yet grasp what constitutes a definition in mathematics so help along those lines would be appreciated.
Neither, I would think.
Shannon was giving a mathematical theory of information, which is not at all the same as a definition of information.
Compare with the theory of sets (from mathematics). The theory does not tell us what a set is. It does define or specify the operations (mathematical operations) that we are allowed to use on sets. But it never tells us what a set is.
Similarly, Shannon’s theory never tells us what information is. But it does tell us the kind of things we can do with information (in terms of mathematics).
Yes, Shannon did define a way of measuring information, although he did not define what it was that we would be measuring. But, at least in my opinion, the measurement of information was not of central importance. He did call it a theory of communication.
BruceS,
Later in this post you mention Hazen and Szostak functional information. Do you consider Shannon information and functional information the same?
No thy are not the same in general.
But you may find certain empirical cases where you can make assumptions about probability of certain physical situations and also take into account what we know about protein-related genome sequences, and then show a relation in that particular case.
As I recall, Swamidass provides such an example and explains it in his replies to English in the comments section of his OP at TSZ last year. But I am going by memory on that statement.
Possibly the cancer function stuff at PS can be looked at that way too, since it involves both function and Shannon MI. However I have been simply assuming that for Swamidass in that case, ‘function’ meant a protein with a biological function (eg to drive the survival and proliferation of cancer cells that possessed the protein in the body environment).
BruceS,
That too, I think, but I was making a comparison between Levin’s Information Non-Growth and Rao-Blackwell. RB shows us how to find a minimum variance estimator. If I am correct about ING, serving the same purpose for algorithmic statistics, then conditioning on the algorithmic sufficient statistic show how to gain the algorithmic information (which Holloway claims cannot be gained).
IOW: I think Holloway is misinterpreting the meaning of Information Non-Growth.
Just to add to Neil’s reply:
1. I don’t think Shannon was trying to or did provide a definition of the word. He could have talked about a measure of the S-factor, I suspect. Possibly he intended the word as a way for people to start to think about his work before absorbing the math. Of course, he was careful to exclude the semantic connotations of the word right at the start.
2. You can find examples of math definitions related to Shannon info in the summary section of chapter 2 of Cover and Thomas, p. 41 in 2nd edition.
3. If you want to try to associate a meaning with mathematical symbols alone, you need philosophy of mathematics to lay out the options.
4. Scientific and mathematical papers may use that word in the English gloss of the work being described. Only the math matters. The authors may include a discussion of what they think the word means and eg say it is entropy or negative entropy or surprisal or whatever;. Concentrate on the math and substitute the math wherever the word occurs in the text..
5. If you want to associated a meaning with a math term in the sense of what it refers to in the world when the math is used in the model, you have to understand how the science uses the model to make predictions about the world.
I may have understood the RB stuff back in the days when I studied statistics, but definitely not any more. I glanced at the Wiki article but that would take too much background reading for me to work through.
So I’ll take your word for it.
On Holloway’s use of the phrase: I take his English that same way I recommended Mung use English. That is, forget about what the phrase might mean to you in English and just focus on the math he associates it with. I think Levin’s result is consensus math.
I don’t think Holloway’s math per se is wrong as far as it goes; it’s the lack of a detailed biological model that is my concern. Calling something the KMSS for the complexity of the organism is a long way from what I would like to see. And in any event, I think his approach fails because it only addresses single organisms, not populations.
All old ground, of course.
Eric Holloway has now given an explanation of his argument in several comments in the thread “Does information theory support design in nature?” at Uncommon Descent. In particular, comments 539, 542, 561, and 562. I have now replied to these at Panda’s Thumb in the thread linked to above, in the post. I ended up practically spluttering. If you want to be flabbergasted too, try also comment 564.
BruceS,
I think a better argument is that cancer is the cell not functioning properly as it degrades the overall function of the organism. Kirk Durston made a solid argument that cancer is a loss of functional information as in his example it takes less specificity of p53 for cancer then from normal cellular function including apoptosis.
I agree.