Evo-Info sidebar: Conservation of performance in search

Introduction to Evolutionary Informatics, by Robert J. Marks II, the “Charles Darwin of Intelligent Design”; William A. Dembski, the “Isaac Newton of Information Theory”; and Winston Ewert, the “Charles Ingram of Active Information.” World Scientific, 332 pages.
Classification: Engineering mathematics. Engineering analysis. (TA347)
Subjects: Evolutionary computation. Information technology–Mathematics.

Denyse O’Leary, an advocacy journalist employed by one of the principals of the Center for Evolutionary Informatics, reports that I have essentially retracted the first of my papers on the “no free lunch” theorems for search (1996). What I actually have done in my online copy of the paper, marked “emended and amplified,” is to correct an expository error that Dembski and Marks elevated to “English’s Principle of Conservation of Information” in the first of their publications, “Conservation of Information in Search: Measuring the Cost of Success.” Marks, Dembski, and Ewert have responded, in their new book, by deleting me from the history of “no free lunch.” And the consequence is rather amusing. For now, when explaining conservation of information in terms of no free lunch, they refer over and over to performance.1 It doesn’t take a computer scientist, or even a rocket scientist, to see that they are describing conservation of performance, and calling it conservation of information.

The mathematical results of my paper are correct, though poorly argued. In fact, the theorem I provide is more general than the main theorem of Wolpert and Macready, which was published the following year.2 If you’re going to refer to one of the two theorems as the No Free Lunch Theorem, then it really should be mine. Where I go awry is in the exposition of my results. I mistake a lemma as indicating that conservation of performance in search is due ultimately to conservation of information in search.

The root cause of my error is a failure to recognize that the “no free lunch” theorems actually address sampling, not search. There are two main components of a search, one of which generates a sample of possible solutions to a problem, and the other of which outputs the best solution it can find in the sample. The theorems address the choice of a sampling component, assuming that the solution-seeking component is fixed. My lemma indicates that sampling processes are devoid of information. There is no conservation of something that does not exist in the first place. As everyone knows, if only by reading the news, sampling processes are distinguished by their biases. The performance (utility) of a sampling component in generating a sample of possible solutions for use by the solution-seeking component has nothing to do with information.

Evolutionary informatics is founded on the conflation of evolution and search. The main topic of the book is evolutionary search for a solution to a problem. What I hope you will remember always, after reading this post, is:

Only the sampling component of an evolutionary search is evolutionary.

The sampling component simulates an evolutionary process in which the “fitness” of a solution is its goodness. (What biologists mean by fitness is not goodness, but instead the expected number of offspring left by an organism, depending on its heritable traits.) The solution-seeking component bears no relation to biological evolution. My lemma says that the evolutionary sampling process gains no information about the fitnesses of unsampled solutions by processing the fitnesses of sampled solutions (and has no information in the first place). Technically, the sample is statistically independent of the fitnesses.

If you remember now what I hope you will remember always, then you will notice that the opening of Chapter 3, “Design Search in Evolution and the Requirement of Intelligence,” is ever so slightly misleading:

Evolution is often modeled by as a [sic] search process. Mutation, survival of the fittest and repopulation are the components of evolutionary search.

Both of the sentences are false. The first is the opposite of the truth. And the problem is not just with this passage. The authors repeatedly conflate scientific modeling of evolution with engineering of an evolutionary search for a solution to a problem. It is vital that they lead readers to misbelieve that the two are the same, because they develop engineering analysis of evolutionary search only for misapplication to models of evolution. Analysis of how well models work, under the unwarranted assumption that modelers do not model, but instead engineer evolutionary searches to solve problems, is an empty accusation of misconduct. The results of such an analysis are not evidence that the assumption holds. Marks et al. write, in the second paragraph of the preface:

Evolutionary models to date point strongly to the necessity of design. Indeed, all current models of evolution require information from an external designer in order to work. All current evolutionary models simply do not work without tapping into an external information source.

Hopefully you see now that they are referring to an evolutionary search as an evolutionary model. It is an evolutionary search for a solution to a problem that performs well (“works”). What they mean by “external information source” is a fitness function. But I established, 21 years ago, that an evolutionary sampling process does not gain information by processing fitnesses. And what sense does it make, when addressing biological evolution, to regard the probabilistic propensity of a (type of) organism to leave offspring as information coming from an external source? The fact of the matter is that Dembski et al. refer to performance as information, and to everything that causes an evolutionary search to perform well as a source of information.3 It is performance that is conserved.

1 A particularly revealing passage from the opening of Section 5.2 (emphasis added): “Here is an illustration of COI [conservation of information] using a waterbed metaphor. Because water is incompressible, if you push down on a framed waterbed at one point, it will bulge somewhere else. Consider Fig. 5.1 which is similar to a figure in Schaffer’s seminal paper [Figure 1 in “A Conservation Law for Generalization Performance“]. Each of the six images in the figure corresponds to a specific search algorithm across a space of problems. The square marked 1 is flat, illustrating an algorithm that performs exactly the same on all problems. In 2, every place the waterbed is pushed in is labeled with a ‘o’ and every place it bulges with a ‘+’. A bulge means that the algorithm performs better than average on a set of problems. An indentation indicates the algorithm does [performs] worse. For every place there is a bulge in 2, COI dictates there must be a corresponding indentation so that, on average, the algorithm here illustrated performs, on average, like the algorithm illustrated in 1. […]. Design expertise or other sources of knowledge are needed to choose a better than average [performing] search on a bulge and away from indentations. Squares 4, 5 and 6 illustrate violations of the law of information conservation. In 4, the algorithm performs better than that in 1 for a number of problems without doing worse anywhere else. A waterbed cannot bulge at a number of points without being indented somewhere else. Likewise, square 5 illustrates many indentations without any bulges. Conservation of information requires a balance between better and worse performing algorithms.”

2 Wolpert and Macready first disseminated their theorem in a 1995 technical report, “No Free Lunch Theorems for Search.” I’d already sketched a proof of the theorem in 1994, challenging Aspi Havewala in his thesis defense.

3 From Section 5.5, “Sources of Information in Evolutionary Search” (emphasis added): “Sources of information embedded in any evolutionary search are mined for active information. […] Evolutionary search mines information rather poorly. The sources of information in the fundamental Darwinian evolutionary model include (1) a large population of agents, (2) beneficial mutation, (3) survival of the fittest and (4) initialization.”

102 thoughts on “Evo-Info sidebar: Conservation of performance in search”

  1. John HarshmanJohn Harshman

    Erik: Oh wait, you do think that Indo-European and Semitic languages are related?

    Are you acquainted with Nostratic?

  2. RumraketRumraket

    Erik: Rumraket: There was never any organism born, in the entire history life life, that didn’t belong to the same species as it’s parent. That sounds counterintuitive in the context of macro evolution and common descent, I know, but think about it for a moment and you will discover there isn’t actually any issue.

    So you can get over the problem just by thinking for a moment?


    I know the trail of thought of Darwin and he did it plain wrong. As far as I know about you, you did it the same way.

    No, he didn’t do it wrong. He understood it very well.

    Erik: …which is evidence for that they are NOT related.

    Rumraket: No, it isn’t actually. For the reasons just stated.

    What reasons? “Just think about it for a moment”? Ah, you probably mean “all organisms can reproduce”.

    No, because (as I stated and you ignored), there was never an organism born that wasn’t the same species as it’s parent, yet evolution happens anyway.
    You erect an observation as a supposed barrier to common descent, but that observation isn’t a barrier to common descent, common descent is compatible with the observation.

    This is not many reasons

    It doesn’t matter how many there are. One is enough.

    it is just one false reason for which I already brought a counterexample

    Your problem here is that your analogy breaks down where biology is different from linguistics. Linguistics might be a good tool to use as an analogy, to get someone to understand how change and divergence happens over timer. So stop fussing about the linguistics and focus on the biology.

    So, you are saying all organisms can reproduce. I’d add that all organic life is cellular. Is this all or you have something more to convince me they are all related?

    I already explicitly stated that it is not merely the fact that commonalities exist, which implies common descent. How many times do I need to repeat that? Stop fucking implying that anyone thinks a mere similarity in and of itself, implies descent. NOBODY THINKS THAT, stop bringing it up.

    It is that the commonalities can be objectively sorted into nesting hierarchies, and that we know from direct observation that the entities in question(organisms) produce nesting hierarchical patterns over multiple generations when they reproduce .
    So we have the observed mechanism which predicts the pattern. And we have the observed pattern. And there isn’t any other mechanism that exclusively predicts the pattern.

    For now it’s about as good as: All languages have vowels and consonants and some of the words, such as “internet”, are practically universal, therefore all languages are related.

    If someone actually reasoned like that, I would be right there with you. But nobody here engages in that train of thought in support of common descent.

Leave a Reply