Compressed Sensing / Sampling

I’m still trying to push ID forward as science. I previously suggest Bendford’s Law might be a fruitful avenue for ID research, but there were no takers I know of. Recently I came across Compressed Sensing, and I think this might also be a concept IDist want to explore. Here is the Wikipedia page:

http://en.wikipedia.org/wiki/Compressed_sensing

See also:

http://authors.library.caltech.edu/10092/1/CANieeespm08.pdf

It seems to be able to recreate structured datasets with surprisingly high fidelity from very low samples. Could it be used to find a hallmark of design?
So, Barry@UD – time to stop the apologetics wagon and do some science. Unfortunately you’ve banned the brightest minds at UD but a couple of the regulars might want to have a crack at this?

53 thoughts on “Compressed Sensing / Sampling

  1. Richard,

    Cool stuff, and interesting that the amazing Terence Tao was involved in the proof.

    It seems to be able to recreate structured datasets with surprisingly high fidelity from very low samples. Could it be used to find a hallmark of design?

    You mean, to test the idea that some natural signals are Designed to be sparse and therefore compressively sampleable?

  2. Isn’t compressed sampling a way to detect a signal that IS THERE? Using compressed sampling to search for biological design does not imply that the design is assumed a priori to be real?

  3. Guillermoe,

    Isn’t compressed sampling a way to detect a signal that IS THERE?

    The point isn’t to detect the signal, it’s to sample the signal in a way that preserves most of the information at minimal cost.

    Using compressed sampling to search for biological design does not imply that the design is assumed a priori to be real?

    Richard is talking about design generally, not biological design. He may be suggesting that compressibility itself could be a hallmark of design. I’m not sure; hence my question above.

    Periodic signals, modulo any noise, are infinitely compressible in the limit, so I don’t think high compressibility can serve as a marker of design. At the opposite extreme, purely random signals are barely compressible, so low compressibility won’t do the job either. Is there a “Goldilocks” range of medium compressibility that would work?

    Richard ‘s point is that IDers should get off their apologetic asses and do some actual science — or at least try. They can leave the apologetics to William Lane “Genocide is good if God commands it” Craig.

  4. The key mistake is to assume that what people do when designing is categorically different from what evolution does.

  5. keiths:

    Richard ‘s point is that IDers should get off their apologetic asses and do some actual science — or at least try.

    I don’t think this is the science they should apply. This is a technique to work with signals – when you know you are dealing with a signal.

  6. petrushka,

    The key mistake is to assume that what people do when designing is categorically different from what evolution does.

    But it is categorically different.

    In design, the goal is whatever the designer chooses. In evolution, the goal is improved survival and reproduction for the genes.

    Evolution is 100% trial and error, with the trials “chosen” randomly with respect to “desired” outcomes. There is no foresight.

    Design takes advantage of the capabilities of the human brain, including analysis, foresight, and the ability to simulate things mentally. Even when we use trial and error, it isn’t of the blind sort employed by evolution. When Edison was working on the light bulb, he tried lots of different filament materials, but he didn’t try disconnecting the filament from one or both of the terminals. Why? Because he knew that doing so would break the circuit, so that current would no longer flow through the filament.

    And perhaps most importantly, designers can make major leaps of a kind completely out of evolution’s reach. Evolution proceeds mutation by mutation.

  7. Guillermoe,

    I don’t think this is the science they should apply. This is a technique to work with signals – when you know you are dealing with a signal.

    ‘Signal’ is a very broad term.

    For example, the branching pattern of the objective nested hierarchy is a signal — and a very strong one. Unguided evolution explains that signal literally trillions of times better than ID does. The war is over, but most ID proponents haven’t gotten the news: ID is ruled out by the evidence.

  8. I agree that foresight is something new under the sun, but it seems just moments ago, you were arguing that brain activity entirely follows the laws of physics.

    Depending on one’s perspective, biological evolution writ large is foresightful.

    Mutations are not foresightful, but the overall process works because it invents things that prove adaptive. One might argue that the business of trying and testing is inherently foresightful.

    There is no magic way of knowing the future. brains are faster than genomes at producing variants, and human brains are vastly better at taking advantage trends, but they do not actually know the future.

    We tend to notice and remember the winners that successfully “predicted” trends, and we tend to forget all those who got it wrong. What we call foresight is chock full of confirmation bias.

  9. petrushka:
    The key mistake is to assume that what people do when designing is categorically different from what evolution does.

    I would say that they do categorically different things but, since they have some common characteristics, the results are not that different.

  10. Guillermoe: I would say that they do categorically different things but, since they have some common characteristics, the results are not that different.

    How is it categorically different?

    If brains are constrained by the laws of physics, they cannot have true foresight. They have the means to discern and project regularities and trends. Evolution does the same thing, but slower.

  11. This is not a new claim by me, nor is it original to me. I have been arguing for several years that evolution and design are conceptually the same thing.

    It’s not that there is no such thing as design; it’s that design is an evolutionary process. Even when done by humans.

  12. petrushka,

    I agree that foresight is something new under the sun, but it seems just moments ago, you were arguing that brain activity entirely follows the laws of physics.

    Sure. I’m not sure why you see that as a contradiction.

    Depending on one’s perspective, biological evolution writ large is foresightful.

    Mutations are not foresightful, but the overall process works because it invents things that prove adaptive. One might argue that the business of trying and testing is inherently foresightful.

    No, because foresight involves modeling the future. Evolution doesn’t do that. It tries things out without the slightest idea of whether they will work or not.

    There is no magic way of knowing the future.

    Of course, but there are non-magical ways of predicting the future. They are extremely useful, despite being imperfect. For example, weather forecasts have real value.

    brains are faster than genomes at producing variants, and human brains are vastly better at taking advantage trends, but they do not actually know the future.

    Brains don’t merely produce variants faster. They can model the future, albeit imperfectly.

    The engineers designing the International Space Station knew better than to install conventional flush toilets. They foresaw the disgusting (and hazardous) consequences of that design choice. Evolution can’t do that.

    Evolution and design are both capable of producing intricate systems, but they operate in categorically different ways.

  13. keiths,

    How is design possible? Say the laws of the psyche reduce to the laws of physics. Where does design come in?

    Also, short of knowing a thing’s history, how can we know it was designed? What sign in the thing itself indicates design? I suppose this is what the school of ID is seeking.

  14. petrushka: “The key mistake is to assume that what people do when designing is categorically different from what evolution nature does.”

    this is the only logical position to hold; unless of course you (pl) are able to come up with evidence for a dichotomy separating humanity from nature.

    nature has been using a slew of design tools and principles, only a fraction of which Man has only recently discovered.

    so Sir Richard is right (that didnt hurt a bit, people). those design principles are not only detectable in principle but beg to be discovered (AI engineers, we hear ya). But it aint gonna be discovered by design deniers. we know that.

    Richard is throwing flash bangs now. So this is it.

    So folks, it game time. Please hand me that laser. I need to study the intelligent properties of light.

    Oh, and get me some flash bang goggles, too.

  15. Steve: nature has been using a slew of design tools and principles

    Does this mean that foresight is involved, in evolution, or in the workings of the laws of physics?

    PS I’m not a design denier. I just don’t think these questions have been answered. If you’re saying that you know where design comes into the picture, please lay it out.

  16. Paul Amrhein:

    How is design possible? Say the laws of the psyche reduce to the laws of physics. Where does design come in?

    Well, brains are physical, and brains can do design. Is there some aspect of design that you think defies physical explanation?

    Also, short of knowing a thing’s history, how can we know it was designed? What sign in the thing itself indicates design? I suppose this is what the school of ID is seeking.

    Exactly. First they suggested irreducible complexity, but that failed. Then it was CSI, but that also failed. They are seeking, but they have not found.

  17. keiths,

    Doesn’t the ability to act on foresight presuppose an ability to make choices? But if physicalism is right, our actions are compelled by forces we know not of. That is, we don’t really have choices. Where does the ability to choose come into the picture?

  18. Perhaps if you think of it as ‘reacting to the probable future’?

    Paul Amrhein:
    keiths,

    Doesn’t the ability to act on foresight presuppose an ability to make choices? But if physicalism is right, our actions are compelled by forces we know not of. That is, we don’t really have choices. Where does the ability to choose come into the picture?

  19. I detect an appeal to an homunculus. A little guy in the machine that designs or makes models.

    Is it that what it is, or is it molecules doing what molecules do?

  20. Actually, irreducible complexity has been rebutted but not debunked. Different animals.

    CSI has yet to be formalized to a degree that it can become a tool. Its in the works.

    So there is no failure of these two design concepts.

    It is only a failure in the minds of nervous design deniers, worried about any traction ID may be gaining in the science community or the public.

    ID waxing, ND waning…..and that’s the way it is….good night and good luck.

    keiths:
    Paul Amrhein:

    Well, brains are physical, and brains can do design.Is there some aspect of design that you think defies physical explanation?

    Exactly.First they suggested irreducible complexity, but that failed.Then it was CSI, but that also failed.They are seeking, but they have not found.

  21. Well, yeah

    if you mean there’s a homunculus that causes thoughts to pop up in your brain and commands you to paint scribbles or kick a tin can or talk to your dog.

    If YOU are not in command of your body, then what is?

    Riiiiiiiightt. Of course there is no one doing any commanding. There is no central scrutinizer whispering conspiratorial messages into the ears of molecules.

    Molecules just know what to do, where to go, when to stop-go, how to block and tackle.

    They don’t need no bossman. I don’t need no bossman. There is no bossman.

    We’re freeeeeeeeeeeeee.

    petrushka:
    I detect an appeal to an homunculus. A little guy in the machine that designs or makes models.

    Is it that what it is, or is it molecules doing what molecules do?

  22. petrushka: How is it categorically different?

    If brains are constrained by the laws of physics, they cannot have true foresight. They have the means to discern and project regularities and trends. Evolution does the same thing, but slower.

    You don’t need absolute foresight in order to design.

    But, above all, what makes evolution and design very different is the intention. The definition of design involves intention.

  23. Steve:
    Actually, irreducible complexity has been rebutted but not debunked.Different animals.

    How’s that? The concept itself is absurd (that IC systems cannot evolve gradually). But I know of some papers that prove that IC biological systems can be the result of gradual evolution.

  24. Steve,

    Actually, irreducible complexity has been rebutted but not debunked. Different animals.

    It’s been debunked as a reliable indicator of design, which is what matters for the purposes of the ID debate.

    CSI has yet to be formalized to a degree that it can become a tool. Its in the works.

    Dembski seems to have given up on it. The only people “working” on it, as far as I can tell, are UDers like KF and gpuccio, but they’re certainly not making any progress. They don’t even recognize the flaws in Dembski’s version, as far as I can tell. By his definition, you can’t decide that something exhibits CSI unless you already know that it didn’t evolve. It’s useless.

    So there is no failure of these two design concepts.

    It sounds like an abject failure to me. 16 years after the Wedge Document and ID proponents still have no way of reliably detecting design in biology.

    ID waxing, ND waning….

    Looks like you swapped your ‘x’ and your ‘n’.

  25. Paul,

    Doesn’t the ability to act on foresight presuppose an ability to make choices?

    In some cases, yes.

    I add the “some” qualifier because of cases like ducking to avoid being hit by a baseball. The ducking is involuntary (at least as far as my conscious self is concerned), but there is a kind of foresight involved — the subconscious determination that the ball may hit my head if I don’t move.

    In other cases, we clearly do make voluntary choices based on what our foresight tells us.

    But if physicalism is right, our actions are compelled by forces we know not of. That is, we don’t really have choices.

    Choices are still choices, even if they are ultimately determined by causes lying outside of us. We still take in information, consider the alternatives, deliberate, and then decide.

  26. petrushka,

    I detect an appeal to an homunculus. A little guy in the machine that designs or makes models.

    There’s no homunculus problem here, because there is no infinite regress.

    There’s a representation, and there are the parts of the brain that create, manipulate and respond to the representation. The regress problem would arise only if the latter had to create a secondary representation of the primary representation, and so on ad infinitum.

  27. keiths: Choices are still choices, even if they are ultimately determined by causes lying outside of us.

    I can’t deny “choices are choices.” But I can deny “Choices are still *made freely*, even if they are ultimately determined by causes lying outside of us.” That strikes me as a flat contradiction. And I still have no answer as to where choice comes into the physicalist picture. How can something *be* that is not in the physical description of the world? Or, where is “choice” in the physical description of the world?

  28. Yet another stark difference between evolution and design is that designers can reuse complicated subsystems in completely different contexts. The same 10-million-transistor chip can be used in a microwave oven and a business jet.

    Evolution can’t do that, except in much simpler cases involving horizontal gene transfer.

  29. So a 747 has features lacking in a Piper Cub.They are both airplanes.

    Evolution and human design are bound by the laws of physics. There is no magic homunculus in the human brain. It tries and tests and changes. Human invention is incremental. Inventions build on prior inventions. Even horizontal transfer is not categorically different.

  30. Paul,

    I can’t deny “choices are choices.” But I can deny “Choices are still *made freely*, even if they are ultimately determined by causes lying outside of us.” That strikes me as a flat contradiction.

    Then you may be a libertarian (in the philosophical sense, not the political sense).

    It’s a flat contradiction only if you define “a free choice” as “one that could have been made differently under exactly (and I mean exactly) the same circumstances.” There’s an entire school of philosophical thought, called compatibilism, that rejects that definition. To a compatibilist, the important thing about a free choice is that it reflects one’s own nature, not that it could have been made differently under identical circumstances.

    And I still have no answer as to where choice comes into the physicalist picture. How can something *be* that is not in the physical description of the world? Or, where is “choice” in the physical description of the world?

    A choice is just a succession of brain states leading to a set of actions. There are brain states corresponding to all of the stages: taking in information, considering the alternatives, deliberating, and deciding. Brain states are physical things, responding to the laws of physics.

  31. keiths,

    One of your objections runs like this.

    “We make choices with our physical brains. Therefore physics implies that choice is real.”

    As far as I know “physics implies that choice is real.” is simply false. I think you agree with that. So doesn’t that make the premise “We make choices with our physical brains.” false too?

    PS I have a book arguing for compatibilism in my queue.
    PPS Sent this message before seeing your last.

  32. Why were people talking about compression of data on forms as showing that the forms were designed? Does the observation of streamlined forms in fishes, or spiral forms in flower buds, automatically imply Design?

    Observing forms that adhere to some regularities proves just that they adhere to some regularities. The regularities can be explained by natural selection, acting in the context of a developmental system that constrains the outcomes. Or perhaps they are explained by Design. But just observing the regularities by itself does not settle the issue.

  33. petrushka,

    Evolution and human design are bound by the laws of physics. There is no magic homunculus in the human brain. It tries and tests and changes. Human invention is incremental. Inventions build on prior inventions. Even horizontal transfer is not categorically different.

    All true, but you’re omitting the vast differences between evolution and design:

    1. Designers employ foresight, while evolution does not.
    2. Designers can make major leaps, while evolution works mutation by mutation.
    3. Designers can reuse complicated components in vastly different contexts; evolution has to reinvent them.
    4. Designers can select their goals; evolution is stuck with promoting the survival and reproduction of genes.

    Those enormous differences leave a characteristic signature, which is why we can confidently say that unguided evolution is trillions of times better than ID at explaining the diversity of life on earth. They’re categorically different.

  34. keiths: To a compatibilist, the important thing about a free choice is that it reflects one’s own nature, not that it could have been made differently under identical circumstances.

    Pardon my obstinance, but this back-and-forth is a welcome distraction from my reality, which is a little dark lately.

    I think this presupposes that there is such a thing as a choice which does not reflect one’s nature. But how is that possible?

  35. Joe,

    Why were people talking about compression of data on forms as showing that the forms were designed? Does the observation of streamlined forms in fishes, or spiral forms in flower buds, automatically imply Design?

    Well, the technique applies to signals generally, not just to biological forms. But your question is essentially the same as the one I asked Rich:

    You mean, to test the idea that some natural signals are Designed to be sparse and therefore compressively sampleable?

    I don’t think that would work, because:

    Periodic signals, modulo any noise, are infinitely compressible in the limit, so I don’t think high compressibility can serve as a marker of design. At the opposite extreme, purely random signals are barely compressible, so low compressibility won’t do the job either.

    I haven’t thought much about the mid-range, but I doubt that it would work either.

  36. Features do not add up to a new category. Brains embody an evolutionary algorithm. There is no magic designer hiding in the brain.

    You have dualism lurking somewhere in your designer.

  37. Paul,

    Pardon my obstinance, but this back-and-forth is a welcome distraction from my reality, which is a little dark lately.

    No problem. I enjoy discussing this stuff. All the more so if it provides a welcome distraction for you.

    I think this presupposes that there is such a thing as a choice which does not reflect one’s nature. But how is that possible?

    I actually don’t think it’s possible, unless the choice is totally random. That’s why I think that libertarian free will is incoherent. A choice is either determined by our natures, or by random factors, or by some combination of the two. But our natures aren’t freely chosen (in the libertarian sense), and neither are the random factors. So libertarian free will cannot exist.

    Note that the above argument does not depend on an assumption of physicalism. It works equally well against the idea of a soul as the source of free will, for example.

    Out of curiosity, what book on compatibilism are you planning to read?

  38. keiths: Out of curiosity, what book on compatibilism are you planning to read?

    *Making Sense of Freedom & Responsibility* by Dana Kay Nelkin

  39. petrushka:

    Features do not add up to a new category.

    It depends on the features. If features never added up to a new category, there would be only one category — for everything!

    Brains embody an evolutionary algorithm.

    Brains do not have to try everything out in reality. Evolution does. It has no foresight.

    There is no magic designer hiding in the brain.

    You have dualism lurking somewhere in your designer.

    Why do you think that modeling requires a homunculus? Google’s self-driving car models its environment, but there’s no little man doing the driving — or the modeling.

  40. You can make categories on any arbitrary criteria, but brains implement a learning algorithm that is analogous to evolution. Designs and inventions and ideas evolve incrementally by cut and try.

    What is your alternative process that merits a new category?

    To the best of my knowledge, all research in advanced AI involves some kind of evolutionary algorithm, and hardware designed to learn by trial and feedback.

  41. petrushka,

    I found this comment of yours on another thread, but I think you intended it for this one, so I’ll answer it here.

    Humans have capabilities that chimps do not have. Are humans categorically different from mammals? From animals? From living things?

    The point of assigning categories is to make useful comparisons.

    Darwin made a useful comparison berween Adam Smith’s invisible hand and biological evolution. I see a useful comparison between thought and evolution. I see large differences in the substrate, but I and large numbers of computer scientists see something useful in the concept of variation and feedback.

    Of course! I don’t disagree that trial and error is useful, or that humans do it. I simply disagree with your statement that

    The key mistake is to assume that what people do when designing is categorically different from what evolution does.

    It is categorically different, for all the reasons I’ve already mentioned and then some.

    And even trial and error is different when done by humans. Humans can remember their mistakes so that they don’t have to keep repeating them. Edison didn’t keep trying cotton thread over and over for his light bulb filament. He recognized that it wouldn’t work and moved on. Evolution doesn’t do that, because it has no memory for mistakes. Mutations are random, so the same mistake may be made again and again.

  42. Yes, I have to admit that humans don’t keep repeating our mistakes. Human history is a testament to that.

    But seriously, category lines can be drawn on any arbitrary criteria. My criterion for inclusiveness is utility.

    I note that the evolutionary paradigm is a learning paradigm. I don’t agree with your concept of mistake. Variants are useful or detrimental in context, and context changes. That’s true in biochemistry, and it’s true in the world of ideas.

    The category I am referring to is the category of system capable of learning from feedback and capable of inventing. Focusing on differences in details blinds one to the potential benefits that can be derived from considering the similarities. GAs are a case in point.

  43. I think the key difference between evolution and design is keiths‘s point 4:

    4. Designers can select their goals; evolution is stuck with promoting the survival and reproduction of genes.

    We know there is a goal behind design. There must be a goal for design: something that is intended a priori. Evolution does not need a goal. Perhaps there is something that intended life to exist or to be in a certain way, but evolution works just fine without acknowledging it.

  44. petrushka,

    But seriously, category lines can be drawn on any arbitrary criteria. My criterion for inclusiveness is utility.

    And earlier:

    The key mistake is to assume that what people do when designing is categorically different from what evolution does.

    By your logic, motor oil isn’t categorically different from design, since both have utility. I think your category may be a bit too broad. 🙂

  45. petrushka,

    Focusing on differences in details blinds one to the potential benefits that can be derived from considering the similarities.

    And focusing on similarities blinds one to the potential benefits of considering the differences. Best to combine the two and give both the similarities and the differences their due.

    And by the way, the differences between design and evolution are not merely “differences in details”. They’re huge and important.

  46. “Unfortunately you’ve banned the brightest minds at UD but a couple of the regulars might want to have a crack at this?”

    Rich, have you been banned again? I am definitely not one of the “brightest minds at UD”, but Barry certainly takes pride in banning me.

  47. william_spearshake:

    “Unfortunately you’ve banned the brightest minds at UD but a couple of the regulars might want to have a crack at this?”

    Rich, have you been banned again? I am definitely not one of the “brightest minds at UD”, but Barry certainly takes pride in banning me.

    william,

    Rich wrote that before Barry declared his “general amnesty”.

    Barry hasn’t banned anyone since the amnesty, as far as I know. His ban finger must be securely tied to something, like Odysseus to the mast.

  48. “Barry hasn’t banned anyone since the amnesty, as far as I know”

    Sorry to burst the bubble. I have been banned, again. After Barry calling me a “pathetic, snivelling coward” for refusing to respond to a loaded question.

    What Barry is doing differently is that he is no longer announcing a ban. Although he came close when he responded to a question by Mr. Mullings (which he deleted) that “he sent ’em packin’, as usual”.

    If you look at the recent comments part of UD you will notice a significant reduction in anti-ID commenters. The optimist in me would like to think that they simply got bored debating mental midgets. But, somehow, I don’t think this is the case. The one thing that is noticeable is the absence of Barry in the commentariate. I think that he is busy stemming the flow of rational arguments that are counter to ID.

Leave a Reply