I want to thank OMagain in advance for doing the heavy lifting required to make my little tool/game sharable. His efforts will not only speed the process up immeasurably they will lend some much needed bipartisanship to this endeavor as we move forward. When he is done I believe we can begin to attempt to use the game/tool to do some real testable science in the area of ID . I’m sure all will agree this will be quite an accomplishment.
Moving forward I would ask that in these discussions we take things slowly doing our best to leave out the usual culture warfare template and try to focus on what is actually being said rather than the motives and implications we think we see behind the words.
I believe now would be a good time for us to do some preliminary definitional housework. That way when OMagain finishes his work on the gizmo I can lay out some proposed Hypotheses and the real fun can hopefully start immediately.
It is always desirable to begin with good operational definitions that are agreeable to everyone and as precise as possible. With that in mind I would like to suggest the following short operational definitions for some terms that will invariably come up in the discussions that follow.
1. Random– exhibiting no discernible pattern , alternatively a numeric string corresponding to the decimal expansion of an irrational number that is unknown to the observer who is evaluating it
2. Computable function– a function with a finite procedure (an algorithm) telling how to compute the function.
3. Artifact– a nonrandom object that is described by a representative string that can’t be explained by a computable function that does not reference the representative string
4. Explanation –a model produced by a alternative method that an observer can’t distinguish from the string being evaluated
5. Designer– a being capable of producing artifacts
6. Observer– a being that with feedback can generally and reliably distinguish between artifacts and models that approximate them
Please take some time to review and let me know if these working definitions are acceptable and clear enough for you all. These are works in progress and I fully expect them to change as you give feedback.
Any suggestions for improvement will be welcomed and as always please forgive the spelling and grammar mistakes.
peace
When I ask for examples I expect you to walk us through some of your test cases.
Show us how the strings are generated.
Here is something else that seems to be relevant to federalism and the idea that systems can choose
https://en.wikipedia.org/wiki/Wigner's_friend
Not to get all “woo” on you but I think that in the end we will find some individual conscious agent at the heart of any choice
I think I get that as far as it goes.
If an entity is merely following phyisical rules then no real choice has been made OK.
However pragmatism is only one means that an agent to use in making a choice
I would say that in order for it to be considered a “real” choice no there must be physical constraints involved
peace
Expect an op as soon as OMagain is finished. There is really nothing to the generation it is so basic and simple. But if previous experience is any guide I expect a have to do a lot of explaining
A normal comprehensive test scenario would look like this
You would have at minimum 5 strings 1 real and 4 fakes
1) the “original” string produced by representing an object or process numerically. There are a near-infinite number of ways this can be done.
2) a “complexity” string that is created choosing a random spot to cut the original sequence and reassemble so that you have new string that identical to the original except it begins in a different spot
3) a “random” string that is created by simply randomizing the original string as to order
4) a “model” string that is created by any algorithmic process er choose. (I usually use an EA). What is important is that it be close but not identical to the original and that the algorithm not target the specific digits in the original string
As a bonus we might include one or more “manual” copies in which we take a “model” string and do some post processing smothing
Then we will load the real string an a fake into the game and start the fun
In order to actually infer design the observer needs to be unable to distinguish the real string from the “manual” and “complexity” strings but able to distinguish when it comes to the “model” and “random” strings.
One of the first things I want to see is if different observers with conflicting goals biases come to the same conclusions.
It will be interesting.
I have no idea how OMagain is going to handle the string loading process but it is important for a valid test that the observer be kept in the dark as to which fake is loaded at any one time.
like I said I will put together an OP with instructions and rules once I’m familiar with what OMagain has cooked up
peace
fmm,
I have a couple of questions.
The financial game uses feedback, i.e. users get to see a series of different actual market returns and each time compare them with a fake to make a call on which is which. They receive feedback on each decision and then go to the next comparison. Over time their calls become more reliable.
In your examples you run your game with just one single original. How do you do the feedback runs? Do users each time get to see the same original with a different fake?
My other question goes back to my previous comment. From a perturbed copy of the original A you generate a fake B that is close to, but not identical with, the original A using an EA algorithm. Right? If users can tell B from A you conclude that A is designed.
You could repeat this procedure, and run the EA on a perturbed copy of the first fake B to create a second fake C that is close to but not the same as B, and run the game pretending that B is an original and C is a fake. You need to use a different observer. If this observer can tell C from B you would have to conclude that B is designed, according to the rules of the game.
But we know it is not, because it was made using an EA and used as an example of an algorithmically generated string that users can tell apart from a designed one.
How do you resolve this paradox?
fG
To prevent a repeat of the usual endless thread that goes nowhere, could you give an example of an object that can be completely represented by a finite string?
An some examples of things that cannot?
This will probably set us off on a tangent, but I am curious under what definition of ‘choose’ do computers not choose?
fG
any phyisical thing can be represented by a finite string
peace
Same original and same fake.
You make a guess and the game tells you if you are correct. Then the same strings are reset to a random starting position. And the process repeats
This continues till you “learn” what it is that distinguishes one string from another.
Once you play the game it will become obvious.
If I understand you correctly there is no paradox
Both the original string and the model are designed. A model is an artifact by definition.
Artifacts can be made using algorithms they just can’t be explained by an algorithm.
Look again at Jenny’s number 8675309
lots of algorithms can produce Jenny’s number for example (8675308 plus one)
However there is no algroythym to tell us which algroythym to run to produce Jenny’s number unless we already know it
That is what mean when we say Jenny’s number in noncomputable
Does that make sense?
peace
Under he standard definition
choose— pick out or select (someone or something) as being the best or most appropriate of two or more alternatives.
or this one works as well
Choose—decide on a course of action, typically after rejecting alternatives.
What a computer does is simply follow a constrained predetermined course when confronted with a particular input .
When a program is presented with an IF statement it simply does what it’s “programmed” to do
The programmer does the deciding
peace
That is clear enough. I would think that you need quite large strings for the game otherwise a observer would very quickly recognise the original from repeat runs. What is the minimum length you recommend?
First of all, if a string ‘B’ generated by an EA from a designed string ‘A’ counts as an artifact, and if the observer correctly flags A as designed, he has in fact somehow managed to distinguish one artifact from another. This tells me that something else is going on than raw ‘design detection’, because if both A and B are designed there is no reason why the observer would notice a difference in ‘designed-ness’ between the two. So what is going on? Your telephone number example gives us a clue.
Your game is about the meaning of a string, which in many cases has nothing whatsoever to do with the string itself. There is nothing in 8675309 as a number that makes it principally different from 8675308. The game would not help in distinguishing the two unless the observer knows (or is) Jenny. Conversely, when 0163547752 comes up, I would flag it as designed because it looks familiar whereas 0163247753 does not – because the first number is a telephone number I know and the second is not. You, on the other hand, would not recognise either as designed.
Under your usage of the word computable, whether or not a number is computable is not a function of the number itself, but of the observer’s pre-existing knowledge. The game doesn’t let you distinguish designed from non-designed, it lets you distinguish stuff you know, or stuff that looks a bit like stuff you know, from stuff you don’t know. It isn’t about the strings at all, it is about the observer and the extent of their knowledge.
fG
A computer selects the most appropriate of two alternatives whenever it executes an If…Then statement, therefore per your first definition it chooses. There is no requirement under your own definition that whoever specifies what is most appropriate has to be the same entity as what does the choosing.
You are of course referring to Free Will here, which is not a scientific concept. Scientifically speaking we simply don’t know if humans are also ‘programmed’ to do what they do, or not. As far as I understand there are experimental results that suggest that whoever or whatever does ‘our’ choosing may in fact be something else than our ‘conscious controller’.
No reason to pursue this here, I just wanted to make it clear.
fG
I have struggled with this one. What I came up with is way cool I think
The solution is in the “complexity” string.
Only if we can’t distinguish between an original string and one that is identical but starts at a different place can we say have enough length and complexity to compare.
Simple or short strings are easily distinguished from exact copies that don’t match digit for digit.
long and complex strings look alike regardless of where you start.
peace
I think this is close to the mark but miss just a little.
I would say that design inference is actually a form of communication between the designer and the observer.
When we recognize a nonrandom noncomputable pattern in a string what we are doing is sensing something in the process that produced it that is similar to the way we think.
It all has to do with our theory of mind IMO.
When we infer design in an object we are expressing a kinship with the entity that produced that object.
It’s possible that our intuition is wrong and there is no mind at all behind the artifact. After all there is no way to prove that any other minds exist whatsoever.
This is speculation I’d like to explore as we mover forward
peace
I agree
If it is the case that humans are programmed to do what they do then I would say we don’t choose either and only God actually does so.
Keep in mind as a Calvinist I believe that our choices are predetermined I just don’t think they are physically determined.
Anyway this is not something we will solve here and it does not as far as I can tell effect the game. So we can just leave it as a mystery
peace
Then show me how to represent a glass of water as a finite string. Give me an example.
So are we talking about substrings of very long strings? What sizes substring does your game prefer?
You could
1) measure the temperature change over time
2) measure the distance of the perimeter at each spot for 360 degrees from a particular point
3) measure the evaporation rate over time
4) measure the relative depth from the rim around the circumference of the glass
etc etc etc
The possibilities are endless
I could do sub-strings or not there is really no rule,
like I said the string just has to be long and complex enough that I can’t distinguish the real one from an identical string that begins at a different place.
I think this all will be obvious to you when you actually give it a try
peace
So when you use the word represent, you mean some bits of data that could be measurements of the object.
I see no reason to give it a try. You have mad such slippery definitions that you can keep coming back and saying, no, that isn’t playing the game correctly.
I will consider playing after I see what you do for a few rounds. I hope others also hold back until you have defined things unambiguously.
The whole point of making the game shareable is to make this a cooperative effort with input from both sides.
None of my definitions are set in stone I expect to modify them as new data and feed back come in,
I am disappointed but not surprised that apparently your culture war footing prevents you from being involved in the early stages of this enterprise.
you will miss out on some fun
peace
No, you don’t get to modify your definitions after the fact. A essential part of doing science is to define the conditions unambiguously so that anyone can do the experiment and see if they get the same results.
If you can’t define your terms so that everyone can agree that the rules are followed, you are being dishonest.
the science doesn’t start till the definitions are agreed on and we are familiar with the game . Think of it as calibration if you like.
That was the point of this tread if you recall
peace
Coming up with useful definitions is part of science.
touche.
So the science has already started 😉
peace
But you won’t get much credit for that, until you have something actually works and that scientists find useful.
Going back to your definitions, we need to be clear on what the program actually flags up if and when it manages to identify an artifact.
Say you somehow manage to represent a squirrel as a string (don’t ask me how, but I might ask you how), and observers using your program flag the squirrel as an artifact with enough statistical significance to warrant a ‘design’ conclusion.
What can we now say about the designer, apart from it being an ‘entity that can produce a squirrel’? Could it be God, or evolution, or a pair of other squirrels? Can we say anything at all apart from noting that a squirrel is non-random (which is hardly a surprise)?
fG
ah the million dollar question I knew that was coming
We know that it whatever produced the squirrel is an entity that is capable of producing nonrandom noncomputable strings. It is not an algroythym like RM/NS
It shares this capability with humans. So in that sense at least it is like us and not like the straw man version of Darwinism .
We can if we choose separate all causes in the universe into two categories those that are like us and those that are not,
That is I would argue this is the beginning of a theory of mind. In the universe we have those things with mind and those things without.
Yes it could be any of those things
The game does not tell us but it does tell us that what ever produced the squirrel “thinks” like me and not like my PC.
Like Neil and I were discussing earlier if we determine that the behavior of sunflowers is nonrandom and noncomputable I would say that somehow there is consciousness at the base of it of it.
This is where the validity of Integrated Information theory will come into play
Of course we can choose to say that just because a process is nonrandom and noncomputable does not necessarily mean that it has consciousness at it’s root.
There is really no way to scientifically demonstrate that other minds exist at all. The skeptic will always have an escape clause if he chooses to use it.
In the meantime there is lots of work we can do with out exploring the ultimate implications at all.
We can also say that wherever produced it it is noncomputable. As such it is a rare thing indeed.
It is different than most things in the universe.
The only other thing that I can say for sure right now that produces nonrandom noncomputable stuff is me.
I realize that this is the most controversial thing about the game and I fully expect you to have a different theory about what is going on.
When the time comes I will love to hear it
peace
I guess we are going to get another 600 posts without fifth committing to any usable definitions.
I am betting when the game starts we will see a great deal of that’s not what I meant.
So give us a worked out example of your four strings.
petrushka,
To give readers an idea of how ridiculous the goalpost-shifting is likely to become, here’s an example from June of this year.
I pointed out something I dubbed “the resolution problem”:
In response, fifth moved the goalposts all the way from one endzone to the other:
I was incredulous:
fifthmonarchyman,
Let’s indeed be very clear.
In the referenced paper, the authors had six sets of data, each based on different financial instruments and time periods. For each instrument they chose a large number of time series ranging from about one hundred to about five hundred consecutive data points each. From each time series they created a new time series by rearranging the deltas between each point. This means that the fake time series begins and ends at the same value as the real one but a graph of the values doesn’t have the same shape.
Participants were asked to select the real time series for a number of trials within the same instrument data set. They received immediate feedback after each time series.
The result reported were that participants were able to identify the real time series approximately 73% of the time.
So, if I create similar data sets for a few financial instruments and train a model to have 73% or greater accuracy on test data that is not part of the training data (but that is, of course, for the same instrument), that would meet your challenge?
So far so good.
How do you address the fact that neural networks behave exactly as the Maguire et al. paper describe integrated memories working? They are an existence proof that your idea is incorrect.
I’ve done a lot of work in the financial industry and I see no way to make money off such a model, let alone crash the stock market. You’re leaving out the middle portion of your argument again.
fifthmonarchyman,
I notice that you never quote from the paper to support your assertions. That’s probably because nothing in the paper does support them:
“Tononi (2008) explains the foundation of his theory through two thought experiments . . . The second establishes the requirement for a conscious observation to be integrated with previous memories, hence generating integrated information.”
“While it seems intuitive for the brain to discard irrelevant details from sensory input, it seems undesirable for it to also hemmorrhage meaningful content. In particular, memory functions must be vastly non-lossy, otherwise retrieving them repeatedly would cause them to gradually decay.”
Nowhere does the paper make the distinction without a difference that you’re claiming it does. The authors talk about memory and make a demonstrably incorrect claim. Memories do change when they are retrieved. This is an essential claim in the paper, which leaves the conclusions unsupported at best.
I have prepared four files (strings) for submission. One is a transcription of a famous published text. I don’t know what it says, because I don’t speak the language.
I have snipped out two sections of equal length.
I have made a half dozen character changes to one section. I don’t know anything about the text, so the changes are meaningless to me.
I have made a scrambled copy of the same section.
Am I on the right track for playing the game?
How did you represent the text numerically?
peace
I expect there will be a lot of clarification and back and forth we have radically contradictory biases and goals
Often we talk past each other as witnessed by the recent data compression verses reminiscence debacle.
I would hope that the effort will be worth it
we shall see
peace
I am not an expert in neural networks but I have seen convincing evidence that in fact they don’t integrate information in the way that the paper describes
check it out
http://www.evolvingai.org/fooling
peace
Are you suggesting that humans don’t make mistakes and have high confidence?
It’s simple. Generally humans can predict the direction of the market better than software.because we recognize global patterns that computers do not.
Of course software does perform better in certain specific cases but not generally and reliably better.
If software could generally and reliably out predict humans then the possessor of that software would literally always win in the market.
It would be the equivalent of Biff’s sports almanac
peace
Do humans or software place more orders?
fifthmonarchyman,
Your understanding of complex software systems is lacking. There are systems in production that monitor themselves, model alternatives based on historical data, and continuously optimize themselves under changing conditions. Many of the behaviors of such systems meet both your definitions of “choose”.
http://www.bloomberg.com/news/articles/2014-02-18/fx-traders-facing-extinction-as-computers-replace-humans
Just on general principles, I’d say that the errors made by image recognition software are a symptom of integration, rather than a symptom of lack of integration.
Integration will always produce a statistical value when face with ambiguous input.
petrushka,
Is that supposed to prove something? Of course software enables us to do more with less people that is sort of the nature of tools
Just because traders use software to do their jobs does not mean that software always outperforms humans at predicting the market.
I use software in my job as well but models are always wrong. that is the nature of modeling
peace
On sign that integration is improving is the increasing difficulty of producing Captcha images for internet identification security. All this is in its infancy.
The models are not wrong.
What goes wrong is there’s a Red Queen competition going on in which many traders are all trying to take advantage of weaknesses in each other’s software. It’s all artificial, unless you have insider knowledge, or correctly guess that some real change is about to happen in the value of a company.
FMM: Are you going to tell me whether I have produced qualifying strings for your game?
What? You are going to have to clarify this
It’s obvious that the software is not losslessly integrating the information in the images
Did you look at the pictures? The researchers do just what I do in the Game.
They run an EA to produce an image that is “close” but not identical to the original
However unlike humans the software is unable to distinguish between the real and the fake image
seems pretty cut and dried to me, What am I missing?
peace
Did you represent your text as a numerical string?
peace
If software has weaknesses the models are wrong. That is pretty much what wrong means after all
peace
hex.
If the paper is correct there will always be somethings that computers can’t do like losslessly integrate information
That is what we mean by noncomputable
peace
Are you going to say whether my method of producing test strings is acceptable, and if not, will you show some correctly produced samples, and explain how they are made?