A look at Keiths paper

So, here is the link to a paper which Keiths claims says something about income inequality, and I say is another example of the proliferation of shoddy science.

http://pss.sagepub.com/content/early/2015/03/19/0956797614567511.abstract

The highlight of the paper is this claim:

“We found that of the 40 search terms used more frequently in states with greater income inequality, more than 70% were classified as referring to status goods (e.g., designer brands, expensive jewelry, and luxury clothing). In contrast, 0% of the 40 search terms used more frequently in states with less income inequality were classified as referring to status goods.”

Where does one begin to critique the ridiculousness of this claim? 70% of the majority of searches are for luxury goods in some states, 0% of the most searched items in other states?

If one claims the difference in search patterns from one state to another is that dramatic, shouldn’t ones bs detector already be ringing alarm bells?

And what is considered a luxury good? What is the cut-off for equal states and unequal states? Did they decide the luxury terms before or after they viewed the data? Who do they claim is doing all this searching for luxury, the haves or the have nots?

The red flags are everywhere. Isn’t it likely that they had a conclusion that they wished to reach, and that they fulfilled their own prophecy?

140 thoughts on “A look at Keiths paper

  1. Also, Walasek and Brown have made their data available on the internet. See the ‘Open Practices’ section of the paper.

    I look forward to your demonstration that their conclusions are not supported by the data.

  2. phoodoo:

    Where does one begin to critique the ridiculousness of this claim? 70% of the majority of searches are for luxury goods in some states, 0% of the most searched items in other states?

    That’s not what the paper claims. Did you even read the abstract? It says:

    We used Google Correlate to find search terms that correlated with our measure of income inequality, and we controlled for income and other socioeconomic factors. We found that of the 40 search terms used more frequently in states with greater income inequality, more than 70% were classified as referring to status goods (e.g., designer brands, expensive jewelry, and luxury clothing). In contrast, 0% of the 40 search terms used more frequently in states with less income inequality were classified as referring to status goods.

  3. Have you read the paper (not the abstract), Phoodoo? Because I don’t think you have. (I haven’t, but then I’m not writing a post about a paper I haven’t read..)

  4. Don’t dodge the question, Phoodoo. The only information you’ve disclosed is in the abstract. How pathetic would it be to ‘critique’ a paper on the abstract alone? I don’t think you have read it, and I think KeithS has. If you’d have read it your post would have been more insightful.

    Fondle your watch and think on.

  5. There is no such thing as income inequality. left wing commie jazz.
    People get the income based on abilities or lack of it.
    if there is a interference with what a income should be relative to the market then okay.
    Yet that would be minor cases relative to hundreds of millions.
    Why is this on a origins forum? Id evolution tied to left wing commie worldviews? Say it ain’t so? Could explain why it stucjk around and why its going along with commie to the ashheap of history as Reagan.

  6. Richardthughes,

    I don’t think keiths has read it.
    But if he has, he can either submit a link to it, or he can explain HOW they determined these insane statistics.

  7. Yes, phoodoo, I’ve read the paper. Have you?

    As Rich says:

    How pathetic would it be to ‘critique’ a paper on the abstract alone?

  8. Byers:

    There is no such thing as income inequality. left wing commie jazz.

    That’s right, Robert. Everyone makes exactly the same amount of money.

  9. phoodoo,

    As Creodont suggested, you could always hock your watch to pay for the paper.

    If you’re worried about the loss of status, then just strap the paper to your wrist, where the watch used to be.

  10. phoodoo:
    Richardthughes,

    I don’t think keiths has read it.
    But if he has, he can either submit a link to it, or he can explain HOW they determined these insane statistics.

    Have you read it Phoodoo? As you’re critiquing it an all..

  11. “But if he has, he can either submit a link to it,”

    Imagine a guy who wears an expensive watch but wont pay to read a paper and tries to get others to find a free copy for him. How sad would that be?

  12. blah blah blah, something off topic.

    blah blah blah, you are wrong.

    blah blah blah, there is a better way but you can’t see it.

    blah blah blah.

    And to answer your question

    Where does one begin to critique the ridiculousness of this claim?

    I’m not surprised you are at a loss. Where does one begin indeed? Answer that and you’ll understand why your previous “critiques” are not taken seriously.

  13. Isn’t it likely that they had a conclusion that they wished to reach, and that they fulfilled their own prophecy?

    Now, apply that thinking to ID….

    Out of interest, have you ever heard of “projection”? It’s where, in essence, you project the flaws you have onto other people.

    So, please please please, do more OP’s. As far as “projection” goes, this is a solid gold classic.

    Tell me, did you decide the universe was designed before or after having “evidence” for that?

  14. keiths,

    You claim to have read it…so I claim the results are bogus.

    How did they determine what was a luxury search item?

    What was the dividing line between an “equal” state and an “unequal state?”

    What ranking did they use to decide which states were equal and which weren’t?

  15. phoodoo: You claim to have read it…so I claim the results are bogus.

    Claims based on nothing are worth precisely that.

  16. Claims based on nothing are worth precisely that.
    OMagain,

    I agree, keiths claim that he read it is worthless until he proves otherwise.

    Very good point Omagain!

  17. I’m with OMagain: given phoodoo’s continued inability (despite repeated correction) to understand what the abstract actually says, rather than the caricature he wants it to be,

    Isn’t it likely that they had a conclusion that they wished to reach, and that they fulfilled their own prophecy?

    is deee-elicious.

    I don’t hold out much hope that the situation would improve if he actually read the paper.

  18. It’s the “Barry Arrington Quotemine incident” all over again. We gave Phoodoo authroship to talk about a paper he hasn’t read. ID-tastic.

    Nice watch though.

  19. keiths:
    Another phoodoo flameout.

    Keiths,

    You have the nerve to claim it is a flame-out from me??

    You post an abstract to a paper which you claim says something about your topic. When you are challenged on that information, all you have done is run away and hide. You claim to have read the extended paper, and yet you refuse to comment on the paper.

    This forum exists as a way for outsiders to see the point of view of those who claim to be scientific. I post here for one reason-to show just how poorly informed, biased, and uninterested in truth, those who claim to be “scientific skeptics” actually are.

    I am perfectly happy for you to leave a record of dubious postings, and uncritical thought-for anyone who wishes to take an honest view of. I don’t mind if your obvious intellectual dishonesty is pleasing for the Omagains and DNA Jocks of the world. There are others will actually have an interest in truth, who will read what you write and see that the truth is, the science skeptics are full of hot air and no substance. For that cause your style is very enlightening.

  20. DNA_Jock,

    Has someone prevented you from saying what I stated that was wrong?

    Or are you just another keiths, a guy who throws out bs science but is unable to defend it.

    Do you believe the figures in this paper? Is it possible that the difference in search patterns from state to state are so dramatic and so tied to the income inequality of the state?

    Do you want to buy some land in Florida. Or maybe a watch from Keiths? Just how gullible are you science skeptics?

  21. What the authors did (explanation without commentary):

    They determined the income inequality level for each state using the Gini coefficient, which is a commonly used measure. It provides a metric for how distributed or concentrated income is in a population (e.g., if 20% of people make 80% of the income, the Gini value is high).

    The goal was to see if interest in “status” goods is affected by inequality. But since interest in status goods may also be affected by other factors as well, the authors did a regression analysis to control for overall income levels, population, urban population, and numbers of domestic vs. foreign born residents, in order to isolate the effects of inequality from these other factors.

    They thus derived a metric for each state that represents income inequality adjusted to account for those confounding factors.

    To answer one of Phoodoo’s questions, there was no cut-off for equal or unequal states, rather each state was assigned a value that could be higher or lower.

    To answer another of the questions, the authors did not decide what was a luxury good, nor did they determine what Google search terms to look for.

    They used Google Correlate. Google Correlate allows you to upload a set of data by state with which you can then correlate search terms. Google Correlate doesn’t tell you the most frequently searched for terms by state; that is a misinterpretation of the results. Rather, it tells you the search terms that are most highly correlated with the data you uploaded.

    For example, if you had data for how many members of the Star Trek fan club resided in each state, and uploaded that data to Google Correlate, it will tell you which search terms are most highly correlated with that data – that is, those search terms for which the distribution by state of the frequency with which people search for them most closely matches the distribution by state of your original data. So, in our Star Trek fan club example, we might find that states with higher concentrations of fan club members than other states will have a correspondingly relatively higher frequency with which people search for terms like “Kobayashi Maru”, “stardate”, etc. It doesn’t mean that “stardate” is one of the most searched for terms in a state, just that the frequency of search on that term is most highly correlated by state with the original data, in this case the number of Star Trek fans. From this we might surmise that Star Trek fans have a higher than average interest in Star Trek terminology. (This may seem obvious, but the point of the method is to discover correlations that may not be obvious.)

    Google Correlate will also give you the search terms that are most negatively correlated with the original data. For instance, the term “romance” might be the top term negatively correlated to Star Trek fan club membership (that’s a joke, by the way, for the sake of the hypothetical example; sorry Star Trek fans!). We might surmise from this that Trekkies have little interest in information about romance.

    Anyway, back to the study. The authors submitted the control-adjusted values of inequality by state to Google Correlate, which then output the top terms that were most positively correlated to that data and the top terms that were the most negatively correlated. The authors didn’t know in advance what those terms would be.

    The next step was to determine if those terms were indicative of interest in status goods. The authors didn’t make that determination. They got 60 volunteers to participate in the study. They gave them the following definition:

    “Some things that people are interested in, or like to buy or find information about, are things that show how rich or successful they are compared to other people. These are sometimes called “positional goods” or “status goods”. Someone who buys such goods may be particularly concerned to demonstrate their social status”

    They then gave them the lists of the search terms positively and negatively correlated with income inequality (without telling them anything about the study, the distinction, or the how the terms were generated) and asked them to determine for each term whether it indicated an interest in status goods based on the above definition. They could respond yes, no, or not sure.

    The result was that of the 40 terms most positively correlated with inequality distribution by state, 29 (72.5)% were viewed as referring to status goods by the majority of the test subjects. Of the 40 terms most negatively correlated with inequality distribution by state, none of them (0%) were viewed as referring to status goods by the majority of the test subjects.

    This doesn’t mean that status goods were the most searched for terms in high inequality states, nor does it mean that status goods were never searched for in low inequality states. It means that most of the search terms most closely correlated with inequality make reference to status goods (e.g., people in relatively higher inequality states seem to be proportionately more interested in “David Yurman earrings”), while none of the search terms least closely correlated with inequality make reference to status goods (e.g. people in relatively lower inequality states seem to be proportionately more interested in “pirate talk”).

  22. phoodoo,

    I post here for one reason-to show just how poorly informed, biased, and uninterested in truth, those who claim to be “scientific skeptics” actually are.

    And you fail miserably. However, you do succeed in showing just how poorly informed, biased, and uninterested in truth you are.

    And you amuse us, which is worth something.

  23. Still waiting for answers to the questions I posed in the very first comment:

    phoodoo,

    Have you read the paper? Can you identify a flaw in it?

  24. Hobbes,

    Thanks for your summary. I’m afraid it will be wasted on phoodoo, however.

  25. keiths:
    Hobbes,

    Thanks for your summary.I’m afraid it will be wasted on phoodoo, however.

    You are thanking him for showing that you have not read the paper , and that you are incapable of commenting on it? For exposing that you are a fraud?

    What were the terms keiths?

  26. phoodoo,

    You are thanking him for showing that you have not read the paper…

    Hobbes has read the paper, therefore I can’t have? What is this, the Law of Conservation of Readers?

    Keep it up, phoodoo. You’re doing great.

  27. Hobbes,

    I am glad you put in a little more effort than Keiths (none) but lets look at bit more closely at what is being said here. First:

    “The goal was to see if interest in “status” goods is affected by inequality. But since interest in status goods may also be affected by other factors as well, the authors did a regression analysis to control for overall income levels, population, urban population, and numbers of domestic vs. foreign born residents, in order to isolate the effects of inequality from these other factors.

    They thus derived a metric for each state that represents income inequality adjusted to account for those confounding factors.”

    I see one problem here already. If they already had the Gini index to work off of, then skewing the results of this by separating out foreign born, from non-foreign born, high population, rural areas, etc..you are now creating a separate index of people that doesn’t necessarily reflect their income equality as much as it does their lifestyle choices.

    Problem number one, but there are more that I will continue with later…

  28. Rich:

    Anyone care to write the last sentence of the paper here?

    Let’s let phoodoo do it. Maybe in the process of typing it in, he’ll actually ponder what it says.

  29. Hobbes,

    Secondly, You say that, “The result was that of the 40 terms most positively correlated with inequality distribution by state, 29 (72.5)% were viewed as referring to status goods by the majority of the test subjects.”

    Yet the abstract says this:

    “We found that of the 40 search terms used more frequently in states with greater income inequality, more than 70% were classified as referring to status goods

    So I am not clear, are you saying this is a state by state comparison, or a comparison of inequality within the state?

  30. phoodoo:
    Hobbes,
    I see oneproblem here already. If they already had the Gini index to work off of,then skewing the results of this by separating out foreign born, from non-foreign born, high population, rural areas, etc..you are now creating a separate index of people that doesn’t necessarily reflect their income equality as much as it does their lifestyle choices.

    I’m afraid you have it exactly backwards here. The purpose of the regression to control for confounding variables is to eliminate the effects of what you refer to as “lifestyle choices”, and leave a measure of inequality that is independent of those effects. For example, a population that has income distributed in a particular way between a high of 200k per year and a low of100k per year may have the same Gini index as a population whose incomes are distributed in a similar way, but between a high of 100k and a low of50k. But the former population is likely to have much different lifestyles and attitudes than the latter population because of the difference in absolute income levels. The regression is eliminate that part of the results that is due to the “lifestyle” variable (in this case absolute income level) and leave only that part of the result that is due to the variable of interest, inequality.

  31. Hobbes,

    Its looking to me more and more that what we really are saying is that in those populations where the population is relatively well off, they look at luxury items more.

    Isn’t that was is really being explained here?

  32. phoodoo,

    You crack me up. If I haven’t read the paper, how did I know about the ‘Open Practices’ section?

    And if you’ve read it, why do you keep quoting the abstract and nothing but the abstract?

    As Rich says:

    How pathetic would it be to ‘critique’ a paper on the abstract alone?

    You’re terrible at bluffing. Keep up the good work.

    Now go hock that watch and buy the paper.

  33. Hobbes,

    Of course the claim is that is what is being done. The question is what else are you changing by weeding out variables like where someone was born and if they live in a rural versus city environment.

    This is their formula, and yet they are testing to prove their hypothesis. Whose to say that their formula is the correct one?

  34. Hobbes,

    So again, are we looking at state by state comparisons of the inhabitants search habits, or are we looking at the search habits within the state given variables of where the people live, where they were born, if they prefer city living compared to rural life….

  35. phoodoo, to Hobbes:

    Its looking to me more and more that what we really are saying is that in those populations where the population is relatively well off, they look at luxury items more.

    Isn’t that was is really being explained here?

    No, phoodoo, and this part is even in the abstract, which you supposedly have read:

    We used Google Correlate to find search terms that correlated with our measure of income inequality, and we controlled for income and other socioeconomic factors.

    [Emphasis added]

  36. phoodoo:
    Hobbes,

    Secondly, You say that, “The result was that of the 40 terms most positively correlated with inequality distribution by state, 29 (72.5)% were viewed as referring to status goods by the majority of the test subjects.”

    Yet the abstract says this:

    “We found that of the 40 search terms used more frequently in states with greater income inequality, more than 70% were classified as referring to status goods

    So I am not clear, are you saying this is a state by state comparison, or a comparison of inequality within the state?

    It was not a comparison of inequality within a state. Every state was assigned an inequality value. For simplicity, imagine there are only three states, and those three states had inequality values of 1, 2, and 3, respectively. That data was uploaded to Google Correlate which then determined what search terms were distributed by frequency in a similar manner across the states. So, for example, if the term Gucci was searched for 100 times in the first state, 200 times in the second, and 300 times in the third, then the distribution (1x, 2x, 3x) is exactly the same as the inequality values, hence perfect correlation and this term would be flagged. It the term “backache” was searched for 500 times in each of the three states, then there would be no correlation between this term and the inequality distribution across the three states, so this term wouldn’t be flagged (even though it was searched for more times that Gucci). If the term “picnic” was searched for 300 times in the first state, 200 times in the second state, and 100 times in the third state, this would be a perfect negative correlation and the term would be flagged because it’s distributed across states in the exact opposite way as the inequality.

  37. Hobbes,

    Can you first show us the inequality rankings they gave each state? Or maybe keiths can if he read the paper?

  38. If we have an environment where most of the residents are wealthy, is that state a high equality state? And does that state demonstrate a low propensity to search for items of luxury status?

    I am pretty sure the answer is not that the places where all residents have high income do little searching for luxury items.

  39. phoodoo:
    Hobbes,

    Its looking to me more and more that what we really are saying is that in those populations where the population is relatively well off, they look at luxury items more.

    Isn’t that was is really being explained here?

    No. The purpose of controlling for confounding variables is to do exactly the opposite of what you say. As in my previous example two populations with the same number of people, one wealthy and one poor, can have similar Gini indices if income is distributed similarly within the two populations. Clearly, as you suggest, the wealthier population is more likely to search more frequently for luxury items, so just using the Gini index would give you a potentially false conclusion that searching for luxury items is not related to inequality since we have a difference in frequency of searches between the populations, but similar Gini indices. The point of controlling for income is to eliminate that part of the search difference that is due purely to income level differences and to leave only that part of the search difference that is due to the difference in income distribution.

  40. phoodoo:
    Hobbes,

    Can you first show us the inequality rankings they gave each state?Or maybe keiths can if he read the paper?

    The values are the “inequality residuals” after the regression to control for confounding variables. The larger negative values mean lower inequality. The larger positive values mean higher inequality. So the list goes from lower to higher inequality.

    State Residual_GINI
    Utah -0.02769
    California -0.02413
    Alaska -0.02396
    Idaho -0.02376
    Wyoming -0.02121
    Hawaii -0.0204
    Iowa -0.0188
    Wisconsin -0.01693
    Nevada -0.01616
    Indiana -0.0146
    Nebraska -0.01277
    Washington -0.01073
    New Hampshire -0.01015
    Vermont -0.0087
    Montana -0.00778
    Texas -0.0076
    Maine -0.00758
    Delaware -0.00565
    Minnesota -0.00551
    Florida -0.00443
    South Dakota -0.00372
    Kansas -0.00357
    Ohio -0.00335
    Arizona -0.00298
    Michigan -0.00289
    Oregon -0.00249
    Maryland -0.00137
    Missouri 0.0026
    Pennsylvania 0.00328
    North Dakota 0.00543
    Illinois 0.00594
    Oklahoma 0.00597
    North Carolina 0.00648
    Arkansas 0.0065
    Colorado 0.00711
    West Virginia 0.00766
    South Carolina 0.01014
    Virginia 0.01107
    New Mexico 0.01111
    New Jersey 0.0114
    Georgia 0.012
    Kentucky 0.01286
    Rhode Island 0.01408
    Tennessee 0.01539
    Alabama 0.01693
    Mississippi 0.02257
    Massachusetts 0.02431
    New York 0.02584
    Louisiana 0.02771
    Connecticut 0.04252

  41. phoodoo:
    If we have an environment where most of the residents are wealthy, is that state a high equality state?And does that state demonstrate a low propensity to search for items of luxury status?

    If everyone in a state makes the same income, 10k per year, then they are dirt poor but the state has zero inequality. If everyone in a state makes the same income,1M per year, then they are all wealthy but the state also has zero inequality. If half the people in a state make 10k and the other half all make500k, then there is high inequality. If half the people in the state make 500k and the other half make2.5M, then there is also high inequality. This is why researchers control for confounding variables: states can have vastly different levels of wealth, yet similar levels of inequality, either high or low. Those different levels of absolute wealth can greatly affect the results and drown out any effect caused by inequality. This is what researchers painstaking design their methods to avoid.

  42. phoodoo:
    Hobbes,

    This is their formula, and yet they are testing to prove their hypothesis.Whose to say that their formula is the correct one?

    Yes, this is their formula. The goal of research is confident understanding. Toward that end they constructed a model that attempts to determine whether income inequality is correlated to interest in status goods. They made choices in their analysis. They chose the Gini index, which is not the only measure of inequality that people use. They chose to control for four other confounding variables using a particular statistical method. They could have used other methods and they could have included other possible confounding variables. Their choices were no doubt based on their experience, expertise, and reasoning as to which confounding variables were likely to have the most impact on the results.

    Who is to say their formula is the correct one? Actually who is to say it is “a good one” – there is no “correct one” since all models are approximations that are either better or worse? Anyone who wants to challenge, verify, or improve upon their work is free to do so. They published their method and explained it. They made their data available for others to see, download, and manipulate. If someone can show any error in their methods or calculations, the same journal would no doubt be happy to publish that analysis. If someone can add to, modify, extend, or completely supplant their analysis with a better one, and explain why it is better, in order to improve upon their results, they are free to do so and the same journal would no doubt be happy to publish that as well. That is the way science works.

Leave a Reply