Statistical Hypothesis Testing

Posted on by

Wikipedia:

https://en.wikipedia.org/wiki/Statistical_hypothesis_testing

The page is quite good, but I’d like the Math Gods that write here to give their thoughts on the relative strengths and applicability of the various models (T test, Chi squared and F-test / Anova)

I’d also welcome discussion on the null hypothesis and what that rejection of it does and doesn’t entail.

Questions welcomed, Design detectives feel free to contribute.

54 thoughts on “Statistical Hypothesis Testing”

  1. Oh, I just saw you mentioned two groups. I thought you were talking about two tails.

    Need to think some more, but I take your point.

    I guess mine was that they are all applications of the GLM.

    Could you still get an F value by dividing the mean model sum of squares (where the model sum of squares is the distance between the mean and the test value squared) by the mean error sum of squares (where error sum of squares is the squared deviations of the data from the test value)?

    I think. Never tried it. And I’m not sure what the dfs would be….

    ETA: the model df would be 1 and the error df N-1. So yeah, the F value will still be the square of the t-value.

  2. Elizabeth:
    Need to think about that…

    I agree that the F distribution only has one tail.Not sure it makes it a different test though – wouldn’t you just move the alpha criterion to .1?

    No, that is not good enough. If the t-test is set up so that it tests whether Group 1 has a higher mean than Group 2, a strong result in that direction does correspond to a high value of F. But if the result is in the other direction, with Group 1 having a substantially lower mean than Group 2, then the t-test will not reject the null hypothesis, but the value of F will be large and thus will be out in the upper tail of the F distribution.

  3. OK, have to think about that 🙂

    ETA: OK, yes thought about it.

    Yes, that’s the interesting thing about F tests generally though – often the F is not a direct test of your hypothesis. So I take your point.

  4. Elizabeth:

    …

    Yes, that’s the interesting thing about F tests generally though – often the F is not a direct test of your hypothesis. SoI take your point.

    Indeed.

    If you have three groups, G1, G2, and G3, and use an F test to see whether there is evidence that their means are different, it can be inefficient. For example, if the most likely difference is that G3 has a different mean, the F test will waste some of its power trying to detect whether G2 is different from G1.

Leave a Reply