Viewing 11 posts - 1 through 11 (of 11 total)
  • stats help needed – significance tests
  • Premier Icon joemarshall
    Free Member

    I’ve got some data:

    It’s basically a small N study, where people are tested for something, then are given some help, and then tested again.

    So I have something like this:


    before | after
    yes 3 | 9
    no 13 | 7

    and I’m trying to work out what statistical test I need to use.

    The closest I can get is Fisher’s Exact Test, but I think I can’t use that because the before/after results aren’t independent?

    Any statisticians here?

    Joe

    Premier Icon joemarshall
    Free Member

    Aha, I think I have *just* enough data to use McNemar’s test.

    Although what the hell significance means in this small an N study I don’t know.

    Joe

    Premier Icon joemarshall
    Free Member

    or maybe I can’t. This is doing my head in.

    Premier Icon zokes
    Free Member

    chi squared?

    Premier Icon joemarshall
    Free Member

    chi squared?

    Not for things with this small a number of samples, and not where the two tests are correlated (it’s a before-after study).

    Joe

    Premier Icon joe1983
    Free Member

    Guessing you can’t test for structural break due to only having two data points?

    Premier Icon neverfastenuff
    Free Member

    Is it attribute data or variable data ?

    Premier Icon joemarshall
    Free Member

    Is it attribute data or variable data ?

    It’s basically –

    People are first given a problem to solve with no help.

    There is a tactic that makes this class of problem easier to solve, but to a lot of people it seems like cheating.

    After a first failure (all except one of them failed) We give them a hint, saying that they can do anything they like to solve the problem, then next we see if they use the easy tactic or not.

    It’s about rules and expectations and stuff.

    It is pretty obvious when you’re running this that it has a significant effect, but I’d really like to have some number to back it up.

    Joe

    Premier Icon neverfastenuff
    Free Member

    There seems to me to be no Statistical in the data.

    A simple Ratio / Percentage is the answer

    Premier Icon joemarshall
    Free Member

    There seems to me to be no Statistical in the data.

    Huh? There must be tons of trials like this – test, intervention, test. Surely they do some kind of statistical test for it?

    Obviously it isn’t a properly controlled experiment – so I can’t claim that this was definitely the thing that made the difference occur, but surely there is some way to test whether something has changed?

    Joe

    Premier Icon joemarshall
    Free Member

    Aha –

    Got it. I write it out in terms of before/after changes like this:


    before
    y n
    a
    f y 3 6
    t
    e n 0 7
    r

    There are 6 positive changes, and 0 negative changes.

    I can apply a simple sign test to it, giving a significance of P<0.031

    We are talking stupidly silly small N here though, but I guess that is the nature of small scale pilot studies.

    Ta da!

    Joe

Viewing 11 posts - 1 through 11 (of 11 total)

The topic ‘stats help needed – significance tests’ is closed to new replies.