Viewing 11 posts - 1 through 11 (of 11 total)
• stats help needed – significance tests
• joemarshall
Free Member

I’ve got some data:

It’s basically a small N study, where people are tested for something, then are given some help, and then tested again.

So I have something like this:

``` before | after yes 3 | 9 no 13 | 7 ```

and I’m trying to work out what statistical test I need to use.

The closest I can get is Fisher’s Exact Test, but I think I can’t use that because the before/after results aren’t independent?

Any statisticians here?

Joe

joemarshall
Free Member

Aha, I think I have *just* enough data to use McNemar’s test.

Although what the hell significance means in this small an N study I don’t know.

Joe

joemarshall
Free Member

or maybe I can’t. This is doing my head in.

zokes
Free Member

chi squared?

joemarshall
Free Member

chi squared?

Not for things with this small a number of samples, and not where the two tests are correlated (it’s a before-after study).

Joe

joe1983
Free Member

Guessing you can’t test for structural break due to only having two data points?

neverfastenuff
Free Member

Is it attribute data or variable data ?

joemarshall
Free Member

Is it attribute data or variable data ?

It’s basically –

People are first given a problem to solve with no help.

There is a tactic that makes this class of problem easier to solve, but to a lot of people it seems like cheating.

After a first failure (all except one of them failed) We give them a hint, saying that they can do anything they like to solve the problem, then next we see if they use the easy tactic or not.

It’s about rules and expectations and stuff.

It is pretty obvious when you’re running this that it has a significant effect, but I’d really like to have some number to back it up.

Joe

neverfastenuff
Free Member

There seems to me to be no Statistical in the data.

A simple Ratio / Percentage is the answer

joemarshall
Free Member

There seems to me to be no Statistical in the data.

Huh? There must be tons of trials like this – test, intervention, test. Surely they do some kind of statistical test for it?

Obviously it isn’t a properly controlled experiment – so I can’t claim that this was definitely the thing that made the difference occur, but surely there is some way to test whether something has changed?

Joe

joemarshall
Free Member

Aha –

Got it. I write it out in terms of before/after changes like this:

``` before y n a f y 3 6 t e n 0 7 r```

There are 6 positive changes, and 0 negative changes.

I can apply a simple sign test to it, giving a significance of P<0.031

We are talking stupidly silly small N here though, but I guess that is the nature of small scale pilot studies.

Ta da!

Joe

Viewing 11 posts - 1 through 11 (of 11 total)

The topic ‘stats help needed – significance tests’ is closed to new replies.