 This topic has 10 replies, 4 voices, and was last updated 11 years ago by joemarshall.

stats help needed – significance tests

joemarshallFree Member
I’ve got some data:
It’s basically a small N study, where people are tested for something, then are given some help, and then tested again.
So I have something like this:
before  after
yes 3  9
no 13  7
and I’m trying to work out what statistical test I need to use.
The closest I can get is Fisher’s Exact Test, but I think I can’t use that because the before/after results aren’t independent?
Any statisticians here?
Joe
Posted 12 years agojoemarshallFree MemberAha, I think I have *just* enough data to use McNemar’s test.
Although what the hell significance means in this small an N study I don’t know.
Joe
Posted 12 years agojoemarshallFree Memberor maybe I can’t. This is doing my head in.
Posted 12 years agozokesFree Memberchi squared?
Posted 12 years agojoemarshallFree Memberchi squared?
Not for things with this small a number of samples, and not where the two tests are correlated (it’s a beforeafter study).
Joe
Posted 12 years agojoe1983Free MemberGuessing you can’t test for structural break due to only having two data points?
Posted 12 years agoneverfastenuffFree MemberIs it attribute data or variable data ?
Posted 12 years agojoemarshallFree MemberIs it attribute data or variable data ?
It’s basically –
People are first given a problem to solve with no help.
There is a tactic that makes this class of problem easier to solve, but to a lot of people it seems like cheating.
After a first failure (all except one of them failed) We give them a hint, saying that they can do anything they like to solve the problem, then next we see if they use the easy tactic or not.
It’s about rules and expectations and stuff.
It is pretty obvious when you’re running this that it has a significant effect, but I’d really like to have some number to back it up.
Joe
Posted 12 years agoneverfastenuffFree MemberThere seems to me to be no Statistical in the data.
A simple Ratio / Percentage is the answer
Posted 12 years agojoemarshallFree MemberThere seems to me to be no Statistical in the data.
Huh? There must be tons of trials like this – test, intervention, test. Surely they do some kind of statistical test for it?
Obviously it isn’t a properly controlled experiment – so I can’t claim that this was definitely the thing that made the difference occur, but surely there is some way to test whether something has changed?
Joe
Posted 12 years agojoemarshallFree MemberAha –
Got it. I write it out in terms of before/after changes like this:
before
y n
a
f y 3 6
t
e n 0 7
rThere are 6 positive changes, and 0 negative changes.
I can apply a simple sign test to it, giving a significance of P<0.031
We are talking stupidly silly small N here though, but I guess that is the nature of small scale pilot studies.
Ta da!
Joe
Posted 11 years ago
The topic ‘stats help needed – significance tests’ is closed to new replies.