At 04:12 PM 1/16/2002 -0800, Horace Jeffery Hodges wrote:
>How can the "P" value tell us whether or not we have
>enough data? This seems at odds with what you state in
>the following passage about the 'arbitrariness' in
>selecting a proper "P" value.
The 'p' value is a function of the sample size. The
larger the sample size, the better the 'p' value. The
notion of "enough" is where the arbitrariness comes in.
>(By the way, are we using t-tests or z-tests?)
The correlations, it is a t-test.
>> The selection of the proper 'p' value is arbitrary
>> (which is still a bit of a scandal in the
>> profession) and a convention has been adopted over
>> the years that a statistically significant is one in
>> which there is one chance in 1/20 that the null
>> hypothesis may still be correct given the 'r' value
>> and the sample size.
>This is the "P" for a single set of data? Or for a
>single correlation? Or something else?
The 'p' value is for a single test. However, if you
run hundreds of tests, as we do in this correlation
analysis, a 'p' value as 1/20 will generate far too
many false positives.
>> Because we have 19 different sets of data, for a
>> total of 171 different correlations, it is clear
>> the 1/20 level of significance is much too high,
>> because just from random chance we would 171/20 = 8
>> apparently significant results. A good rule of thumb
>> in such cases is to divide the 1/20 level by the
>> number of different comparisons, which yields the
>> P<=0.0003 significance level I and others have been
>So ... you're dividing 1/20 by 171?
>Is this your own personal rule of thumb? Or a rule of
>thumb for statisticians generally? Either way, how was
>this rule of thumb arrived at -- what justifies it?
It's not my personal rule of thumb; other people use it.
It is arrived out to make the false positive rate the
same for the whole suite of tests.
Stephen C. Carlson mailto:scarlson@...
Synoptic Problem Home Page http://www.mindspring.com/~scarlson/synopt/
"Poetry speaks of aspirations, and songs chant the words." Shujing 2.35
Synoptic-L Homepage: http://www.bham.ac.uk/theology/synoptic-l
List Owner: Synoptic-L-Owner@...