Precision - Overview
The discussion to this point has focused on power analysis, which is the logical precursor to a test of significance. If the researcher designing a study to test the null hypothesis, then the study design should ensure, to a high degree of certainty, that the study will be able to provide an adequate (i.e. powerful) testing of the null hypothesis.
The study may be designed with another goal as well. In addition to (or instead of) testing the null hypothesis the researcher might use the study to estimate the magnitude of the effect - to report, for example that the treatment increases the cure rate by 10 points, or by 20 points, or by 30 points. In this case, study planning would focus not on the study's ability to reject the null hypothesis but rather on the precision with which it will allow us to estimate the magnitude of the effect.
Assume, for example, that we are planning to compare the response rates for treatments, and anticipate that these rates will differ from each other by 20 percentage points. We would like to be able to report the rate difference with a precision of plus/minus 10 points.
The precision with which we will be able to report the rate difference is a function of the confidence level required, the sample size, and the variance of the outcome index. Except in the indirect manner discussed below, it is not affected by the effect size.
Previous | Next