significance test
power analysis significance test

Power Analysis - Role of Tails

The significance test is always defined as either one-tailed or two-tailed. A two-tailed test is a test that will be interpreted if the effect meets the criterion for significance and falls in either direction, and is appropriate for the vast majority of research studies. A one-tailed test is a test that will be interpreted only if the effect meets the criterion for significance and falls in the observed direction (i.e. the treatment improves the cure rate), and is appropriate only for a specific type of research question.

Cohen gives the following example of a one-tailed test. An assembly line is currently using a particular process (A). We are planning to evaluate an alternate process (B) which would be expensive to implement but could yield substantial savings if it works as expected. The test has three possible outcomes: (1) Process A is better; (2) There is no difference between the two; (3) Process (B) is better. However, for our purposes, outcomes (1) and (2) are functionally equivalent since either would lead us to maintain the status quo. Put another way, we have no need to distinguish between outcomes (1) and (2).

A one-tailed test should be used only in a study in which, as in this example, an effect in the unexpected direction is functionally equivalent to no effect. It is not appropriate to use a one-tailed test simply because one is able to specify the expected direction of the effect prior to running the study. In medicine, for example, we typically expect that the new procedure will improve the cure rate, but a finding that it decreases the cure rate would still be important, since it would demonstrate a possible flaw in the underlying theory.

For a given effect size, sample size, and alpha, a one-tailed test is more powerful than a two-tailed test (a one-tailed test with alpha set at .05 has approximately the same power as a two-tailed test with alpha set at .10). However, the number of tails should be set based on the substantive issue (will an effect in the reverse direction be meaningful). In general, it would not be appropriate to run a test as one-tailed rather than two-tailed as a means of increasing power. (Power is higher for the one-tailed test only under the assumption that the observed effect falls in the expected direction. When the test is one tailed, power for an effect in the reverse direction is nil).

Previous  |  Next



Home   ·   What's Power Analysis?   ·   Precision Analysis   ·   In Context   ·   Significance Test
Software Specifications    ·    Product Tour     ·     Site Map