Multiple Comparisons: Statistics

ClusterSeer offers two ways to evaluate your results after multiple testing, a variety of significance level adjustments and a combined P-value for all the tests.

Adjusted significance levels

 adjustments.gif

The Bonferroni adjustment is the classical approach, but it is known to be overly conservative. Recently, other approaches have been developed that are less conservative and have more power for a large number of comparisons (Sarkar and Chang 1997), such as the Sidak (1967), Simes (1986), and Modified Holm's (Holland and Copenhaver 1987) adjustments.

These approaches provide you with adjusted significance level, alphacrit.gif (c for critical level). This new critical level reflects your initial significance level (alpha.gif) and the number of comparisons conducted at that initial significance level (j). The Simes and Holm's adjustments are performed for each test, sequentially ordered from lowest to highest P-value, with i denoting the sequential index (range: 1..j) for the individual test considered.

Combined P-values

ClusterSeer will also provide a combined P-value for all tests performed at one initial alpha level. This is accomplished for Bonferroni and Simes/Holm's adjustments.

combined.gif

In this case, Pc denotes the combined P-value for all tests, Pi the value for an individual test, j the number of comparisons, and i the individual test considered.

See Also