On the false discovery rate and an asymptotically optimal rejection curve

Statistics & Probability Letters, 2008

The Benjamini-Hochberg step-up procedure controls the false discovery rate (FDR) provided the test statistics have a certain positive regression dependency. We show that this procedure controls the FDR under a weaker property and is optimal in the sense that its critical constants are uniformly greater than those of any step-up procedure with the FDR controlling property.

2008

Applicationes Mathematicae, 2009

Institute of Mathematical Statistics Lecture Notes - Monograph Series, 2006

Consider the problem of testing multiple null hypotheses. A classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (F W ER), the probability of even one false rejection. However, if s is large, control of the F W ER is so stringent that the ability of a procedure which controls the F W ER to detect false null hypotheses is limited. Consequently, it is desirable to consider other measures of error control. We will consider methods based on control of the false discovery proportion (F DP) defined by the number of false rejections divided by the total number of rejections (defined to be 0 if there are no rejections). The false discovery rate proposed by Benjamini and Hochberg (1995) controls E(F DP). Here, we construct methods such that, for any γ and α, P {F DP > γ} ≤ α. Based on p-values of individual tests, we consider stepdown procedures that control the F DP , without imposing dependence assumptions on the joint distribution of the p-values. A greatly improved version of a method given in Lehmann and Romano [10] is derived and generalized to provide a means by which any sequence of nondecreasing constants can be rescaled to ensure control of the F DP. We also provide a stepdown procedure that controls the F DR under a dependence assumption.

The Annals of Statistics, 2009

The concept of k-FWER has received much attention lately as an appropriate error rate for multiple testing when one seeks to control at least k false rejections, for some fixed k ≥ 1. A less conservative notion, the k-FDR, has been introduced very recently by Sarkar [Ann. Statist. 34 (2006) 394-415], generalizing the false discovery rate of Benjamini and Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289-300]. In this article, we bring newer insight to the k-FDR considering a mixture model involving independent p-values before motivating the developments of some new procedures that control it. We prove the k-FDR control of the proposed methods under a slightly weaker condition than in the mixture model. We provide numerical evidence of the proposed methods' superior power performance over some k-FWER and k-FDR methods. Finally, we apply our methods to a real data set.

The Annals of Statistics, 2007

Some effort has been undertaken over the last decade to provide conditions for the control of the false discovery rate by the linear step-up procedure (LSU) for testing n hypotheses when test statistics are dependent. In this paper we investigate the expected error rate (EER) and the false discovery rate (FDR) in some extreme parameter configurations when n tends to infinity for test statistics being exchangeable under null hypotheses. All results are derived in terms of p-values. In a general setup we present a series of results concerning the interrelation of Simes' rejection curve and the (limiting) empirical distribution function of the p-values. Main objects under investigation are largest (limiting) crossing points between these functions, which play a key role in deriving explicit formulas for EER and FDR. As specific examples we investigate equi-correlated normal and t-variables in more detail and compute the limiting EER and FDR theoretically and numerically. A surprising limit behavior occurs if these models tend to independence.

2012

The False Discovery Rate (FDR) was proposed in Benjamini and Hochberg (1995) as a powerful approach to the multiplicity problem that does not require strong control of the familywise error rate (FWER). The original approach was developed for independent test statistics and was later extended to dependent statistics in Benjamini and Yekutieli (2001) and Yekutieli (2008). In this paper we extend the existing results by showing that the assumptions on the dependence structure among classes of univariate statistics, that lead to the FDR control, may be represented by specific copulas.

Biometrical Journal, 2001

The paper is concerned with expected type I errors of some stepwise multiple test procedures based on independent p-values controlling the so-called false discovery rate (FDR). We derive an asymptotic result for the supremum of the expected type I error rate (EER) when the number of hypotheses tends to infinity. Among others, it will be shown that when the original Benjamini-Hochberg step-up procedure controls the FDR at level a, its EER may approach a value being slightly larger than a=4 when the number of hypotheses increases. Moreover, we derive some least favourable parameter configuration results, some bounds for the FDR and the EER as well as easily computable formulae for the familywise error rate (FWER) of two FDR-controlling procedures. Finally, we discuss some undesirable properties of the FDR concept, especially the problem of cheating.

The Annals of Statistics, 2014

The probability of false discovery proportion (FDP) exceeding γ ∈ [0, 1), defined as γ-FDP, has received much attention as a measure of false discoveries in multiple testing. Although this measure has received acceptance due to its relevance under dependency, not much progress has been made yet advancing its theory under such dependency in a nonasymptotic setting, which motivates our research in this article. We provide a larger class of procedures containing the stepup analog of, and hence more powerful than, the stepdown procedure in Lehmann and Romano [Ann. Statist. 33 (2005) 1138-1154] controlling the γ-FDP under similar positive dependence condition assumed in that paper. We offer better alternatives of the stepdown and stepup procedures in Romano and Shaikh [IMS Lecture Notes Monogr. Ser. 49 (2006a) 33-50, Ann. Statist. 34 (2006b) 1850-1873] using pairwise joint distributions of the null p-values. We generalize the notion of γ-FDP making it appropriate in situations where one is willing to tolerate a few false rejections or, due to high dependency, some false rejections are inevitable, and provide methods that control this generalized γ-FDP in two different scenarios: (i) only the marginal p-values are available and (ii) the marginal p-values as well as the common pairwise joint distributions of the null p-values are available, and assuming both positive dependence and arbitrary dependence conditions on the p-values in each scenario. Our theoretical findings are being supported through numerical studies.

TEST, 2008

We are extremely appreciative of the insightful comments made by all the responders. The goal of constructing useful multiple testing methods which control the false discovery rate and other measures of error is currently a thriving and important area of research. On the one hand, the bootstrap method presented in the present work seems to work quite well and is supported by some theoretical analysis. On the other hand, many more important practical, computational, and mathematical questions remain, some of which are addressed by the responders and which we touch upon below. We also appreciate the added references, which help to provide a more thorough discussion of the available methods. Our paper was the development of a particular methodology and was by no means a comprehensive account of the burgeoning FDR literature.