key: cord-0170393-t20slxt9
authors: Tan, Nelvin; Tan, Way; Scarlett, Jonathan
title: Performance Bounds for Group Testing With Doubly-Regular Designs
date: 2022-01-11
journal: nan
DOI: nan
sha: 353d113cffc9e6b01ca13c371e92361d71feb68b
doc_id: 170393
cord_uid: t20slxt9

In the group testing problem, the goal is to identify a subset of defective items within a larger set of items based on tests whose outcomes indicate whether any defective item is present. This problem is relevant in areas such as medical testing, DNA sequencing, and communications. In this paper, we study a doubly-regular design in which the number of tests-per-item and the number of items-per-test are fixed. We analyze the performance of this test design alongside the Definite Defectives (DD) decoding algorithm in several settings, namely, (i) the sub-linear regime $k=o(n)$ with exact recovery, (ii) the linear regime $k=Theta(n)$ with approximate recovery, and (iii) the size-constrained setting, where the number of items per test is constrained. Under setting (i), we show that our design together with the DD algorithm, matches an existing achievability result for the DD algorithm with the near-constant tests-per-item design, which is known to be asymptotically optimal in broad scaling regimes. Under setting (ii), we provide novel approximate recovery bounds that complement a hardness result regarding exact recovery. Lastly, under setting (iii), we improve on the best known upper and lower bounds in scaling regimes where the maximum allowed test size grows with the total number of items.

In the group testing problem, the goal is to identify a small subset of defective items of size k within a larger set of items of size n, based on a number T of tests. This problem is relevant in areas such as medical testing, DNA sequencing, and communication protocols [2, Sec. 1.7] , and has recently found utility in COVID-19 testing [3] .

In non-adaptive group testing, the placements of items into test can be represented by a binary test matrix of size T × n. The strongest known theoretical guarantees are based on the idea of generating this matrix at random and analyzing the average performance. Starting from early studies of group testing, a line of works led to a detailed understanding of the i.i.d. test design [4] - [7] , and more recently, further improvements were shown for a nearconstant tests-per-item design [8] , [9] , whose asymptotic optimality in sub-linear sparsity regimes was established in [10] . Recently, there has been increasing evidence that doubly-regular designs (i.e., both constant tests-per-item and items-per-test) also play a crucial role in various settings of interest:

• The work of Mezárd et al. [11] uses heuristic arguments from statistical physics to suggest that doubly-regular designs achieve the same optimal threshold as the near-constant tests-per-item design when k = Θ(n θ ), at least when θ is not too small.

• In constrained settings where the number of items per test cannot exceed a pre-specified threshold, doublyregular designs have been used to obtain performance bounds that appear to be difficult or impossible to obtain using the other designs mentioned above [12] , [13] .

• In the linear sparsity regime (i.e., k = Θ(n)), various two-stage adaptive designs were studied in [14] , and using a doubly-regular design in the first stage led to strict improvements over the other designs.

In this paper, motivated by these developments, we seek to provide a more detailed understanding of non-adaptive doubly-constant test designs, particularly when paired with the Definite Defectives (DD) algorithm [6] . Briefly, our contributions are as follows: (i) For θ ∈ 1 2 , 1 , we rigorously prove the above-mentioned result shown heuristically in [11] , albeit with a slightly different version of the doubly-regular design; (ii) We establish new performance bounds for non-adaptive group testing in the linear regime (k = Θ(n)) with some false negatives allowed in the reconstruction, complementing strong impossibility results for exact recovery [15] , [16] ; (iii) We provide improved upper and lower bounds on the number of tests for the constrained setting with at most ρ items per test. Our bounds apply to general scaling regimes beyond the regime ρ = O(1) recently studied in [13] , and our consideration of the DD algorithm leads to strict improvements over the COMP algorithm considered in [12] .

Let n denote the number of items, which we label as [n] = {1, . . . , n}. Let K ⊂ [n] denote the fixed set of defective items, and let k = |K| be the number of defective items. We adopt the combinatorial prior [2] , where K is chosen uniformly at random from n k sets of size k. We let T = T (n) be the number of tests performed. The i-th test takes the form

where the test vector X (i) = X (i) 1 , . . . , X (i) n -Linear: We have k = pn for some prevalence rate p ∈ (0, 1). This regime may potentially be of greater relevance in certain practical situations, e.g., with p representing the prevalence of a disease.

• Regarding the constraints (or lack thereof), we consider the following:

-Unconstrained: There is no restriction on the number of tests-per-item or the number of items-per-test in the test design.

-Size-constrained: Tests are size-constrained and thus contain no more than ρ = Θ n k β items per test, for some constant β ∈ (0, 1). Note that if each test comprises of Θ(n/k) items, then Θ(k log n) tests suffice for group testing algorithms with asymptotically vanishing error probability [6] - [8] , [17] . One can alternatively use exactly n tests via one-by-one testing, and it has recently been shown that taking the better of the two (i.e., Θ(min{k log n, n}) tests) gives asymptotically optimal scaling [16] . Hence, to avoid essentially reducing to the unconstrained setting, the parameter regime of interest in the size-constrained setting is ρ = o(n/k), which justifies the scaling ρ = Θ n k β .

• Regarding the recovery criteria, we consider the following:

-Exact recovery: We seek to develop a testing strategy and decoder that produces an estimate K such that the error probability P e = P K = K is asymptotically vanishing as n → ∞.

-Approximate recovery: We seek to characterize the per-item false-positive rate (FPR) and false-negative rate (FNR), which are defined as the probability that a non-defective item (picked uniformly at random)

is declared defective (i.e., FPR = E[| K\K|] |[n]\K| ), and the probability that a defective item (picked uniformly at random) is declared non-defective (i.e., FNR = E[|K\ K|] |K| ), respectively.

With these definitions in place, the settings that we focus on are (i) the unconstrained sub-linear regime with exact recovery, (ii) the unconstrained linear regime with approximate recovery, and (iii) the size-constrained sub-linear regime with exact recovery. More specifically, in the second of these, we only consider the FNR; the FPR is not considered, since the DD algorithm that we study never declares a non-defective item to be defective. 1 While the above definitions lead to 2 3 = 8 possible settings of interest, our focus is on three that we believe to be suitably representative and of the most interest given what is already known in existing works (e.g., due to the hardness of exact recovery in the linear regime [15] , [16] ).

Notation. Throughout the paper, the function log(·) has base e, and we make use of Bachmann-Landau asymptotic notation (i.e., O, o, Ω, ω, Θ).

We focus on non-adaptive and noiseless group testing with a combinatorial prior. We begin by introducing two common decoding algorithms, Combinatorial Orthogonal Matching Pursuit (COMP) and Definite Defectives (DD), in Algorithm 1. A key difference between the COMP and DD algorithm is that the COMP algorithm produces only false positives (i.e., no false negatives), while the DD algorithm produces only false negatives (i.e., no false positives). 1 See also Appendix B for a result regarding the COMP algorithm with only false positives and no false negatives. Algorithm 1 COMP and DD algorithms [6] , [17] . Require: T tests. 1: Initialize two empty sets PD (possibly defective set) and DD (definitely defective set). 2: Label any item in a negative test as definitely non-defective, and add all remaining items to PD. 3: for each test do 4: If the test contains exactly one item from PD, then add that item to the set DD. 5: return K = PD for COMP, or K = DD for DD.

Next, we introduce some test designs [2, Section 1.3], which will be useful for purposes of comparison later:

• Bernoulli design: Each item is randomly included in each test independently with some fixed probability.

• Near-constant tests-per-item: Each item is included in some fixed number of tests, with the tests for each item chosen uniformly at random with replacement, independent from the choices for all other items.

• Constant tests-per-item: Each item is included in some fixed number of tests, with the tests for each item chosen uniformly at random without replacement, independent from the choices for all other items.

• Doubly-regular design: Both the number of tests-per-item and the number of items-per-test are fixed to prespecified values. In the following discussion, we consider the uniform distribution over all designs satisfying these conditions, though our own results will use a slightly different block-structured variant.

We proceed to review the related work for each of the settings.

In the unconstrained setting with sub-linear sparsity, the following number of tests multiplied with (1 + ) (where is any positive constant) are sufficient to attain asymptotically vanishing error probability:

• Bernoulli testing & COMP decoding [18] , [19] : ek log n ≈ 2.72k log n;

• Bernoulli testing & DD decoding [18] , [19] : e max{θ, 1 − θ}k log n ≈ 2.72 max{θ, 1 − θ}k log n;

• Near-constant tests-per-item & COMP decoding [8] : k log n log 2 2 ≈ 2.08k log n; • Near-constant tests-per-item & DD decoding [8] : max{θ,1−θ} log 2 2 k log n ≈ 2.08 max{θ, 1 − θ}k log n.

These results indicate that the near-constant tests-per-item is superior to Bernoulli testing, with the intuition being that the former avoids over-testing or under-testing items. We also observe that DD decoding is superior to COMP decoding, with the intuition being that the information from positive tests is "wasted" in the latter. Further improvements for information-theoretically optimal decoding are discussed below.

Additionally, converse results have been proven for each of these test designs: In the sub-linear regime with unconstrained tests, any decoding algorithm that uses the following number of tests multiplied by 1 − (where is arbitrarily small) is unable to attain asymptotically vanishing error probability:

• Bernoulli testing [18] :

• Near-constant tests-per-item [8] , [9] : max θ log 2 2 , 1−θ log 2 k log n. For both designs, the DD algorithm's performance matches the converse for θ ≥ 1 2 . For information-theoretically optimal decoding, exact thresholds on the required number of tests were characterized for Bernoulli testing in [7] , [18] , and for the near-constant tests-per-item design in [8] , [9] . Perhaps most importantly among these, it was shown in [10] that the above converse for the near-constant tests-per-item design extends to arbitrary non-adaptive designs, and that a matching achievability threshold holds for a certain spatially-coupled test design with efficient decoding. Hence, max θ log 2 2 , 1−θ log 2 k log n is the optimal threshold for all θ ∈ (0, 1), and for θ ≥ 1 2 the DD algorithm with near-constant tests-per-item is asymptotically optimal. Mézard et al. [11] considered doubly-regular designs, and designs with constant tests-per-item only. Their analysis used heuristics from statistical physics to suggest that such designs can improve on Bernoulli designs, and match the above near-constant tests-per-item bound for the DD algorithm. The analysis in [11] contains some non-rigorous steps; in particular, they make use of a "no short loops" assumption that is only verified for θ > 5 6 and conjectured for θ ≥ 2 3 , while experimentally being shown to fail for certain smaller values. One of our contributions in this paper is to establish a rigorous version of their result for θ ≥ 1 2 . A distinct line of works has sought designs that not only require a low number of tests, but also near-optimal decoding complexity (e.g., k poly(log n)) [20]- [25] . However, our focus in this paper is on the required number of tests, for which the existing guarantees of such algorithms contain loose constants or extra logarithmic factors.

2) Linear Regime: Under the exact recovery guarantee, it turns out that the known methods for deriving achievability bounds on T in the unconstrained sub-linear regime do not readily extend to the linear regime.

In fact, as the following results assert, individual testing is optimal for exact recovery in the linear regime.

• Weak converse [15] : In the linear regime with prevalence p ∈ (0, 1), if we use T < n − 1 tests, there exists = (p) > 0 independent of n such that P e ≥ .

• Strong converse [16] : In the linear regime with any fixed p ∈ (0, 1), if T ≤ (1 − )n for some constant > 0, then P e → 1 as n → ∞.

These results imply that individual testing is an asymptotically optimal non-adaptive strategy for exact recovery.

Hence, for this regime, we will investigate the FNR instead of the error probability.

While not immediately applicable to our setup (recall that we consider the non-adaptive setting), a recently proposed two-stage adaptive algorithm [14] turns out to be relevant: (i) Conduct non-adaptive testing and identify a set of definitely non-defective items, leaving only the possibly defective items. (ii) Conduct individual testing on the remaining possibly defective items. It was shown in [14] that using a doubly-regular design in the first stage gives us the lowest expected number of tests required to attain zero error probability, with strict improvements over the near-constant tests-per-item design. This motivates us to analyze the FNR of the DD algorithm with a doubly-regular design.

3) Size-Constrained Sub-Linear Regime: The results most relevant to this setting are described as follows, where k = Θ(n θ ) with θ ∈ [0, 1) throughout:

• Converse [12] : For ρ = Θ n k β with β ∈ [0, 1), and an arbitrarily small > 0, any non-adaptive algorithm with error probability at most requires T ≥ 1−6 1−β · n ρ , for sufficiently large n. • Improved Converse for β = 0 [13] :

for some constant > 0, then any non-adaptive algorithm fails (with 1 − o(1) probability if θ 1−θ is a non-integer, and with Ω(1) probability if θ 1−θ is an integer). • Achievability [12] : Under a doubly-regular random test design and the COMP algorithm, for ρ = Θ n k β with β ∈ [0, 1), and an arbitrarily small > 0, the error probability is asymptotically vanishing when T ≥

(1−θ)(1−β) · n ρ . • Improved Achievability for β = 0 [13] : In the regime ρ = Θ(1), under a suitably-chosen near-regular random test design (which slightly differs depending on whether or not θ ≥ 1 2 ), the error probability is asymptotically vanishing when T ≥ (1 + ) max 1 + θ 1−θ n ρ , 2n ρ+1 . The above results for β = 0 (i.e., ρ = Θ(1)) strictly improve on the results for general β, and enjoy matching achievability and converse thresholds. Thus, it is natural to ask whether we can attain similar improvements for the case that ρ = Θ (n/k) β , for β = (0, 1) (i.e., large ρ). We partially answer this question in the affirmative.

Regarding our use of a doubly-regular design, the column weight restriction is not strictly imposed by the testing constraints, but helps in avoiding "bad" events where some items are not tested enough (or even not tested at all).

For example, in the case of the COMP algorithm, the doubly-regular design helps to reduce the number of tests by a factor of O(log n) compared to i.i.d. testing.

Additional results are given for the adaptive setting in [13] , [26] , and for the noisy non-adaptive setting in [12, Section 7.2] . Another notable type of sparsity constraint is that of finitely divisible items (i.e., bounded tests-peritem) constraint, which are studied in [12] , [13] , [26] . The main reason that we do not consider such constraints here is that it was already studied extensively in [13] without any analog of the above-mentioned restrictive assumption ρ = O(1).

Our main contributions are as follows:

• Unconstrained sub-linear regime with exact recovery: We provide an achievability result for the DD algorithm with a doubly-regular design that matches the existing result of [8] for the DD algorithm with a near-constant tests-per-item design, which is asymptotically optimal when θ ≥ 1 2 . Thus, for this range of θ, we provide a rigorous counterpart to the result shown heuristically in [11] .

• Unconstrained linear regime with approximate recovery: We provide an asymptotic bound on the FNR for DD algorithm with a doubly-regular design, and further characterize the low-sparsity limit analytically, while evaluating various higher-sparsity regimes numerically.

• Size-constrained sub-linear regime with exact recovery: Motivated by recently-shown gains for the regime ρ = O(1) [13] , we show that analogous gains are also possible for more general ρ = o n k . We improve on the best known achievability and converse bounds in such regimes, in particular using the DD algorithm to improve over known results for the COMP algorithm.

Our analysis techniques build on the existing works outlined above, but also come with several new aspects and challenges; specific comparisons are deferred to Remarks 1 and 2.

We first describe a randomized construction of a doubly-regular T × n test matrix X, with T = rn s , where r and s are variables to be chosen according to the setting being studied. We select the test matrix in the following manner:

• Sample r matrices X 1 , . . . , X r independently, where each matrix X j (j ∈ {1, . . . , r}) is sampled uniformly from all n s × n binary matrices with exactly s items per test (i.e., a row weight of s) and one item per column (i.e., a column weight of one). 2

• Form X by concatenating X 1 , . . . , X r vertically.

In other words, we perform r independent rounds of testing, where each round randomly partitions the items into n s tests of size s. This approach was also used as the first step of a two-stage procedure in [14] .

After testing the items using our test matrix, we run the DD algorithm, shown in Algorithm 1, to attain our estimate K of the defective set.

We state our first main result as follows, and prove it in Section III-A.

Theorem 1. Under the doubly-regular design described above, when there are k = Θ n θ defective items with constant θ ∈ (0, 1), the DD algorithm attains vanishing error probability if

where is an arbitrarily small positive constant (i.e., number of sub-matrices satisfies r ≥ (1 + ) max{θ,1−θ} log 2

).

We state our main result for the linear regime as follows, and prove it in Section III-B. In Appendix B, we also provide a counterpart of this result for the case that there are false negatives but no false positives.

Under the doubly-regular design described above with parameters s and r, when there are k = pn defective items with constant p ∈ (0, 1), the DD algorithm attains FNR ≤ min{1, FNR max (1 + o(1))}, where

Next, we pause momentarily to introduce the following definition to help us evaluate our result:

Definition 1 (Rate). Under the combinatorial prior with n items, k defectives and T tests, the rate is equal

, which measures the average number of bits of information that would be gained per test if K were recovered perfectly. In the linear regime with k = pn, this can be simplified to nH2(p)

is the binary entropy function.

Returning to (3), we observe that this expression is minimized when r is large and s is small. However, this conflicts with our goal of minimizing the number of tests T = rn s , which implies that we should be aiming for small r and large s instead. To balance these conflicting goals, we evaluate our results in terms of their achievable rates, subject to a maximum permissible FNR. We first partially do so analytically, and then turn to numerical evaluations.

The following corollary concerns the limit of a small proportion of defectives, i.e., the low-sparsity regime. Corollary 1. Under the setup of Theorem 2, there exist choices of r and s (depending on p) such that, in the limit as p → 0 (after having taking n → ∞), we have (i) FNR max approaches 0, (ii) the rate approaches log 2, and

Proof. We start by choosing s = log 2 p (the effect of rounding is negligible as p → 0) and assuming that r = ω(1), which will be satisfied when we set its specific value. By Theorem 2, we have

where (a) applies

Hence, we have r =

where ( 2) For each p, numerically optimize the free parameters (s, r) to minimize the aspect ratio T n = r s , subject to FNR ≤ α.

3) Compute the rate nH2(p) T , and plot this over the chosen values of p.

The rates attained by the doubly-regular design (from Theorem 2) are shown in Figure 1 . The discontinuities in the plot can be explained by the fact that r (tests per item) and s (items per test) must both be integers. Since FNR max is increasing in p, the pair (r, s) will change whenever FNR max exceeds α. This leads to a downward jump in rate, albeit with FNR max potentially being significantly smaller than α. 3UHYDOHQFHS 6XEPDWULFHVU 3UHYDOHQFHS

,WHPVSHUWHVWV At this stage, one may wonder why some of the curves in Figure 1 appear to approach a value strictly smaller than log(2) ≈ 0.693 despite Corollary 1. The reason is that this behavior is only observed for extremely small p, and the rate almost instantaneously drops to a significantly smaller value as p increases. In fact, we found that if we set s = log 2 p and r = log( 1 p ) log 2 as suggested by Corollary 1, our upper bound on the FNR exceeds one even when p is brought down to the order of 10 −14 . These findings suggest that asymptotic results for the regime k = o(n) should be interpreted with caution when it comes to practical problem sizes.

Discussion on possible converse results. It is difficult to gauge the tightness of our achievability result, due to the lack of converse results in this setting. While we do not attempt to make any formal statements addressing this, we believe that the constant log 2 in Corollary 1 is likely to be the best possible. This is supported by the following:

• Under the near-constant tests-per-item design, the DD algorithm is known to fail to attain FNR → 0 at rates exceeding log 2 bits/test [9] . Furthermore, it appears unlikely that any algorithm could outperform DD for the goal of attaining both FPR = 0 and FNR → 0. 3

• In Appendix B, we study the "opposite" goal of attaining FNR = 0 and FPR → 0, and in that case one can rigorously show that log 2 bits/test is the best possible.

We state our achievability result below, and prove it in Section III-C.

Theorem 3. For k = Θ(n θ ) with θ ∈ [0, 1), and ρ = Θ n k β with β ∈ (0, 1), for any integer r satisfying:

; the DD algorithm with rn ρ tests, chosen according to the above randomized design, recovers the defective set with asymptotically vanishing error probability.

We additionally provide a converse result, which is stated as follows and proved in Section IV.

Theorem 4. Suppose that k = Θ n θ , for θ ∈ (0, 1), and let X be a non-adaptive test matrix such that each test contains at most ρ = Θ n k β items, for β ∈ (0, 1). Given 4 r = max 2,

for an arbitrary constant > 0, if there are (1 − ) rn ρ or fewer tests, then any decoder has error probability 1 − o(1).

The plots of the constant r against θ are displayed in Figure 3 . Our main results do not apply to the case that β = 0 exactly (which was already handled in [13] ), but we can plot the relevant limits as β → 0. Similarly, the results of [13] do not apply when β > 0, but we can plot the relevant limits as ρ → ∞. In Figure 3 (top-left), the same curve is obtained in both limits, and in both cases we have matching achievability and converse bounds.

For the other values of β shown, we observe strict improvements of DD over COMP, and the gap widens as β increases. For the converse, we similarly observe a strict improvement over the previous converse for β ∈ (0, 1). 

In this section, we prove the three achievability results stated above. In general, doubly-regular designs come with more complicated dependencies that are difficult to handle. The construction that we consider (i.e., concatenating independent doubly-regular sub-matrices with column weight one) allows us to simplify the analysis of the DD algorithm by extending the analysis of one sub-matrix to the entire test matrix.

We proceed by outlining the key steps of the analysis and introducing the relevant notation. This applies to all three settings that we consider, and the differences lie in the specific details (e.g., the choices of r and s) and the subsequent parts that extend from these steps. The key steps are:

1) Determine concentration of #non-defective items in PD: Let G be the set of non-defective items in PD,

This step concerns the concentration of G around its mean. We start by considering the probability that a particular non-defective item i is found in a negative test in a single sub-matrix X j . Consider the unique test which item i is found in. The probability that all the other s − 1 items in the test are non-defective is

Hence, consider all of the r sub-matrices, we obtain

where (a) applies E[1{A}] = P[A], and (b) applies (10) . Hence, we have

Depending on the setting, we use an appropriate concentration inequality (e.g., Chebyshev's inequality) to

show that G is close to this value with probability 1 − o(1).

2) Determine concentration of #masked defective items: We introduce two further definitions.

Definition 2. Consider a defective item i and a set L that does not include i. We say that defective item i is masked by L if every test that includes i also includes at least one item from L.

Definition 3. We call an item i masked if it is masked by K \ {i}. 5 If the matrix is not specified then this property is defined with respect to the full test matrix X, but we will also use the same terminology with respect to a given sub-matrix X j .

For a given defective item i, let M j i = 1{i is masked in X j }, and let M j = i∈K M j i be the number of masked defective items in X j . Without loss of generality (WLOG), suppose that items 1, . . . , k are defective.

We have

where the last equality uses the same steps as (10) . Summing over all k defective items, it follows that

Depending on the setting, we use an appropriate technique to show that M j concentrates around this value for all X j simultaneously, with probability 1 − o(1).

3) Establish conditional independence: In this step, we condition on the preceding high-probability bounds on G and M j holding. To facilitate the analysis, it is useful to not only condition on such events, but to condition on more specific events that ensure such concentration (apart from these, one final conditioning event will also be given below):

• We condition on a fixed set of tests being positive, and the remaining tests being negative. This fixed set has no explicit constraints.

• We condition on a fixed realization of the defective set K.

• We condition on G (the non-defectives that are marked as possibly defective) being a fixed set with some fixed size G. This value of G is assumed to satisfy the above-established concentration behavior.

• For j = 1, . . . , r, we condition on the number of masked defectives in sub-matrix X j being M j , whose values again satisfy our established concentration results. Note that unlike with G, we do not condition on the specific set of masked item indices, but instead only on the total number.

Once we show that the conditional error probability is suitably small, it follows that the same is true after averaging (over all realizations of positive tests, all possible K, and so on).

By the symmetry of the test design, without loss of generality, we can consider the following:

• The defective items are indexed by 1, . . . , k.

• The items in G indexed by k + 1, k + 2, . . . , k + G.

A slight issue here is that if we naively condition on the above sets of size G and M j , the resulting submatrices X 1 , . . . , X r will typically not be conditionally independent, since the sub-matrices are coupled via G. Specifically, conditioning on G amounts to conditioning on (i) items k + 1, k + 2, . . . , k + G only being in positive tests in all sub-matrices, and (ii) each of items k + G + 1, k + G + 2, . . . , n being in a negative test in some sub-matrix. The latter of these means, for example, that if we are told that item k + G + 1 is in a negative test in submatrix 1, it reduces the probability of the same being true in submatrix 2 (thus violating independence).

To overcome this difficulty, we additionally condition on the fixed and specific placements of items k + G + 1, k + G + 2, . . . , n into negative tests (but not positive tests). This must be done subject to each of them being in some negative test, but the placements are otherwise arbitrary and do not impact our analysis.

When this additional conditioning is done, the overall conditioning involving G and M j becomes an "AND" of r sub-events (one per sub-matrix), with each sub-event requiring that (i) M j defectives are masked in submatrix j, (ii) items k + 1, k + 2, . . . , k + G are only included in positive tests, and (iii) items k + G + 1, k + G + 2, . . . , n are included in the specific negative tests indicated. Due to this "AND" structure, conditional independence is maintained among the r sub-matrices.

Bound the total error probability: Recall that M j i is the indicator event that defective i is masked by K \{i} in X j , and let MG j i be the event that defective item i is masked by G in X j . For a given X j , we have the following, where E is the event described in step 3 above and the subscript P E [·] indicates conditioning:

where:

• (a) uses the fact that that we conditioned on having M j masked defective items, so by the symmetry of the randomized test design, we have P M j i = M j k . • (b) holds because conditioned on ¬M j i , item i is in one of the k − M j tests with a single defective item (recall that k − M j is the number of non-masked defective items). Again using symmetry, each non-defective item has at most 1 k−M j probability of being in that particular test (due to there existing k − M j equally likely options). Taking the union bound over all items in G gives the desired bound.

By (19) and the conditional independence of the X j 's, the probability of defective item i being masked by PD \ {i} in all sub-matrices satisfies

Remark 1. (Comparison to existing techniques) Our approach is distinct from existing analyses of the DD algorithm [6] , [8] , [13] , which use a "globally symmetric" random matrix with no sub-matrix block structure. Like all of these works, we still adopt the the high-level steps of characterizing G and then characterizing M j conditioned on G, but the details are largely different, including the unique aspect of transferring results regarding simpler sub-matrices to the entire test matrix. Moreover, regarding the size-constrained regime, we note that the analysis of [13] relies strongly on the assumption ρ = O (1), which is what led us to adopting a distinct approach. Compared to [13] , we believe that our test design and analysis are relatively simpler, though the regimes handled are different and neither subsumes the other (β = 0 vs. β > 0).

We follow the four steps described above to obtain our result, and begin by selecting the relevant parameters.

We choose s = n log 2 k and r = c log n, where c is a constant to be specified shortly 6 . This results in each X j being a k log 2 × n sub-matrix, and X being a ck log n log 2 × n matrix.

Step 1: Substituting s = n log 2 k and r = c log n into (12), we have Markov's inequality, we have

where ( Step 2: Substituting s = n log 2 k and r = c log n into (14), we obtain

where the last equality uses the same steps as those in (21)- (24) to simplify the product to 1 2 (1 + o (1)). Next we have

We proceed to evaluate the variance and covariance terms separately. The calculation is the same for any value of i, so we fix i = 1 and write

since the equality P M j 1 = 1 = 1 2 (1 + o (1)) is implicit in (27) . Next, we define S j 12 to be the event that defective items 1 and 2 are in the same test in X j , yielding

In addition, we have

Focusing on the first term, by the law of total probability, we have 

where (a) computes the probabilities in the same way as (36), and (b) follows by expanding the square and simplifying.

The combinatorial terms in (39) can be bounded using manipulations that are elementary but tedious, so we merely state the resulting bound here and defer the proof to Appendix A.

Applying (29) and Lemma 1 in (28), we obtain Var M j = O max k, k 3 n . By Chebyshev's inequality, it follows that

where we used the fact that k = Θ n θ with θ ∈ (0, 1). (1)) for all X j simultaneously with probability

where (a) uses the fact (1 + a) b = (1 + ab)(1 + o(1)) when |a| < 1 and ab = o(1).

Step 3: We condition on the events described in the general description of Step 3 following (14) . Here the highprobability bounds dictate that G = o(k) and M j = k 2 (1 + o(1)).

Step 4: Substituting r = c log n into (20) , the conditional probability of defective item i being masked by PD \ {i} in all sub-matrices is

where (a) substitutes M j = k 2 (1 + o(1)) and G = o(k). Taking the union bound over all defective items, the probability of any defective item being masked by PD\{i} in all sub-matrices is at most kn −c log 2+o(1) = Θ n θ−c log 2+o (1) . The power of n is below zero when c ≥ (1+ ) θ log 2 for some positive constant . Together with our previous bound on c (see (26) ), this requires us to choose c ≥ (1 + ) max{θ,1−θ} log 2 , which means it suffices to have

This completes the proof of Theorem 1.

We again follow the four-step procedure to obtain the desired result. We assume that s = Θ(1) and r = Θ(1),

but their values are otherwise generic. Recall that each X j is a n s × n sub-matrix, and X is a rn s × n matrix. It suffices to prove the theorem for s ≥ 2, since otherwise each X j amounts to one-by-one testing and the FNR is trivially zero.

The following technical lemma will be used throughout the analysis. We now proceed with the main steps.

Step 1: Continuing from (12), we have 

where here and in the rest of this step (step 1), we assume for notational convenience that items 1 and 2 are non-defective. 7 We proceed to evaluate the variance and covariance terms separately. We have

where (a) uses the same steps as (10)-(11) followed by Lemma 2. Next, we have

Focusing on the first term, we note that the event (PD 1 = 1) ∩ (PD 2 = 1) holds if items 1 and 2 are both contained in a positive test in each sub-matrix. Since the sub-matrices are sampled independently, we can consider them separately. Let PD j 1 , PD j 2 respectively be the events that items 1, 2 are contained in a positive test in sub-matrix X j . Then P[PD 1 = 1, PD 2 = 1] = P PD 1 1 = 1, PD 1 2 = 1 r .

Let S j 12 be the event that items 1, 2 are placed into the same test in X j . Similar to before (see (30)), we have P S j 12 = s−1 n−1 . Conditioning on event S j 12 , we have:

where the final equality uses the same steps as (10) 

where (a) uses the the inclusion-exclusion principle, (b) uses the same steps as (10) followed by Lemma 2, and (c) uses the fact that s and p are constant. Hence, by the law of total probability, P PD 1 1 = 1, PD 1 2 = 1 = P S j 12 P PD 1 1 = 1, PD 1 2 = 1|S j 12 + P[¬S j 12 ]P PD 1 1 = 1, PD 1 2 = 1|¬S j

(63) 7 This should not be confused with the convention K = {1, . . . , k} and G = {k + 1, . . . , k + G} used when analyzing the defective items in other steps. The precise indices are inconsequential and merely a matter of notational convenience, so there is no contradiction in using indices 1 and 2 for non-defectives here but for defectives in other parts.

where (a) substitutes P S j 12 = s−1 n−1 along with (57) 

since the leading terms cancel and only leave the O 1 n remainder. Substituting (54) and (67) into (52), we obtain

Since E[G] = Θ(n) (see (51)), it follows from Chebyshev's Inequality that

Step 2: Continuing from (14), we have

where the last equality uses the same steps as those in (49) Step 3: We again condition on the events described in the general description of Step 3 following (14) . Here the high-probability bounds dictate that (1)).

Step 4: Continuing from (20) , the conditional probability of defective item i being masked by PD \ {i} in all sub-matrices (which is precisely the FNR) satisfies substitutes k = pn. This completes the proof of Theorem 2.

In this setting, to reduce the number of constants throughout, we consider k = n θ and ρ = n k β (i.e., implied constants of one in their scaling laws), but the general case follows with only minor changes. We again follow the above four-step procedure to obtain our result, and begin by selecting the relevant parameters. We choose s = ρ, and let r be a constant (not depending on n) to be chosen later. This results in each X j having size n ρ × n, and X having size rn ρ × n.

Step 1: Substituting s = ρ into (12), we obtain (recall that ρ → ∞ with ρ = o n k ). We now consider two cases:

• For θ ≥ 1 2 , we use Markov's inequality to obtain

where (a) substitutes ρ = (n/k) β and k = n θ . Note that when r ≥ 2−β 1−β , the expression in (77) scales as O 1 log n . • For θ < 1 2 , we similarly have

where (a) substitutes ρ = (n/k) β and k = n θ . When r ≥ 1−θβ (1−θ)(1−β) , the expression in (78) scales as O 1 log n . As a side-note, to see why we split θ into two cases here, we note that 2−β 1−β ≤ (for any β ∈ [0, 1)).

Step 2: Substituting s = ρ into (14), we have

where the last equality uses the same steps as those in (74)-(76). By Markov's inequality, it follows that

Turning to the desired event for all r sub-matrices simultaneously, we have M j < k 2 ρ n log n with probability at least 1 − 1+o (1) log n r = 1 − O 1 log n , using the fact that r = Θ(1).

Step 3: We again condition on the events described in the general description of Step 3 following (14) . Here the high-probability bounds dictate that

and M j ≤ k 2 ρ n log n for all X j simultaneously.

Step 4: Continuing from (20) , the probability of defective item i being masked by PD \ {i} in all sub-matrices is

where we substituted M j ≤ k 2 ρ n log n and G as shown in (81) and applies some simplifications. Taking the union bound over all k defective items and equating the resulting bound with a target value of 1 log n , we obtain the following conditions for the desired events to hold with probability at least 1 − 1 log n :

In other words, there exist constants C 1 and C 2 such that it suffices to have

Substituting ρ = n k β and k = n θ , and performing some simplifications, we obtain the conditions

Observe that for any integer r satisfying

the exponent of n on the left-hand side of (87) becomes negative, so that (87) is satisfied. Similarly, observe that for any integer r satisfying

the exponent of n on the left-hand side of (88) becomes negative, so that (88) is indeed satisfied. Hence, together with the bounds on r from step 1, we require r to be an integer such that

Note that (92) simplifies to r ≥ 1−θβ (1−θ)(1−β) alone, because for any θ ∈ 0, 1 2 and β ∈ [0, 1), we have (i) 1−θβ > θ, which implies that

Finally, we used a total of rn/ρ tests and incurred a total error probability of at most O 1 log n . This completes the proof of Theorem 3.

In this section, we prove Theorem 4. We prove the results corresponding to r = 1−(1−θ)(2β+1)

and r = 2 separately in Section IV-A and Section IV-B respectively; the remaining term 1 1−β is already known from [12] .

Compared to the achievability part, the analysis in this section builds more closely on that of the prior work [13] handling the regime ρ = O(1). In particular, for the first part (Section IV-A), we mostly follow [13] but with different choices of parameters to carefully account for the scaling of ρ. On the other hand, for the second part (Section IV-B), more substantial changes are needed: In [13] , the scaling ρ = O(1) makes it relatively easier to identify positive tests with multiple items of degree one (leading to failure), whereas in our setting with ρ = ω(1)

the argument is somewhat more lengthy and technical, though still adopts a similar high-level approach.

To reduce the number of constants throughout, we consider k = n θ and test sizes ρ = n k β = n β(1−θ) (i.e., implied constants of one in their scaling laws), but the general case follows with only minor changes. Without loss of generality, we may assume that θ > 1+β 2+β ; this is because if θ ≤ 1+β 2+β , then 1−(1−θ)(2β+1)

≤ 1, which implies that it will not be the maximum among the three constant terms in (9) . We make use of the notion of masked items in Definition 3, and we are now interested in characterizing both masked defectives and masked non-defectives.

Let the true defectivity vector be u ∈ {0, 1} n , where u i = 1 indicates that the i-th item is defective, and u i = 0 otherwise. Furthermore, let the test outcomes be represented by the vector y ∈ {0, 1} T , where y i = 1 denotes that the i-th test is positive and y i = 0 otherwise. We also define

and we note that our assumption θ > 1+β 2+β implies that d − > 0. We can visualize any non-adaptive group testing design matrix X as a bipartite graph G = (V ∪ F, E) with |F | = T factor nodes {a 1 , . . . , a T } (tests) and |V | = n variable nodes {u 1 , . . . , u n } (items). An edge between an item u i and test a j indicates that u i takes part in a j . We let {∂ G a 1 , . . . , ∂ G a T } and {∂ G u 1 , . . . , ∂ G u n } denote the neighborhoods in G ; whenever the context clarifies what G is, we will drop the subscript. The sparsity constraint on the test implies that |∂ G a j | ≤ ρ. Finally, let V 1+ (G ) be the set of defective items that are masked, V 0+ (G ) be the set of non-defective items that are masked, and V + (G ) = V 1+ (G ) ∪ V 0+ (G ). We have the following auxiliary results. 

Our goal is to show that with T = (1 − ) d + n ρ , for some constant > 0, we have |V 1+ (G)|, |V 0+ (G)| = ω(1) with high probability (i.e., with probability 1 − o(1)), which implies (using Lemma 3) that P e = 1 − o(1). As a stepping stone to our result for the combinatorial prior, we first study the i.i..d. prior, whose defectivity vector we denote by u * , such that each entry is one independently with probability p = k− √ k log n n . The following existing result allows us to transfer from the latter prior to the former.

Lemma 5. [13, Corollary 3.6] Given non-negative integers C 1 , C 2 and fixed 1 , 2 ∈ (0, 1), if the modified model

then the original model (combinatorial prior) satisfies

In view of this result, we proceed by working with u * instead of u. Next, we introduce a key fact [9] : Whenever items have distance at least 6 in the underlying graph, the events of being masked are independent. Leveraging on this fact, we introduce a procedure for obtaining a set B of size N (to be specified shortly), such that each item has a degree of at most d − , and the pairwise distance between items is at least 6. The procedure is as follows:

1) Create G 0 from G by executing the following in the order provided: a) Remove all items with degree greater than log n.

b) Remove all tests with degree less than ρ log n , and their respective items. c) Remove all tests with degree at most one, and their respective items. We now proceed to analyze each of the steps.

Analysis of step 1: In step 1(a), we remove all items with degree greater than log n, and we suppose that their defectivity status is given to the decoder by a genie. This genie information only makes decoding easier, i.e., if we cannot decode with these items removed (and with the genie information), then we cannot decode in G .

Afterwards, in step 1(b), we can further remove tests with degree less than ρ log n and their items. By doing so, the probability of the other items being masked can only decrease. More specifically, the remaining items are contained in the same tests as before, but each test in G 0 may now have fewer items. Finally, in step 1 (c), we remove all tests with at most one item and their corresponding items; this can trivially be done because zero-item tests are useless, while any single-item test immediately reveals the status of that item.

To proceed with G 0 , we need to understand how many items it contains, and the degree properties of its items and tests. To understand the number of items, we investigate how many items of various kinds could have been removed from G . We first note that the number of edges in G contributed by items with degree greater than log n is upper bounded by the total number of edges, which equals

Hence, the number of items in G with degree greater than log n (and hence removed in Step 1(a)) is at most

Next, we note that the number of tests with degree less than ρ log n is trivially less than the total number of tests

. Hence, the number of edges contributed by those tests is less than ρ log n · (1− )d + n ρ = o(n). This implies that the number of items that are removed in Step 1(b) is at most o(n). Furthermore, Step 1(c) amounts to removing at most T = O n ρ = o(n) tests (recall that β > 0) with at most one item, which implies removing at most o(n) items. Combining the above, we conclude that G 0 has n(1 − o(1)) items, all with degree at most log n, and every test has degree between ρ log n and ρ.

Under the preceding setup with T ≤ (1 − ) d + n ρ , the number of items in G 0 with degree at most d − is Θ(n).

Proof. Supposing for contradiction that the number of items with degree at most d − is o(n), the number of items with degree at least d + is n (1 − o(1) ). It follows that the number of edges contributed by these items is at least (1)), which implies that the number of tests required for those edges is at least

for sufficiently large n. This contradicts the assumption on T .

Analysis of step 2: Due to the degree constraints in G 0 established above, in each iteration, we remove at most ρ log n + ρ 2 log 2 n = Θ n 2β(1−θ) log 2 n

items. This scales as o(n) when θ > 1 − 1 2β with β ∈ (0, 1), which is satisfied due to our assumption θ > 1+β 2+β = 1 − 1 2+β (note that 2 + β ≥ 2β). In total, after N = n 1−2β(1−θ) log 3 n iterations, we removed

items with degree at most d − . Since we have Θ(n) items with degree at most d − in G 0 (see Lemma 6) , and we removed only o(n) items with degree at most d − (from (100)), this guarantees that we will always be able to extract N such items.

We now focus our attention on the set B. For each item u ∈ B, we have

where:

• (a) is due to the fact that transforming G into G 0 (refer to step 1 of the procedure) causes u to stay in the same tests of potentially smaller sizes. To see why u remains in the same number of tests, note that in step 1 of the procedure, whenever we remove a test, we also remove all the items in the test. Hence, the fact that u exists in G 0 implies that none of its tests has been removed.

• (b) follows from the fact that the events of u being masked in each of its tests satisfy an increasing property 8 [15, Lemma 4] ; this is a sufficient condition for applying the FKG inequality [28] , which lower bounds the probability by the expression that would hold under independence across a ∈ ∂u. Note also that |∂a| ≥ 2 because of step 1(c) in our procedure.

• (c) follows by first noting that p|∂a| ≤ kρ n (1 − o(1)) = o(1), where the inequality holds by substituting p = k n (1 − o(1)) and |∂a| ≤ ρ, and the equality uses ρ = o n k . We then use a Taylor expansion in (102), followed by some simplifications.

• (d) uses the fact that |∂a| ≥ ρ log n = ω(1) for each test a in G 0 . • (e) uses the fact that pρ log n (1 + o(1)) = O kρ n log n = n −Ω(1) log n < 1 (implying that a higher power in the exponent makes the overall expression smaller) and |∂u| ≤ d − .

We now turn to the number of masked defective items in B. Recall that for any two individuals u, u ∈ B, the events of being masked are independent due to the pairwise distances being at least 6. Furthermore, each item is defective independently with probability p under u * , and the property of any given item being masked is independent of the item's defectivity status (as noted in [10] ). Hence, |V 1+ (G , u * )| stochastically dominates

, which has an expectation of

where (107) 

We again work in the regime where ρ = n k β = n β(1−θ) , with β ∈ (0, 1) (constant factors are set to one). We start by stating an auxiliary result that will be used later. p = z n , and 0 < t < p, we have

where D(a b) = a log a b + (1 − a) log 1−a 1−b is the binary KL divergence.

We wish to show that with T = (1 − ) 2n ρ , 9 where is a positive constant, no algorithm successfully recovers u from (y, G ) with probability Ω(1). We proceed with a proof by contradiction, assuming that the opposite is true, i.e., that with T = (1 − ) 2n ρ , there exists an algorithm that successfully recovers u from (y, G ) with probability Ω(1). Based on this assumption, we show that cannot be a positive constant, which gives us a contradiction. Proof. Let α 0 n and α 1 n be the number of items with degree zero and one respectively. We start by establishing that α 0 = o(1). This is because if there were Ω(n) degree-zero items in each test, then with high probability there would be ω(1) degree-zero defectives and ω(1) degree-zero non-defectives (e.g., using the variant of Hoeffding's bound for the hypergeometric distribution [30] ). Since these are impossible to distinguish from each other, this implies o(1) success probability, in contradiction with the lemma assumption. Now, by lower bounding the number of edges contributed by the items, and upper bounding the number of edges contributed by the tests, we obtain

Combining the left-most and right-most expressions and making α 1 the subject and applying α 0 = o(1), we obtain (1)).

Before proceeding, we introduce some further notation:

• T denotes the set of tests with at least ρ log n items of degree one, and T = |T |. We observe that the lower bound T ≥ n ρ log n must hold, since otherwise the total number of items of degree one would be at most (1 − ) 2n ρ · ρ log n + n ρ log n · ρ ≤ 3n log n , contradicting Lemma 8. • T + denotes the number of tests in T with at exactly one defective item of degree one.

• z denotes the number of items with degree one that appear in the tests of set T . Note that z satisfies z ≥ T ρ log n ≥ n log 2 n because all tests in T has at least ρ log n items of degree one, and z ≤ T ρ because of the ρ-sized test constraint. These inequalities will be used in our analysis later.

• Z 1 denotes the number of defective items with degree one that appear in the tests of set T .

Note that among the four quantities introduced above, only T + and Z 1 are random (i.e., affected by the randomness of the defective set).

Lemma 9. When the number T of tests containing at least ρ log n items of degree one is at least n ρ log n , any algorithm recovering u from (y, G ) has a success probability of o(1).

Before proving this lemma, we state some additional auxiliary results. Proof. For this proof, we momentarily return to the i.i.d. prior, where p = k− √ k log n n = k n (1 − o(1)), to study how T + scales. The number of defective items of degree one in a test containing ρ ≥ ρ log n items of degree one is distributed as Binomial ρ , k n (1 − o(1)) , which implies

where (a) uses ρ ≥ ρ log n and ρ ≤ ρ, (b) uses kρ n = o(1) and a Taylor expansion, followed by some simplifications, and (c) uses kρ n = o(1). Observe that the event of a test in T having at exactly one defective item of degree one is independent of the same event for the other tests in T , since the items are defective independently and cannot be shared across the tests (due to the degree of one). Hence, T + stochastically dominates Binomial T , kρ n log n (1−o(1)) , which implies

by applying T ≥ n ρ log n and simplifying. By Lemma 4 with = 1 − 1+o(1) log n , it follows that

which implies that we have T + > k log 3 n = ω(1) with probability 1 − o(1) This further implies that |V 0+ | > Proof. Due to fact that the defective set K is uniformly distributed, we have P[Z 1 = z 1 ] = z z1 n−z k−z1 n k −1 , i.e., Z 1 ∼ Hypergeometric(n, z, k). Using this fact, we have

where (a) uses Lemma 7 with t = z n 1 − 1 log n , and (b) uses z ≥ n log 2 n . This proves that

log n = zk n log n ≥ k log 3 n (by applying z ≥ n log 2 n ), we have Z 1 > k log 3 n = ω(1) with probability 1 − o(1).

With the auxiliary results in place, we turn to the proof of Lemma 9.

Proof of Lemma 9. Our aim is to show that with T ≥ n ρ log n , the success probability is o (1) . From this point onwards, we condition on |V 0+ | = ω(1) and Z 1 = ω(1), both occurring with probability 1 − o(1) (see Lemma 10 and Lemma 11). Under these conditioned events, we have the following possible cases: (1) |V 1+ | = 0 and |V 0+ | = ω(1); (2) |V 1+ | > 0 and |V 0+ | = ω(1). We proceed to analyze each case separately:

• Case 1: The condition |V 1+ | = 0 implies that every positive test in T must have exactly one defective item, which implies T + = ω(1). For each of these tests, the decoder can do no better than to guess the defective item uniformly at random from all degree-one items in the test, of which there are at least ρ log n . Hence, the success probability in each test is at most log n ρ = o(1). • Case 2: In this case, a direct application of Lemma 3 gives us an error probability of 1 − o(1) (i.e., a success probability of o(1)).

Since the success probability is o(1) in both cases regardless of the specific values of |V + | = ω(1) and Z 1 = ω(1) being conditioned on, it follows that the unconditional success probability is also o(1). This completes the proof of Lemma 9.

In view of Lemma 9, for the assumed condition stated following Lemma 7 to hold (namely, that there exists an algorithm with success probability Ω(1)), it must be the case that T < n ρ log n . We proceed by assuming that this is true, and showing that we arrive at a contradiction.

The condition T < n ρ log n implies that the number of edges contributed by the T tests is below n ρ log n · ρ = n log n = o(n), which further implies that o(n) items of degree one participate in the tests of T . Recalling Lemma 8, this leaves us with greater than 2 n(1 − o(1)) items of degree one that participate in tests containing fewer than ρ log n items of degree one. Comparing the overall degrees, we find that the number of such tests is greater than 

which gives the desired contradiction, as was assumed to be Θ(1). This completes the second part of the proof of Theorem 4.

In this paper, we have analyzed the performance of doubly-regular group testing designs in several settings of interest, with our main results summarized as follows:

• In the unconstrained setting with sub-linear sparsity, the block-structured doubly-regular design with the DD algorithm matches the achievability result of the DD algorithm with near-constant tests-per-item, which is known to be optimal for θ ≥ 1 2 . • In the unconstrained setting with linear sparsity, we complemented hardness results for exact recovery by deriving achievability results with only false negatives in the reconstruction.

• In the setting of ρ-sized tests with sub-linear sparsity and exact recovery, we improved on previously-known upper and lower bounds in regimes of the form ρ = Θ n k β with β ∈ (0, 1), complementing recent improvements that only hold for β = 0. Here we provide an analog of Theorem 2 for the COMP algorithm (see Algorithm 1), which comes essentially "for free" from our analysis of the DD algorithm. However, we note that unlike our DD analysis, the result for COMP would also follow easily from prior work, particularly [14] . Recall that the COMP algorithm only produces false positives, which implies that we only need to look at the FPR (i.e., FNR = 0). 

where the second and third qualities use the fact that s, r, and p are all constant with respect to n.

A given FPR value corresponds to an average of (n − k)FPR false positives, which may potentially be much larger than k. To place the number of false positives and the actual number of defectives on the "same scale", we find it more convenient to work with the normalized quantity FPR n−k k = FPR 1−p p . Then, this quantity equaling a given value α > 0 corresponds to an average of αk false positives.

Recalling the notion of rate in Definition 1, we have the following analog of Corollary 1; the proof is similar but simpler, so is omitted. In this case, we can easily argue that the constant of log 2 is optimal. To see this, note that if we can could attain at most αk false positives on average for arbitrarily small α > 0, then we could use this to construct a two-stage adaptive group testing algorithm where the second stage tests the items outputted in the first stage individually, thus using an average of (1 + α)k tests or fewer. When p approaches zero, these additional tests contribute a arbitrarily small fraction compared to the leading Θ k log n k = Θ k log 1 p term, so the two-stage design attains zero error probability with an arbitrarily small increase in the rate. However, the converse result of [14] shows that rates above log 2 are impossible in this two-stage setting (see also [31] for the sublinear regime). This establishes the optimality of the constant log 2 above.

To visualize the constant-p regime, we plot the rates attained by the COMP algorithm with the doubly-regular design in Figure 4 (see the text following Corollary 1 for discussion on why the behavior of the curves reaching log 2 as p → 0 is not visible; a similar discussion applies here).

An analysis of the DD algorithm for group testing with size-constrained tests

Group testing: An information theory perspective

Pooled testing and its applications in the COVID-19 pandemic. Pandemics: Insurance and Social Protection

The separating property of random matrices

Boolean compressed sensing and noisy group testing

Group testing algorithms: Bounds and simulations

Phase transitions in group testing

Performance of group testing algorithms with near-constant tests-per-item

Information-theoretic and algorithmic thresholds for group testing

Optimal group testing

Group testing with random pools: Phase transitions and optimal strategy

Nearly optimal sparse group testing

Near-optimal sparsity-constrained group testing: Improved bounds and algorithms

Conservative two-stage group testing

Individual testing is optimal for nonadaptive group testing in the linear regime

Optimal non-adaptive probabilistic group testing in general sparsity regimes

Non-adaptive group testing: Explicit bounds and novel algorithms

The capacity of Bernoulli nonadaptive group testing

Non-adaptive probabilistic group testing with noisy measurements: Near-optimal bounds with efficient algorithms

Efficient algorithms for noisy group testing

Saffron: A fast, efficient, and robust framework for group testing based on sparse-graph codes

Efficiently decodable error-correcting list disjunct matrices and applications

Efficiently decodable non-adaptive group testing

A fast binary splitting approach to non-adaptive group testing

Fast splitting algorithms for sparsity-constrained and noisy group testing

Near-optimal sparse adaptive group testing

Randomized Algorithms

Correlation inequalities on some partially ordered sets

Hypergeometric tail inequalities: Ending the insanity

Probability inequalities for sums of bounded random variables

Group testing with random pools: Optimal two-stage algorithms

An immediate direction for future research is to establish converse bounds for approximate recovery in the linear regime with no false positives, and also to establish a more detailed understanding of the entire FPR vs. FNR tradeoff. In addition, in the size-constrained setting, the optimal thresholds still remain unknown in general, despite the gap now being narrower.It suffices to show that the first and second parts of (39) simplify to O max{ 1 k , k n } . First part: Expanding the binomial coefficient in terms of factorials, the first term simplifies as follows: Second part: Expanding the binomial coefficient in terms of factorials, we have