In this annex, we derive an analytical expression for the correlation between the TD50 and the MTD. To this end, suppose that the probability P(d) of a tumor occurring in an animal exposed to dose D = MTD satisfies the Weibull model
in (2.5), where the background parameter α > 0 and the shape parameter k > 0 are known. This is a generalization of the one-stage model used by Bernstein et al. (1985) in which k = 1.
Suppose that x of the n animals exposed to dose D develop tumors. Since and k are assumed known, β may be estimated by

where p0 = P(0) describes the spontaneous response rate. This leads to an estimate
of the TD50.
The estimate of β is appropriate for r ≤ x ≤ n-1. The lower limit of x = r is the minimum value of x that would lead to a statistically significant result at a nominal significance level of 0 < γ < 1; the value of r is determined from the fact that in the absence of a treatment effect at dose D, x follows the binomial distribution Bin (50, P(D)). The upper limit of x = n-1 is included since β, and hence TD50, is undefined for x = n.
The constraint x ≤ n-1 implies that

whereas x ≤ r implies

We wish to find the correlation between Y = logeTD50 and X = log eD. (Although the correlation will be identical using logarithms to the base 10, the derivation of the correlation given here is simpler using natural logarithms.) Suppose now that W = TD50 follows a uniform distribution on the interval [a,b], reflecting the fact that given the value D of the MTD, the estimated value of the TD50 is unrelated to the MTD. Suppose further that X follows some distribution with mean µ and variance σ2. Although Bernstein et al. (1985) observed that the empirical distribution of X is approximately normal, the correlation between Y and X does not depend on the distribution of X other than through its variance σ2.
To calculate corr (Y, X), note that
and

where
and
Thus we have
with V(X) = σ2. Noting that
where µ = E(X), we have
This leads to the desired result:
It can be shown that h2 - h12 + 1 ≥ 0, so that 0 < ρ ≤ 1. It can also be shown that ρ ↓ [σ2/(σ2 + 1)]1/2 as k ↓ 0, and that r 1 as k → ∞. Thus [σ2/(σ2 + 1)]1/2 ≤ ρ ≤ 1. In the limiting case as n → ∞, (D. 13) reduces to
The values of the correlation coefficient ρ in (D.13) as a function of