4
$\begingroup$

I have an investigator who is planning to study a new treatment on a group of 40 patients. He believes that 25% of people will dropout, mostly due to lack of insurance coverage. His goal is to have 70% of people simply complete the treatment (calling this feasible), but also that the percent completed the treatment not be <50% (infeasible). He wants to know what power is needed. It seems like something that doesn't require a power analysis which is what I'm confused about.

He referenced a study which did something similar https://www.clinicalkey.com/#!/content/playContent/1-s2.0-S2452109423002269?returnurl=https:%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS2452109423002269%3Fshowall%3Dtrue&referrer=https:%2F%2Fpubmed.ncbi.nlm.nih.gov%2F

"The primary feasibility endpoint was the ability of the device to deliver an acceptable dose plan for a given patient. The trial was designed as a Simon's 2-stage design with a “good” success rate (P1) defined as 85% and a “poor” rate (P0) as 55%. Power was set at 80%, and the significance level at α = 0.05. Early study stoppage was planned if, after evaluating the device on 8 patients in the first stage, the dose distribution was acceptable for ≤5 patients. However, the trial proceeded past this point based on the recommendation of the Internal Review Board (IRB), and having met this threshold, a total of 17 patients were enrolled. Based on this expanded number, the device would meet the primary endpoint for acceptance if the dose distribution was acceptable in 12 or more patients."

This study seems to be a bit more specific, with a clearly defined outcome of dose distribution. It doesn't seem like what he wants to do requires power analysis at all, but does anyone have any thoughts on this? thank you.

$\endgroup$
2
  • 2
    $\begingroup$ Looks like UNC has an online calculator for such a design cancer.unc.edu/biostatistics/program/ivanova/… $\endgroup$ Commented yesterday
  • $\begingroup$ yep I was using the ph2simon package in R, but that's nice because I can send it to him. thank you. $\endgroup$ Commented 31 mins ago

1 Answer 1

9
$\begingroup$

A power calculation is really just another way of showing that a study can answer the question it was set up to answer to a desirable degree of reliability. In that sense, if you go in with a specific question, it always makes sense to assess this.

That said, I don't think you need a "power analysis" as much as a sample size estimation -- but the two are closely linked. The way I read the plan is that you want to distinguish between $H_0:\pi\le0.5$ versus $H_A:\pi\ge0.7$ with $\pi$ the probability of completing treatment. You'll need to set acceptable rates of type I and type II assertions (respectively deciding in favour of $H_A$ when in truth $H_0$ and vice versa) which I'll fix at $\alpha=0.05$ (one-sided) and $\beta=0.20$.

The question then becomes: what probability do I have for observing a certain proportion of completers under a given hypothesis? This can be derived from the binomial distribution which I'll calculate using R:

## Range of sample sizes under consideration ns <- 10:100 ## 1 - alpha probability of observing this many completers or less under H0 y0 <- qbinom(.95, ns, .50) ## 1 - beta probability of observing this many completers or more under HA ya <- qbinom(.20, ns, .70) 

Plotting both of these bounds: Probability of observing number of completers under each hypothesis

At these operating characteristics you don't want to go in with, say, $n<20$ because there is a range of observations that is compatible with both hypotheses (in the plot: the red line lies above the green line). The ideal size is where you can demarcate an observation that is unlikely under $H_0$ (with probability $1-\alpha$) and one that is likely under $H_A$ (with probability $1-\beta$):

## Where does the red line go below the green line? i <- which.max(y0 < ya) ## Sample size ns[i] #> 37 ## Threshold for accepting H0 y0[i] #> 23 ## Threshold for accepting HA ya[i] #> 24 

An efficient design (for these hypotheses, $\alpha$, and $\beta$!) is to use $n=37$, and to accept $H_0$ when you observe $x\le23$ completers, or $H_A$ when you observe $x\ge24$ instead.

As a sanity check this is easily confirmed empirically:

set.seed(1) mean(rbinom(1E6, 37, .5) > 23) #> 0.049129 mean(rbinom(1E6, 37, .7) < 24) #> 0.192474 

Your referenced study mentioned a "Simon's two-stage design": this adds an early stop for $H_0$. Details are in his paper.

I'll say first that to specify these designs you should really use a more optimized routine such as clinfun::ph2simon or the calculator provided in the comments: this requires numerical evaluation of a sizeable range of binomial (cumulative) densities to identify the combinations of sample sizes and completers in both stages that lead to the desired operating characteristics while minimizing the expected or total sample size. Still, for fun (learning is fun!) here is a very naïve R function that gives the same result:

## Calculate Eqn 1 from Simon 1989 bprob <- \(r1, n1, r, n, p) { x <- if ((r1+1) <= min(n1,r)) seq(r1+1, min(n1,r)) else numeric(0) pbinom(r1, n1, p) + sum(dbinom(x, n1, p) * pbinom(r-x, n-n1, p)) } ## Derive Simon two-stage design. Be warned: not fast. simon <- function(h0, ha, nrange, alpha=0.05, beta=0.20) { rv <- matrix(NA, ncol=6, nrow=length(nrange)) for (i in seq_along(nrange)) { ben <- n <- nrange[i] for (n1 in (1:(n-1))) { for (r1 in (0:n1)) { for (r in (r1:n)) { p0 <- bprob(r1, n1, r, n, h0) pa <- bprob(r1, n1, r, n, ha) if (p0 > (1-alpha) & pa < beta) { pet <- pbinom(r1, n1, h0) en <- n1 + (1-pet) * (n-n1) if (en < ben) { ben <- en rv[i,] <- c(r1, n1, r, n, pet, en) } } } } } } ## Discard non-solutions -- next lines will assume solutions exist! rv <- rv[apply(rv, 1, \(.) !all(is.na(.))), ] ## Minimax: minimize total sample size ## Optimal: minimize expected sample size under H0 (= EN) rv <- rv[c(which.max(rv[,4] == min(rv[,4])), which.max(rv[,6] == min(rv[,6]))),] dimnames(rv) <- list(c("Minimax", "Optimal"), c("r1", "n1", "r", "n", "PET", "EN")) return(rv) } 

You'll have to provide an educated guess for the sample size range -- or wait for the code to crawl through a larger nrange -- but luckily we've already derived the total sample size (and final acceptance threshold) for the "Minimax" solution above:

simon(h0=.50, ha=.70, nrange=35:45) #> r1 n1 r n PET EN #> Minimax 12 23 23 37 0.6611803 27.74348 #> Optimal 8 15 26 43 0.6963806 23.50134 ## Compare versus a proper implementation clinfun::ph2simon(.50, .70, .05, .20) #> r1 n1 r n EN(p0) PET(p0) #> Minimax 12 23 23 37 27.74 0.6612 #> Optimal 8 15 26 43 23.50 0.6964 

I'll just note here that using the above you can identify that the study you referenced was not in fact an optimal design... nor would observing $12/17$ completers lead to rejection of their $H_0$!

$\endgroup$
4
  • $\begingroup$ so in the ph2simon results, i'm using n from minimax, r from minimax, and r from admissible for those 37/23/24 quantities? $\endgroup$ Commented 3 hours ago
  • $\begingroup$ @scott9 no: the n and r from Minimax match the sample size and $H_0$ acceptance threshold from the first part (by design; and exceeding the null acceptance threshold means favouring the alternative). The fact that any others match up is by happenstance. The question is: do you want a two-stage design? I'll reiterate that the above implementations are specified in terms of accepting the null (i.e. r[1] is the value you'd have to exceed to continue the study & declare the alternative respectively). $\endgroup$ Commented 3 hours ago
  • $\begingroup$ Also, by definition the $H_A$ threshold is the one from $H_0$ $+ 1$, in case that wasn't obvious... if you know one you know the other. $\endgroup$ Commented 3 hours ago
  • $\begingroup$ i was thinking that after i posted, just making sure it wasn't more than a weird coincidence. thanks again. $\endgroup$ Commented 43 mins ago

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.