2
$\begingroup$

I am currently reading Casella & Berger and at Paragraph 8.3.1 Error Probabilities and the Power Function It says the following:

Suppose $R$ denotes the rejection region for a test. Then for $\theta_0 \in \Theta_0$, the test will make a mistake if $\mathbf{x} > \in R$, so the probability of a Type I Error is $\mathbb{P}_{\theta}(\mathbf{X} \in R)$. For $\theta \in \Theta_0^c$, the probability of a Type II Error is $\mathbb{P_{\theta}}(\mathbf{X} > \in R^c)$.

So, what I understood is $\mathbf{P}(\text{Sample thrown into Rejection Region}|H_{0} \text{was true})$ is the Type I error. Then, it goes on to give the following definition of the Power Function.

The power function of a hypothesis test with a rejection region $R$ is the function $\theta$ defined by $\beta(\theta)= \mathbb{P}_{\theta}(\mathbf{X} \in R) $

So, does this mean that Power Function is the probability of Type I error? And Type I error (as I understood) is when the test puts the sample into rejection region but the Null Hypothesis was correct (i.e. the sample should've been in the acceptance region) i.e. we falsely rejected the Null Hypothesis.

But in one the statistical inference tutorials I was following said that power function is the probability of rejecting the null hypothesis when alternative hypothesis was true which is equivalent to 1 - P(accepting null hypothesis | alternative was true) which is the same as 1 - P(Type II Error).

So are these two statements equal?

\begin{align} \beta(\theta) &= \mathbb{P}(\text{Reject } H_{0}|H_{0} \text{is True}) \\ &= 1 - \mathbb{P}(\text{Accept }H_{0}|H_{1}\text{ is True}) \\ &= \mathbb{P}(\text{Type I Error}) \\ &= 1 - \mathbb{P}(\text{Type II Error}) \end{align}

Are these equations correct?

Improve this question
$\endgroup$
5
  • $\begingroup$ Power is the complement of the probability of Type II error -- power = $1-\beta$, not $\alpha$. When you moved from the first to the second line of mathematical statements, you made an error. $\endgroup$ Commented Oct 27, 2023 at 23:52
  • $\begingroup$ I've seen many times that the power function is defined in the way Casella and Berger did, i.e., as $\mathbb{P}_{\theta}(\mathbf{X} \in R)$ -- the probability of rejecting $H_0$ for any $\theta \in \Theta_0 \cup \Theta_1$. And that power itself is then defined as the restriction of the power function to $\Theta_1=\Theta_0^c$. $\endgroup$ Commented Oct 28, 2023 at 9:35
  • $\begingroup$ @Glen_b When I moved from the first to the second line, what I meant that that according to C&B, the Power Function is the probability of Type I Error and elsewhere, I saw that Power Function was complement of probability of Type II Error. By that logic, I thought that these two should be equal. Now, I see that its not true. $\endgroup$ Commented Oct 28, 2023 at 10:13
  • $\begingroup$ Yes, some people define the power function to include the rejection rate under the null but I would argue that this terminology is both misleading (you are a case in point) and unnecessary, since there's a perfectly reasonable term for the rejection rate across all $\theta$, the rejection rate function. I define the power function to actually refer to the power, so with a simple null it has a hole in it. I sometimes (loosely) draw the rejection rate function and call it a power curve (even when I don't explicitly draw the hole in) but I wouldn't define the term power function that way. $\endgroup$ Commented Oct 28, 2023 at 16:04
  • $\begingroup$ Yes, it is very confusing. I have another question. Suppose we want to plot $\beta(\theta)$ (where $\beta$ is the power function and $\theta$ is the parameter), would we have two different functions - one for $\theta_0$ and another for $\theta_0^{c}$ i.e. $\beta(\theta|\theta \in \Theta_0)$ and $\beta(\theta|\theta \in \Theta_0^{c})$? $\endgroup$ Commented Oct 28, 2023 at 17:56

1 Answer 1

Reset to default
3
$\begingroup$

Is Power Function the same as Probability of Type I error?

No. Power = $P(\text{reject }H_0|H_0\text{ false})=1-P(\text{fail to reject }H_0|H_0\text{ false})=1-\beta\neq \alpha$.

2x2 table of State of nature (row) vs decision (column). Columns "Don't reject H0" and "Reject H0". Rows "H0 True" and "H0 False". (1,1),Top left cell: "Correct decision. True Negative. Prob: 1 - alpha". (1,2) Top right cell: "Type I error. False Positive. Prob: alpha (sig. level)". (2,1) Bottom left cell: "Type II error. False Negative. Prob: beta". (2,2) Bottom right cell: "Correct decision. True positive. Prob: 1 - beta (power)".
(Larger diagram here. Casella and Berger have a similar table in section 8.3.1 of the second edition)

The definitions you have from the text look like they might be correct apart from an error at the end of the first quote, which should have "$P_θ(\mathbf{X}∈R^c)$".

Note that the situation in which you compute power is when $H_0$ is false ($\theta \in \Theta_0^c$ NOT $\theta \in \Theta_0$).

It is not the case that $\mathbb{P}(\text{Reject } H_{0}|H_{0} \text{ is True}) = 1 - \mathbb{P}(\text{Accept }H_{0}|H_{1}\text{ is True})$ because the state of nature you're conditioning on changed.

It's important to keep in mind that for composite alternatives, the power is typically different at distinct values of the parameter (i.e. at $\theta=\theta_1 \in \Theta_0^c$ and $\theta=\theta_2 \in \Theta_0^c$ the power will typically differ when $\theta_2\neq\theta_1$). In many situations, we tend to speak of a power function, $1-\beta(\theta)$.

Edit: As HeyJude notes in comments, Casella and Berger denote the power function $1-P(\text{reject }H_0|\theta)$ (as a function of $\theta$, with $n$, $\alpha$ etc fixed) as $\beta(\theta)$ in spite of the usual convention that $\beta$ is the type II error rate and $1-\beta$ is the power. This may be confusing if you're not paying close attention to their definition, but they define the function clearly enough. It's unfortunate but similar odd uses of notation happens fairly often across different authors. I'd at least have hoped that they called it $\beta^\prime$ or something.

Improve this answer
$\endgroup$
6
  • 2
    $\begingroup$ "Note that the situation in which you compute power is when $H_{0}$ is false $(\theta \in \Theta^c_{0}$ NOT $\theta \in \Theta_{0}$)." - This statement helps a lot now! So, $\mathbb{P}_{\theta}(\mathbf{X} \in \theta) = \mathbb{P}(\text{Type I Error})$ iff $\theta \in \Theta_{0}$ otherwise if $\theta \in \Theta_{0}^c$, then $\mathbb{P}_{\theta}(\mathbf{X} \in \theta) = 1 - \mathbb{P}(\text{Type II Error})$. This was the source of confusion. Got it now, thanks! $\endgroup$ Commented Oct 28, 2023 at 10:23
  • $\begingroup$ "we tend to speak of a power function, $1−β(θ)$" - I'm confused as to why does the cited book defines "The power function [...] is defined by $β(θ)$"? $\endgroup$ Commented Dec 14, 2023 at 22:06
  • $\begingroup$ Good question, but as to why they'd choose such an anti-conventional notation for the power function (you're certainly correct that they do, I hadn't noticed before), I can't say. I don't know what their thinking was. You'd have to ask them why, given the usual convention that type 2 error rate is $\beta$ and so power is $1-\beta$ why they'd use $\beta(\theta)$ for the power function rather than for the type II error function. They do clearly define it so it's not ambiguous in context but that's pretty much user-vicious behavior for someone whose not already comfortable with the content. $\endgroup$ Commented Dec 15, 2023 at 1:50
  • $\begingroup$ Well George Casella died so he's a bit hard to ask, but I think Roger Berger's still around. $\endgroup$ Commented Dec 15, 2023 at 1:52
  • $\begingroup$ @Glen_b I understood that with this notation, $\beta(\theta) = 1 - P(\textit{Type II Error})$. But the book also says that $\beta(\theta) = P(X \in R) = P(\textit{Type I Error})$ if $\theta \in \Theta_{0}$. What does this statement mean? $\endgroup$ Commented Apr 24, 2024 at 14:31

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.