When testing some null versus alternative hypotheses by a test statistic $U(X)$, where $X = \{ x_i, ..., x_n\}$, apply the permutation test with the set $G$ of permutations on $X$ and we have a new statistic $$ T(X) := \frac{\# \{\pi \in G: U(\pi X) \geq U(X)\}}{|G|}. $$

1. What is the benefit of using the permutation test over not using it? I.e. what is it like when the permutation test works?

2. What conditions to make that happen? Such as some conditions on the test statistic $U$ and/or on the null hypothesis?

For example,

1. Should $T(X)$ be equal to the p-value based on $U(X)$, for sample $X$? If yes, why? (references is also appreciated)

2. Should the permutation test make $T(X)$ distribution-free over the null hypotheses? What conditions will make that happen?

3. Should $T(X)$ be uniformly distributed over $[0,1]$? What conditions will make that happen? Notice that when $U(\cdot)$ is a constant function, $T(\cdot)$ is also constant at $1$ and the distribution of $T(X)$ is far from being uniform over $[0,1]$.

Thanks and regards!

When testing some null versus alternative hypotheses by a test statistic $U(X)$, where $X = \{ x_i, ..., x_n\}$, apply the permutation test with the set $G$ of permutations on $X$ and we have a new statistic $$ T(X) := \frac{\# \{\pi \in G: U(\pi X) \geq U(X)\}}{|G|}. $$

1. What is the benefit of using the permutation test over not using it? I.e. what is it like when the permutation test works?

2. What conditions to make that happen? Such as some conditions on the test statistic $U$ and/or on the null hypothesis?

For example,

1. Should $T(X)$ be equal to the p-value based on $U(X)$, for sample $X$? If yes, why? (references is also appreciated)

   The p-value of $U(X)$ is defined as $$\inf_{c \in \mathbb R: U(x) \geq c} \sup_{F \in H} P(U(X) \geq c | X \sim F)$$. If the permutation test is to estimate the permutation distribution of $U(X) | X=x$, how is $T(X)$ equals the p-value of $U(X)$ at $X=x$? Especially, there may be more than one distributions in the null $H$, and $T(X)$ doesn't consider the null distributions one by one and then take $\sup_{F\in H}$ and $\inf_{c: U(x) \geq c}$.

2. Should the permutation test make $T(X)$ distribution-free over the null hypotheses? What conditions will make that happen?

3. Should $T(X)$ be uniformly distributed over $[0,1]$? What conditions will make that happen? Notice that when $U(\cdot)$ is a constant function, $T(\cdot)$ is also constant at $1$ and the distribution of $T(X)$ is far from being uniform over $[0,1]$.

Thanks and regards!