1
$\begingroup$

Consider the linear model $$ Y={\underbrace{X_i}_{K\times 1 }}^\top\beta+U_i $$ and assume

(0) There is no intercept in the model

(1) $E(X_i U_i)=0_K$ [orthogonality]

(2) $E(X_i X_i^\top)$ has rank $K$

(3) We have an i.i.d. sample $\{Y_i, X_i\}_{i=1}^n$

Then, the OLS estimator $$ \hat{\beta}=({\underbrace{X}_{n\times K}}^\top X)^{-1} X^\top \underbrace{Y}_{n\times 1} $$ is consistent.

The sketch of the proof is: by Law of large numbers and continuous mapping theorem, we have $plim_{n\rightarrow \infty} \frac{1}{n}(X^\top X)^{-1}=E(X_i X_i^\top)^{-1}$ and $plim_{n\rightarrow \infty}\frac{1}{n} X^\top \underbrace{(X^\top \beta+U)}_Y =E(X_i X_i^\top)\beta+ E(X_i U_i) $. By combining the two expressions, we have $$ plim_{n\rightarrow \infty} \hat{\beta}=E(X_i X_i^\top)^{-1}E(X_i X_i^\top)\beta + E(X_i X_i^\top)^{-1}\underbrace{E(X_i U_i)}_0= \beta $$

Question:

(a) Observe that if $E( U_i) =0$, then (1) is equivalent to $cov(X_i, U_i)=0$. Hence, orthogonality is equal to zero covariance. However, if $E( U_i) \neq 0$, then $cov(X_i, U_i) $ may be different from zero. Hence, orthogonality is not equal to zero covariance in this second setting. Still, the consistency proof goes through. Hence, orthogonality is sufficient for consistency and does not require $E( U_i) =0$. Is this correct?

(b) Let's think about the reverse: is orthogonality necessary for consistency? That is, suppose $E(X_i U_i)\neq 0$ but $cov(X_i, U_i)=0$ because $E(U_i)=0$. Is $\hat{\beta}$ inconsistent? Can you show it in the multidimensional case ($K>1)$?

Note: I have read several questions on orthogonality versus zero covariance (for example, here), but they have not cleared my doubt as they look to generic.

Matlab simulation which shows consistency with no intercept, $E(U_i)\neq 0$, $E(X_i)\neq 0$, $cov(X_i, U_i)\neq 0$, $E(X_i U_i)=0$.

clear rng default J=10^4; beta_OLS_temp=zeros(J,1); r=10^7; beta=2.5; k=-4/5; h=2; for j=1:J X=unifrnd(-1,2,r,1); U=k*(X.^2)+h; Y=X*beta+U; beta_OLS_temp(j,:)=(X.'*X)^(-1)*(X.'*Y); end beta_OLS=sum(beta_OLS_temp(:,1))/J;
Improve this question
$\endgroup$
8
  • $\begingroup$ Hi: If $E(U_{i}) \neq 0$, then the model has an error term with a non-zero mean. Intuitively, that would make the $\beta$ estimate not consistent and your equations bear that intuition out. $\endgroup$ Commented Jul 28, 2022 at 19:31
  • $\begingroup$ Why? Please can you show it formally? I never use $E(U_i)=0$ in my proof. This is indeed the point I make in my question! $\endgroup$ Commented Jul 28, 2022 at 19:35
  • $\begingroup$ Hi TEX: You use $E(X_{i}U_{i}) = 0$ in your proof and I assume that that is based on the orthogonality assumption which is based on $E(U_i) = 0$. Is it not ? If it is, then, if you don't have $E(U_i) = 0$, then you won't have orthogonality so that last term in your last equation won't be 0 ? $\endgroup$ Commented Jul 29, 2022 at 3:38
  • $\begingroup$ Why is the orthogonality assumption based on $E(U_i)=0$? Could you explain? It does not seem so if I read Hayashi p.112 google.com/url?sa=t&source=web&rct=j&url=https://… $\endgroup$ Commented Jul 29, 2022 at 4:18
  • $\begingroup$ Also read Hayashi p.109 $\endgroup$ Commented Jul 29, 2022 at 4:23

1 Answer 1

Reset to default
-1
$\begingroup$

Orthogonality is indeed, as the derivation demonstrates, sufficient for consistency.

If $X_i$ does not contain a constant, orthogonality however no longer automatically implies that $E(U_i)=0$ (see the comments below for details).

If we then do have $E(U_i)\neq0$, then whether the OLSE is consistent will depend on other features of the DGP. The OP offers an example where consistency obtains.

On the other hand, take, for example, $X_i$ and $U_i$ to be independent. Then, $$E(X_iU_i)=E(X_i)E(U_i)\neq0,$$ which would imply inconsistency when $E(U_i)\neq0$ - unless the DGP is such that $E(X_i)=0$, which would restore orthogonality.

More specifically, by the OP's, derivation, OLS will, in the case of a simple regression, plim to $$ \beta+\frac{E(X_i)E(U_i)}{E(X_i^2)} $$ Here is a numerical illustration. Note that for $X_i\sim N(\mu,1)$, $E(X_i^2)=1+\mu^2$ from https://en.wikipedia.org/wiki/Noncentral_chi-squared_distribution.

Hence, the above plim becomes $$ \beta+\frac{\mu E(U_i)}{1+\mu^2} $$

n <- 5000 ols.wocst.nonzeromean <- function(n){ x <- rnorm(n, 3) u <- rnorm(n, 2) y <- 4*x+u coef(lm(y~x-1)) } mc <- replicate(10000, ols.wocst.nonzeromean(n)) summary(mc) # 4+2*3/(1+9)=4.6 > Min. 1st Qu. Median Mean 3rd Qu. Max. 4.582 4.597 4.600 4.600 4.604 4.623
Improve this answer
$\endgroup$
10
  • $\begingroup$ Thanks. Can you conclude your answer by addressing my (a) and (b) questions? Thanks $\endgroup$ Commented Jul 30, 2022 at 15:11
  • $\begingroup$ My above answer indeed only adresses a), arguing that a nonzero-mean error term and a regression without constant generally do not produce consistency, as it will violate $E(X_iU_i)=0$. As to b), your derivation shows that consistency obtains only if $E(X_iU_i)=0$. There does not seem to be anything special to show in the multidimensional case. $\endgroup$ Commented Jul 30, 2022 at 15:32
  • $\begingroup$ It is just that when $X_i$ does contain a constant (say, $X_i=(1, D_i)$), then (we are then in a multidimensional case!) $E(X_iU_i)=0$ also implies $E(1U_i)=E(U_i)=0$. Also, $E(X_iU_i)=0$ then implies $E(D_iU_i)=0$ and then also $Cov(D_i,U_i)=E(D_iU_i)-E(D_i)E(U_i)=0-0E(D_i)=0$. $\endgroup$ Commented Jul 30, 2022 at 15:32
  • $\begingroup$ I am not sure the example in your answer makes sense as you are simulating a DGP where $E(U_i X_i)$ is different from zero. You are just showing that if $E(U_i X_i)\neq 0$ we have inconsistency. I don't understand what you want to prove. $\endgroup$ Commented Jul 30, 2022 at 15:56
  • $\begingroup$ Maybe I misunderstand your question. My point indeed precisely is that it is not clear how to get $E(X_iU_i)=0$ when $E(U_i)\neq0$. So clearly, orthogonality is sufficient for consistency, but my example shows that such orthogonality does not generally obtain for $E(U_i)\neq0$, so that your claim that we do not need $E(U_i)=0$ when there is no constant requires care. $\endgroup$ Commented Jul 30, 2022 at 16:13

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.