1
$\begingroup$

Suppose I have a complex (covariance) matrix $\Sigma = \mathbb{E}[XX^*]$, where $X \in \mathbb{C}^n$ is a random vector with distribution $F$. I know that $\Sigma$ is positive semi-definite, with a simple proof, but I'm not sure about extending this to proving positive definiteness. In particular, I am thinking of a counter-example, but I may be misusing the linearity of $\mathbb{E}$.

A matrix $\Sigma$ is positive definite if $z^*\Sigma z > 0$ for all nonzero $z \in \mathbb{C}^n$. We can prove positive semi-definiteness by demonstrating \begin{align*} z^*\mathbb{E}[XX^*]z &= \mathbb{E}[z^*XX^*z]\\ &= \mathbb{E}[z^*X(z^*X)^*]\\ &= \mathbb{E}\left[|z^*X|^2\right] \geqslant 0. \end{align*} Now let $z$ be a sparse random vector such that, if $X = (x_1,\ldots,x_n)^T$, then $z = ((x_1^*)^{-1}, -(x_2^*)^{-1},0,\ldots,0)^T$. Clearly, $z$ is nonzero and $z^*X = x_1x_1^{-1} - x_2x_2^{-1} = 0$, thus $\mathbb{E}\left[|z^*X|^2\right] = 0$ and the matrix cannot be positive definite.

  1. I am not sure that, if $z$ is a random vector, it can "go inside" of the $\mathbb{E}[\,\cdot\,]$ operator (thus the counter-example is wrong).

  2. I am not sure that I can create $z \sim F'$ (where $F'$ has the same probability distribution as $F$ but the vectors drawn from it are sparse and have inverted, conjugated elements, as demonstrated).

  3. If this counter-example is indeed incorrect, then I believe I can prove positive definiteness, but it relies on me knowing the specifics of $F$. Am I correct in assuming that, in general (i.e. without the details of $F$), one cannot prove that $\Sigma$ is positive definite?

And if there is something else more glaringly wrong about my counter-example, please let me know. I read offhandedly that a covariance matrix is positive definite if the variables are linearly independent, but I have been unsuccessful in finding a source for that statement (with proof). If someone could provide that, I would be grateful as well.

$\endgroup$
4
  • $\begingroup$ I'm fairly sure that 1) your z cannot depend on x since it is not a random vector and 2) your covariance can be singular but only if the distribution is trapped in a lower dimension, for example if the last entry of x is 0 with probability 1. But this is in a sense the only way for it to be singular $\endgroup$ Commented Jun 8, 2016 at 17:12
  • $\begingroup$ @davik, So in this conext, does the condition of definiteness require all nonzero, non-random vectors $z$? $\endgroup$ Commented Jun 8, 2016 at 17:33
  • $\begingroup$ It makes no sense for z to be random, you are showing the matrix, which is just a regular matrix, is positive definite. $\endgroup$ Commented Jun 8, 2016 at 17:38
  • $\begingroup$ @davik, Got it, makes sense. Thanks! $\endgroup$ Commented Jun 8, 2016 at 17:42

1 Answer 1

Reset to default
4
$\begingroup$

For $\Sigma = \mathbb E[X X^*]$ to fail to be positive definite, there would have to be a nonzero $z \in \mathbb C^n$ with $0 = z^* \Sigma z = \mathbb E[z^* X X^* z] = \mathbb E[\|z^* X\|^2]$. Now for a nonnegative random variable to have expected value $0$, it must be $0$ almost surely. That is, there must be some $z$ such that, with probability $1$, $X$ lies in the hyperplane $\{x: z^* x = 0\}$. That may or may not be true, depending on the distribution $F$. It's easy to construct examples where your "random" vector $X$ always lies in a particular hyperplane.

$\endgroup$
3
  • $\begingroup$ In my case, 0 is not even an option in my set of possible outcomes for $x_i$, so I believe that should prove that my $\Sigma \succ 0$. Thanks! $\endgroup$ Commented Jun 8, 2016 at 17:41
  • $\begingroup$ Just for slight clarification: when you say $z^*x = 0$ in the definition of the hyperplane, is $z^*$ the vector itself or an element of the vector? (That is, is the "0" a scalar or the zero vector?) $\endgroup$ Commented Jun 8, 2016 at 17:48
  • $\begingroup$ $z^*$ is a row vector (the $*$ denotes conjugate transpose). $z^* x$ is a scalar. $\endgroup$ Commented Jun 8, 2016 at 18:00

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.