Skip to main content
added 83 characters in body
Source Link
Royi
  • 21k
  • 4
  • 200
  • 242

Your formulation:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$

Has 2 elements:

  1. The Fidelity Term
    This is basically measurements term with the model of AWGN with IID noise.
  2. The Regularization Term
    This is a sparse promoting model by using the Laplace Distribution as a prior.

Since your measurement model is not IID but with different variance per measurement what you need is use Mahalanobis Distance in the measurement model (Which matches the Multivariate Gaussian Distribution).

Using some variation of the norm (See Norm with Symmetric Positive Definite Matrix) which is defined as:

$$ {\left\| \boldsymbol{x} \right\|}_{W}^{2} = \boldsymbol{x}^{T} W \boldsymbol{x}, \; \boldsymbol{x} \in \mathbb{R}^{n}, W \in \mathbb{S}_{++}^{n} $$

You may formulate your problem as:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{ {C}^{-1} }^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$

Where $ C $ is your covariance matrix (Basically $ \operatorname{diag} \left( \boldsymbol{c} \right) $ on your question).
The above is easy to solve using many methods as it is basically transformed LS by using Cholesky Decomposition of the Covariance Matrix.

Your formulation:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$

Has 2 elements:

  1. The Fidelity Term
    This is basically measurements term with the model of AWGN with IID noise.
  2. The Regularization Term
    This is a sparse promoting model by using the Laplace Distribution as a prior.

Since your measurement model is not IID but with different variance per measurement what you need is use Mahalanobis Distance in the measurement model (Which matches the Multivariate Gaussian Distribution).

Using some variation of the norm (See Norm with Symmetric Positive Definite Matrix) which is defined as:

$$ {\left\| \boldsymbol{x} \right\|}_{W}^{2} = \boldsymbol{x}^{T} W \boldsymbol{x}, \; \boldsymbol{x} \in \mathbb{R}^{n}, W \in \mathbb{S}_{++}^{n} $$

You may formulate your problem as:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{ {C}^{-1} }^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$

Where $ C $ is your covariance matrix.
The above is easy to solve using many methods as it is basically transformed LS by using Cholesky Decomposition of the Covariance Matrix.

Your formulation:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$

Has 2 elements:

  1. The Fidelity Term
    This is basically measurements term with the model of AWGN with IID noise.
  2. The Regularization Term
    This is a sparse promoting model by using the Laplace Distribution as a prior.

Since your measurement model is not IID but with different variance per measurement what you need is use Mahalanobis Distance in the measurement model (Which matches the Multivariate Gaussian Distribution).

Using some variation of the norm (See Norm with Symmetric Positive Definite Matrix) which is defined as:

$$ {\left\| \boldsymbol{x} \right\|}_{W}^{2} = \boldsymbol{x}^{T} W \boldsymbol{x}, \; \boldsymbol{x} \in \mathbb{R}^{n}, W \in \mathbb{S}_{++}^{n} $$

You may formulate your problem as:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{ {C}^{-1} }^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$

Where $ C $ is your covariance matrix (Basically $ \operatorname{diag} \left( \boldsymbol{c} \right) $ on your question).
The above is easy to solve using many methods as it is basically transformed LS by using Cholesky Decomposition of the Covariance Matrix.

Source Link
Royi
  • 21k
  • 4
  • 200
  • 242

Your formulation:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$

Has 2 elements:

  1. The Fidelity Term
    This is basically measurements term with the model of AWGN with IID noise.
  2. The Regularization Term
    This is a sparse promoting model by using the Laplace Distribution as a prior.

Since your measurement model is not IID but with different variance per measurement what you need is use Mahalanobis Distance in the measurement model (Which matches the Multivariate Gaussian Distribution).

Using some variation of the norm (See Norm with Symmetric Positive Definite Matrix) which is defined as:

$$ {\left\| \boldsymbol{x} \right\|}_{W}^{2} = \boldsymbol{x}^{T} W \boldsymbol{x}, \; \boldsymbol{x} \in \mathbb{R}^{n}, W \in \mathbb{S}_{++}^{n} $$

You may formulate your problem as:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{ {C}^{-1} }^{2} + \lambda {\left\| \boldsymbol{x} \right\|}_{1} $$

Where $ C $ is your covariance matrix.
The above is easy to solve using many methods as it is basically transformed LS by using Cholesky Decomposition of the Covariance Matrix.