FPGA Central

Hello,

Consider the problem:
Given observation Y, estimate X via a linear approach: \hat{X}=HY+b
such that

(1) J1=E[ (X-\hat{X})^T (X-\hat{X}) ] is minimized.
(2) J1 is minimized, and we know that Y=Mx + n
(3) J2=E[ (Y- M \hat{X})^T (Y- M \hat{X}) is minimized

Solution for (1):
The solution of this Linear Mean Square Error (LMSE) error problem is
given by b=m_x+H m_y, where H is solution of K_yy H = K_yx.

[Notation: m_x mean of vector X, K_yx=E[ (y-m_y)^T (y-m_y ] ]

Solution for (2):
Calculation of K_yy and K_yx leads to
H=M K_x (MK_xM^T+K_n)^(-1)

Solution for (3):
Solution for this least squares problem is
H=(M^T M)^(-1) M^T

Question:
- What are the differences and common points of the estimator obtained
in case of the LMSE and the least squares case? How are they related?
- Solution for the weighted least squares problem is given by H=(M^T
S^(-1) M)^(-1) M^T S^(-1) , where for weighting S often the
autocorrelation matrix K_n is taken. Can we motivate the choice of the
weight S by statistical considerations?

Thank you!

Michael

[email protected] said the following on 17/08/2006 12:53:
> Consider the problem:
> Given observation Y, estimate X via a linear approach: \hat{X}=HY+b
> such that
>
> (1) J1=E[ (X-\hat{X})^T (X-\hat{X}) ] is minimized.
> (2) J1 is minimized, and we know that Y=Mx + n
> (3) J2=E[ (Y- M \hat{X})^T (Y- M \hat{X}) is minimized
>
> Solution for (1):
> The solution of this Linear Mean Square Error (LMSE) error problem is
> given by b=m_x+H m_y, where H is solution of K_yy H = K_yx.
>
> [Notation: m_x mean of vector X, K_yx=E[ (y-m_y)^T (y-m_y ] ]
>
> Solution for (2):
> Calculation of K_yy and K_yx leads to
> H=M K_x (MK_xM^T+K_n)^(-1)
>
> Solution for (3):
> Solution for this least squares problem is
> H=(M^T M)^(-1) M^T
>
> Question:
> - What are the differences and common points of the estimator obtained
> in case of the LMSE and the least squares case? How are they related?

(3) is not a least-squares (LS) problem, it's still a LMSE problem. The
fundamental difference between LMSE and LS is that LMSE minimises the
expected error, whereas LS only minimises the sample error.

The cost function for a LMSE problem is of the form:

J(K) = E | x - K y |^2

The cost function for a LS problem is of the form:

J(K) = | x - K y |^2



--
Oli