FPGA Central

Does anyone know of a good resource that explains this well?
Thanks

James

"jquest" <[email protected]> writes:

> Does anyone know of a good resource that explains this well?
> Thanks

A good introductory text is

@BOOK{anton,
title = "{Elementary Linear Algebra with Applications}",
author = "{Howard~Anton, Chris~Rorres}",
publisher = "John Wiley and Sons, Inc.",
year = "1987"}
--
% Randy Yates % "Maybe one day I'll feel her cold embrace,
%% Fuquay-Varina, NC % and kiss her interface,
%%% 919-577-9882 % til then, I'll leave her alone."
%%%% <[email protected]> % 'Yours Truly, 2095', *Time*, ELO
http://home.earthlink.net/~yatescr

For a quick reference, I would use Wikipedia.

http://en.wikipedia.org/wiki/Eigenvalue

There's a good visualization of eigenvectors/values, with a lecture,
at: http://ocw.mit.edu/OcwWeb/Mathematic...ng-2005/Tools/

If you don't understand the basics of this lecture (i.e. about
matrices, determinants and real/imaginary values) you need to go
farther back and study them. It shouldn't take much time to understand
the basics well-enough to understand this lecture.

It's pretty tough to just open a book on linear algebra and get an
understanding of eigenavalues. I studied all this formally at one time
and have forgotten most of it. When I studied linear algebra, we
called it "mystery math".

About 2/3 way through the semester, one guy called out to the prof,
what are we doing? The prof was a nice guy and responded, do you mean
today, or in the course? When we all grunted, "in the course", he took
the time to explain what was going on.

Linear algebra has to be approached on a systematic basis by learning
a series of definitions. The purpose of linear algebra is basically to
manipulate the numerical values in equations in an attempt to find a
solution to simultaneous equations.

The good news is that it is only a 3 month course. If you want to
understand eigenvectors/values, you really need to start at the
beginning of a book on linear algebra and get a feeling for what they
are doing. It shouldn't take any more than a few days intensive study
to glean the basics. Our 3 month course comprised 3 hours per week,
and over 3 months, that's no more than 36 hours lecture time.

Be warned, however, that many mathematicians have a bent for speaking
in a language foreign to most of us. There are basically two
approaches to math: one invloves the old fashioned math where things
tend to make physical sense, and then theres the new math where
reality has been subverted to serve the mathematics entity.

If you open a book on linear algebra, and it doesn't make sense, put
it down and find one that does. If the author is speaking in 'new
math', that is, one space is contained in another, or one space is a
subset of another, I'd chuck it. A book on linear algebra should
start out by explaining in plain English that it is about solving
simultaneous equations and get down to it. There's no need for
mathematical gobbeldy gook.

The whole basis of linear algebra is to remove the numerical values
form equations and plug them into a matrix. Then you can operate on
them using certain rules to reduce them. It's quite simple, actually.
One of the tricks, however, and basic function of linear algebra is to
determine whether a solution exists.

On the other side of the coin is vector calculus. Linear algebra
doesn't deal with vectors per se. A vector needs both magnitude and
direction in it's definiton. We took linear algebra and vector
calculus back to back. You don't need to study an entire course of
vector calculus, but it helps to understand how vecors rotate and/or
translate in space. You do calculations on vectors using their scalar
quantities, and those reduce to simultaneous equations.



On Tue, 07 Mar 2006 10:56:34 -0600, "jquest"
<[email protected]> wrote:

>Does anyone know of a good resource that explains this well?
>Thanks
>
>James
>

Let me explain the eigenvalues and eigenvectors in my way (also in
statistics).

Let X be a matrix (n samples and m variables) and A=X'X (m by m
matrix). But singular value decomposition first before
eigenvalue/vector decomposition (EVD), since they are mathematically
connected.

SVD decompose a matrix (X) into three matrices that convey
information about the underlying structure of X (correlation
structure).

X=USV'
A=X'X=(VSU')USV'=VEV', where U'U=V'V=I and E=SS.

1. The first matrix U explains the SIMILARITY (correlation) of
SAMPLES and the columns of U are called eigenvectors of XX'.
2. The second matrix S shows the DEGREE of similarity and the
diagonal elements of S are called singular values and they are in
decreasing order. The sum of squares of singular values are equal to
the sum of eigenvalues.
3. The third matrix V describes the SIMILARITY of VARIABLES and the
columns of V are called eigenvectors of X'X. (They are called
principal components, or loadings in principal component analysis,
modes in structural dynamics, harmonics (?) in acoustics)

In statistical terms, the definition of similarity is correlation
or covariance among variables depending on whether the columns in X are
normalized (the columns have average values of zero) or standardized
(the columns are scaled to have averages of zero and standard deviation
of one). Therefore the columns of V show the correlation structure
among variables. The columns of U show the correlation among samples.

If the X is normalized, the A=X'X is the same as covariance matrix
and if X is standardized, then A=X'X is the same as correlation matrix.


Therefore if EVD is applied to A=X'X (correlation matrix), the sum
of eigenvalues are EQUAL to the number of variables (columns) because
the diagonal elements of X'X are ones. For example if I have 10 by 10
matrix (positive symmetric matrix) after X is standardized, the sum of
eigenvalues are 10 (eigenvalues are the trace of X'X).

If the first eigenvalue is for example 6, then 60 % of variation (6
out of 10) in X'X are explained by the liner combination of the first
eigenvectors in V (called principal components).

By the way, the SVD of X and EVD of X'X are called principal
component analysis in statistics. They are the most important and
fundamental methods because they are applied in almost all quantitative
disciplines (You name it!). They are the necessity in order to compute
the INVERSE matrix because if a matrix X is ill-conditioned (variables
are highly correlated), the inverse matrix is either non-existent or
have infinite number of solutions (very sensitive). In feedback
control system, the eigenvalue shows whether the system is stable or
not.

Well, there are more things to explain but I guess this is enough to
start. ...... Hughhhhh!!!


Hope this helps.

Sangdon Lee, Ph.D.,
GM Tech. Center


jquest wrote:
> Does anyone know of a good resource that explains this well?
> Thanks
>
> James