Re: [Algorithms] Best-fit brightness plane

[Alex M. <am...@cs...>]

>The SVD also does this - it factors a matrix into:
>
>U D VT
>
>Where U and VT are orthogonal matrices and D is a diagonal matrix.  The
>pseudo inverse is:

I would just add here that U and VT are more than just orthogonal.  They
are chosen rather specially.  They are orthonormal, and together provide
orthonormal bases for all four fundamental subspaces of the original
matrix A -- the column space, the row space, the nullspace and the left
nullspace.

One way to get some intuition for the decomposition is to think of U and
VT as (essentially) rotation matrices (there's a possible reflection in
there).  Then you can roughly consider the decomposition as a rotation
followed by a nonuniform scale, followed by a final rotation.


>So this means that using the normal equations gives you the same result
>as just using the pseudo-inverse of the orignal system of equations -
>but you don't have to square the condition number of the matrix.  With
>large systmes (or ones where the normal equations has a zero eigenvalue)
>using the SVD (and truncating small singular values) is more robust...

I'll vouch for this too.  In my experience, it's been beneficial to zero
out not only very small (in an absolute sense) singular values, but also
those singular values that are below some small fraction of the largest
singular value (to get rid of singular values that are small in a
relative sense).

Alex