Re: [ojAlgo-user] Best way to retreive Diagonal values and to obtain Z-scores?

[Anders P. <an...@op...>]

> On 14 nov. 2015, at 18:27, Uga Buga <mmo...@gm...> wrote:
> 
> Thanks Anders
> 
> When it comes to the z-score I am having difficulties implementing your suggestion.
> I have the data in a PhysicalStore as it will require matrix operations, so the methods (sliceColumn) you've used arent available, and it isn't obvious what's the best way to do this then.

I’ve just now checked in code that will make any MatrixStore “sliceable” just like Array2D


> Should I load in my data into an Array2D first, normalize it, then convert it into a PhysicalStore? How? Would that workflow be efficient? (will be processing GBs of data)

Any Access2D instance can be wrapped in a MatrixStore implementation using: MatrixStore.PRIMITIVE.makeWrapper(Access2D<?>);


> I've tried this, but it didn't work: I've created a PhysicalStore instance from the normalized Array2D, via physicalFactory.columns(tmpArray2D). But that flattened the 2D data into 1D.  It is also unclear if this method allocates new memory, IE creates a copy, or if it just somehow changes the external representation of the same data.
> Any ideas?

You should do:

physicalFactory.copy(tmpArray2D)

The method you used will treat the entire “ tmpArray2D” as one column.


> PS: ojAlgo would disproportionately benefit from a proper tutorial covering workflow and real world uses, beyond the basic 'getting started' ones on GitHub.
> 
> Martin
> 
> On Thu, Nov 12, 2015 at 4:35 PM, Anders Peterson <an...@op...> wrote:
> 
> > On 12 nov. 2015, at 10:59, Uga Buga <mmo...@gm...> wrote:
> >
> > Hi Guys
> >
> > 1) What is the fastest way of getting the diagonal elements of a (square) matrix?
> > (IE extract all the diagonal elements in a square matrix into a vector)?
> > The way I've solved it 'manually' was like this:
> >
> >         double [] diagonal = new double [(int) myMatrix.countColumns()];
> >         for(int i = 0; i < diagonal.length; i++) diagonal[i] = (double) myMatrix.get(i, i);
> >
> > Is there a better/faster (built-in) way of achieving the same thing?
> > (it should give the same results as diag() in R )
> 
> If what you have is a BasicMatrix or a MatrixStore then that’s what you need to do. It’s not slow or bad - it’s just that you had to code it yourself.
> 
> You should do
> 
>         myMatrix.doubleValue(i, i);
> 
> instead of
> 
>         (double) myMatrix.get(i, i);
> 
> 
> If you have an Array2D instead, then you can to this:
> 
>         Array2D<Double> tmpArray2D = … ;
>         Array1D<Double> tmpMainDiagonal = tmpArray2D.sliceDiagonal(0, 0);
>         double[] tmpRawCopyOfMainDiagonal = tmpMainDiagonal.toRawCopy();
> 
> 
> 
> > 2) What is the fastest way to standardise columns or rows? (also known as Z-score, where each entry is part of a standard normal distribution with mean 0, variance 1)
> 
> There is no direct (1-step) way to do this, but you can get some help:
> 
>         Array2D<Double> tmpArray2D = … ;
>         for (long j = 0L; j < tmpArray2D.countColumns(); j++) {
>             Array1D<Double> tmpColumn = tmpArray2D.sliceColumn(0L, j);
>             SampleSet tmpSampleSet = SampleSet.wrap(tmpColumn);
>             double tmpMean = tmpSampleSet.getMean();
>             double tmpStdDev = tmpSampleSet..getStandardDeviation();
>             tmpColumn.modifyAll(new PrimitiveFunction.Unary() {
> 
>                 public double invoke(final double arg) {
>                     return (arg - tmpMean) / tmpStdDev;
>                 }
>             });
>         }
> 
> 
> 
> 
> >
> >
> >
> > Thanks
> > Martin
> >
> > PS: (I've had some problems subscribing so this message may be sent twice, so if this message appears twice admins can delete the first one)
> > ------------------------------------------------------------------------------
> > _______________________________________________
> > ojAlgo-user mailing list
> > ojAlgo-user@lists..sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user