ojalgo-user — For ojAlgo users

Re: [ojAlgo-user] apparent bug in Gamma distribution

[Chris L. <cg...@gm...>]

Great, thanks.

Re the Weibull, it looks like it's a problem with the test; I should
have noticed that one can't draw samples with those parameters
(.01,.01) and expect doubles to contain the result. If you change the
parameter floor to .2 and relax the sensitivity of the variance
unit-test parameter (perhaps to the point of ignoring it for the
really long-tailed Weibull distributions) it should pass.

Chris

Re: [ojAlgo-user] apparent bug in Gamma distribution

[Anders P. <an...@op...>]

Thanks for the contribution!

I've included your additions to RandomNumberTest.

And then...

1) There is now a RandomUtils.gamma(x) function - with junit tests
2) Weibull mean and variance are now calculated analytically, and tested that it matches Exponential if shape=1.0
3) Your testWeibull method still fails. The resulting values are very large. I believe that's part of the problem

Could you have a look at that to see what needs to be fixed; the Weibull class or the test case?

/Anders


On 19 jun 2012, at 20:05, Chris Lucas wrote:

> Thanks for the new snapshot. The gamma distribution looks good.
> 
> I've extended RandomNumberTest a bit:
> 
> * Added a template for checking the mean and variance of a sample
> against the analytic values for two-parameter distributions, using a
> tiny inner-class wrapper.
> * Applied said template to Gamma, Normal, and Weibull dists.
> * Added couple of miscellaneous tests. I noticed that the Weibull mean
> and variance are not computed analytically, which could be an issue
> for users of that distribution.
> 
> The attached file contains those changes. Note that the package name
> is what I used in my own testing, and will need to be changed if you
> replace the current RandomNumberTest.java with it.
> 
> Chris
> 
> On Tue, Jun 5, 2012 at 6:35 PM, Chris Lucas <cg...@gm...> wrote:
>> Hi --
>> 
>> I've noticed what appears to be a bug in org.ojalgo.random.Gamma. I
>> noticed that a Gamma(1,2) getDoubleValue() was returning negative
>> values. After looking at the documentation (thinking it was just a
>> non-standard parameterization at work) and confirming that the
>> empirical means don't align with the value of getExpected(), I looked
>> at the source (via
>> http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/DevProj/src/org/ojalgo/random/Gamma.java?revision=1.14&view=markup)
>> and found the following definition for generate() [which is called by
>> getDoubleValue() in RandomNumber()]:
>> 
>> 60    @Override
>> 61      protected double generate() {
>> 62
>> 63      double tmpVal = ZERO;
>> 64
>> 65      for (int i = 0; i < myCount; i++) {
>> 66      tmpVal += this.random().nextDouble();
>> 67      }
>> 68
>> 69      return -tmpVal / myLambda;
>> 70      }
>> 
>> The function this.random() is inherited from RandomNumber, and appears
>> to be a uniform distribution. Am I missing something? If not, this
>> appears to be a fairly serious bug for anyone who wants to use the
>> random number libraries.
>> 
>> More generally, it appears that the test coverage in the random number
>> unit tests  (at
>> http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/TestProj/src/org/ojalgo/random/RandomNumberTest.java?revision=1.11&view=markup)
>> is fairly limited. Perhaps some tests could be added to establish that
>> large-sample means are within certain easy-to-satisfy tolerances of
>> the intended expected values?
>> 
>> Chris
> <RandomNumberTest.java>------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user

Re: [ojAlgo-user] apparent bug in Gamma distribution

[Chris L. <cg...@gm...>]

Thanks for the new snapshot. The gamma distribution looks good.

I've extended RandomNumberTest a bit:

* Added a template for checking the mean and variance of a sample
against the analytic values for two-parameter distributions, using a
tiny inner-class wrapper.
* Applied said template to Gamma, Normal, and Weibull dists.
* Added couple of miscellaneous tests. I noticed that the Weibull mean
and variance are not computed analytically, which could be an issue
for users of that distribution.

The attached file contains those changes. Note that the package name
is what I used in my own testing, and will need to be changed if you
replace the current RandomNumberTest.java with it.

Chris

On Tue, Jun 5, 2012 at 6:35 PM, Chris Lucas <cg...@gm...> wrote:
> Hi --
>
> I've noticed what appears to be a bug in org.ojalgo.random.Gamma. I
> noticed that a Gamma(1,2) getDoubleValue() was returning negative
> values. After looking at the documentation (thinking it was just a
> non-standard parameterization at work) and confirming that the
> empirical means don't align with the value of getExpected(), I looked
> at the source (via
> http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/DevProj/src/org/ojalgo/random/Gamma.java?revision=1.14&view=markup)
> and found the following definition for generate() [which is called by
> getDoubleValue() in RandomNumber()]:
>
> 60    @Override
> 61      protected double generate() {
> 62
> 63      double tmpVal = ZERO;
> 64
> 65      for (int i = 0; i < myCount; i++) {
> 66      tmpVal += this.random().nextDouble();
> 67      }
> 68
> 69      return -tmpVal / myLambda;
> 70      }
>
> The function this.random() is inherited from RandomNumber, and appears
> to be a uniform distribution. Am I missing something? If not, this
> appears to be a fairly serious bug for anyone who wants to use the
> random number libraries.
>
> More generally, it appears that the test coverage in the random number
> unit tests  (at
> http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/TestProj/src/org/ojalgo/random/RandomNumberTest.java?revision=1.11&view=markup)
> is fairly limited. Perhaps some tests could be added to establish that
> large-sample means are within certain easy-to-satisfy tolerances of
> the intended expected values?
>
> Chris

Re: [ojAlgo-user] apparent bug in Gamma distribution

[Anders P. <an...@op...>]

There's now a v32.7 snapshot. It has both Gamma and Erlang distributions. /Anders

On 6 jun 2012, at 11:17, Anders Peterson wrote:

> Indeed, that's a bug. Thank you for pointing it out!
> 
> I'll release a new snapshot in a couple of days with that bug fixed.
> 
> ... and you're right about the test coverage for that package - it could be better. If you're interested/willing you could help out here. The class SampleSet may be useful when creating test cases.
> 
> /Anders
> 
> 
> On 6 jun 2012, at 00:35, Chris Lucas wrote:
> 
>> Hi --
>> 
>> I've noticed what appears to be a bug in org.ojalgo.random.Gamma. I
>> noticed that a Gamma(1,2) getDoubleValue() was returning negative
>> values. After looking at the documentation (thinking it was just a
>> non-standard parameterization at work) and confirming that the
>> empirical means don't align with the value of getExpected(), I looked
>> at the source (via
>> http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/DevProj/src/org/ojalgo/random/Gamma.java?revision=1.14&view=markup)
>> and found the following definition for generate() [which is called by
>> getDoubleValue() in RandomNumber()]:
>> 
>> 60    @Override
>> 61 	protected double generate() {
>> 62 	
>> 63 	double tmpVal = ZERO;
>> 64 	
>> 65 	for (int i = 0; i < myCount; i++) {
>> 66 	tmpVal += this.random().nextDouble();
>> 67 	}
>> 68 	
>> 69 	return -tmpVal / myLambda;
>> 70 	}
>> 
>> The function this.random() is inherited from RandomNumber, and appears
>> to be a uniform distribution. Am I missing something? If not, this
>> appears to be a fairly serious bug for anyone who wants to use the
>> random number libraries.
>> 
>> More generally, it appears that the test coverage in the random number
>> unit tests  (at
>> http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/TestProj/src/org/ojalgo/random/RandomNumberTest.java?revision=1.11&view=markup)
>> is fairly limited. Perhaps some tests could be added to establish that
>> large-sample means are within certain easy-to-satisfy tolerances of
>> the intended expected values?
>> 
>> Chris
>> 
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and 
>> threat landscape has changed and how IT managers can respond. Discussions 
>> will include endpoint security, mobile security and the latest in malware 
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> ojAlgo-user mailing list
>> ojA...@li...
>> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
>> 
>> 
> 
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> 

Re: [ojAlgo-user] apparent bug in Gamma distribution

[Anders P. <an...@op...>]

Indeed, that's a bug. Thank you for pointing it out!

I'll release a new snapshot in a couple of days with that bug fixed.

.. and you're right about the test coverage for that package - it could be better. If you're interested/willing you could help out here. The class SampleSet may be useful when creating test cases.

/Anders


On 6 jun 2012, at 00:35, Chris Lucas wrote:

> Hi --
> 
> I've noticed what appears to be a bug in org.ojalgo.random.Gamma. I
> noticed that a Gamma(1,2) getDoubleValue() was returning negative
> values. After looking at the documentation (thinking it was just a
> non-standard parameterization at work) and confirming that the
> empirical means don't align with the value of getExpected(), I looked
> at the source (via
> http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/DevProj/src/org/ojalgo/random/Gamma.java?revision=1.14&view=markup)
> and found the following definition for generate() [which is called by
> getDoubleValue() in RandomNumber()]:
> 
> 60    @Override
> 61 	protected double generate() {
> 62 	
> 63 	double tmpVal = ZERO;
> 64 	
> 65 	for (int i = 0; i < myCount; i++) {
> 66 	tmpVal += this.random().nextDouble();
> 67 	}
> 68 	
> 69 	return -tmpVal / myLambda;
> 70 	}
> 
> The function this.random() is inherited from RandomNumber, and appears
> to be a uniform distribution. Am I missing something? If not, this
> appears to be a fairly serious bug for anyone who wants to use the
> random number libraries.
> 
> More generally, it appears that the test coverage in the random number
> unit tests  (at
> http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/TestProj/src/org/ojalgo/random/RandomNumberTest.java?revision=1.11&view=markup)
> is fairly limited. Perhaps some tests could be added to establish that
> large-sample means are within certain easy-to-satisfy tolerances of
> the intended expected values?
> 
> Chris
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> 

[ojAlgo-user] apparent bug in Gamma distribution

[Chris L. <cg...@gm...>]

Hi --

I've noticed what appears to be a bug in org.ojalgo.random.Gamma. I
noticed that a Gamma(1,2) getDoubleValue() was returning negative
values. After looking at the documentation (thinking it was just a
non-standard parameterization at work) and confirming that the
empirical means don't align with the value of getExpected(), I looked
at the source (via
http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/DevProj/src/org/ojalgo/random/Gamma.java?revision=1.14&view=markup)
and found the following definition for generate() [which is called by
getDoubleValue() in RandomNumber()]:

60    @Override
61 	protected double generate() {
62 	
63 	double tmpVal = ZERO;
64 	
65 	for (int i = 0; i < myCount; i++) {
66 	tmpVal += this.random().nextDouble();
67 	}
68 	
69 	return -tmpVal / myLambda;
70 	}

The function this.random() is inherited from RandomNumber, and appears
to be a uniform distribution. Am I missing something? If not, this
appears to be a fairly serious bug for anyone who wants to use the
random number libraries.

More generally, it appears that the test coverage in the random number
unit tests  (at
http://ojalgo.cvs.sourceforge.net/viewvc/ojalgo/TestProj/src/org/ojalgo/random/RandomNumberTest.java?revision=1.11&view=markup)
is fairly limited. Perhaps some tests could be added to establish that
large-sample means are within certain easy-to-satisfy tolerances of
the intended expected values?

Chris

Re: [ojAlgo-user] ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution

[Anders P. <an...@op...>]

Then, can you send me a (self-contained directly-runnable) test case for that bug, please?


I uploaded a new snapshot yesterday.

There's been changes to the classes/interfaces Result and Solution. Right now there is only a class Optimisation.Result, and its method getSolution() is deprecated (although I forgot to mark it as such).

Further the methods minimise() and maximise() in ExpressionsBasedModel now return Optimisation.Result instances rather than just the objective function value as before.


/Anders


On 29 maj 2012, at 08:43, Reda-Jürg MESSIKH wrote:

> Hi Anders, 
> 
> Thanks for your message. 
> 
> Concerning the feasability test, I already took the road you mention. I just have to retest the ojalgo linear solver. 
> 
> For the second point, unfortunately, that is exactely what happen. 
> 
> Reda 
> 
> 
> 
> 
> From:        "Anders Peterson" <an...@op...> 
>                 
> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> Date:        26.05.2012 23:01 
> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> 
> 
> 
> On 25 maj 2012, at 17:01, Reda-Jürg MESSIKH wrote:
> 
> > Hi Anders, 
> > 
> > I just retested your latest snapshot. A lot of bugs that I had before are not repaired.
> 
> I am not aware of these problems. There are no failing unit tests related to problems you (or anyone else) has reported to me. Please supply we with test cases, and descriptions of the problems.
> 
> 
> > But, I have found this situation with an odd behaviour: 
> > 
> > The quadratic solver after a long time says there is a solution and gives a wrong solution. 
> > 
> > But, actually, this particular problem has infeasible constraints.
> 
> An infeasible set of inequality constraints is a bit of a problem for the quadratic solver. I'm not sure how to solve it. Currently you have two alternatives:
> 
> 1) You can check feasibility with the linear solver. If you build your problems using ExpressionsBasedModel you can feed that model to either/both the linear or/and quadratic solvers.
> 
> 2) Add slack variables to your "model" so that you do not have to create any inequalities (no AI and BI matrices) and see how the quadratic solver can handle that. Not sure this is a good idea but you can try it.
> 
> The solvers have time and iterations limits. If these are reached without finding a feasible solution then the solver should definitely not report "optimal". Is that what appends in your case?
> 
> 
> When/if you send me test cases please make things easy for me. Send me complete ready to run test cases. Everything else is too much work on my side.
> 
> 
> /Anders
> 
> 
> > Regards, 
> > Reda 
> > 
> > From:        "Anders Peterson" <an...@op...> 
> >                 
> > To:        Reda-Jürg MESSIKH <rme...@pi...> 
> > Date:        14.05.2012 09:45 
> > Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> > 
> > 
> > 
> > There was a problem with the LInearSolver and constraints with negative valued upper limits. I've released v32.2 with that fixed.
> > 
> > Also; feeding the QuadraticSolver an initial iteration point (feasible solution) is not always helpful. In a general case you're probably better off not doing this.
> > 
> > /Anders 
> > 
> > 
> > On 10 maj 2012, at 09:33, Anders Peterson wrote:
> > 
> > > You're right ojAlgo currently does not have a direct/smart way to detect infeasibility of the inequality constraints in a quadratic program. The LinearSolver should be able to do that for you, and if you get a feasible solution from the linear solver you can feed that as a starting point for the quadratic solver.
> > > 
> > > /Anders
> > > 
> > > 
> > > On 9 maj 2012, at 18:00, Reda-Jürg MESSIKH wrote:
> > > 
> > >> H Anders, 
> > >> 
> > >> I found yet another odd behaviour. 
> > >> 
> > >> The problem defined with the data below (same notation as the former emails) has no solution (the inequalities are impossible to satisfy). 
> > >> 
> > >> 
> > >> 
> > >> Matlab detects this impossibility immediately (less than 3seconds) and exits with the right message. 
> > >> 
> > >> OjAlgo takes a really long time (more than 1 minute, I have interupted the execution afterwards). 
> > >> 
> > >> I susepct that there is no preliminary check whether the inequalities are not overly stringent and hence this behaviour. If you agree with this point, wouldn't it be a good thing to add such preliminary checks ? 
> > >> 
> > >> 
> > >> Reda 
> > >> 
> > >> 
> > >> 
> > >> 
> > >> 
> > >> 
> > >> From:        "Anders Peterson" <an...@op...> 
> > >> 
> > >> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> > >> Date:        09.05.2012 10:11 
> > >> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> > >> 
> > >> 
> > >> 
> > >> There is still the issue that ojAlgo and MATLAB gave different solutions. Which is "better"? What does MATLAB say about the ojAlgo solution, is it a feasible solution? What does ojAlgo say about the MATLAB solution...
> > >> 
> > >> 
> > >> On 8 maj 2012, at 17:18, Reda-Jürg MESSIKH wrote:
> > >> 
> > >>> Hi Anders, thanks for the upload. 
> > >>> 
> > >>> As per FinanceUtils, I am not sure what I have done wrong before, now all is fine. 
> > >>> 
> > >>> Reda 
> > >>> 
> > >>> 
> > >>> 
> > >>> 
> > >>> From:        "Anders Peterson" <an...@op...> 
> > >>> 
> > >>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> > >>> Date:        08.05.2012 16:42 
> > >>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> > >>> 
> > >>> 
> > >>> 
> > >>> Just uploaded a new snapshot to sourceforge.
> > >>> 
> > >>> What problem do you have with FinanceUtils? I don't see any.
> > >>> 
> > >>> /Anders
> > >>> 
> > >>> 
> > >>> On 8 maj 2012, at 16:13, Reda-Jürg MESSIKH wrote:
> > >>> 
> > >>>> Hi Anders, 
> > >>>> 
> > >>>> I just did a few tests that are conclusive, so would it be possible to do another build ? 
> > >>>> 
> > >>>> As a side remark, while building your modified source code, we have seen that the financeUtils does not build. 
> > >>>> 
> > >>>> Reda 
> > >>>> 
> > >>>> 
> > >>>> 
> > >>>> 
> > >>>> 
> > >>>> From:        "Anders Peterson" <an...@op...> 
> > >>>> 
> > >>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> > >>>> Date:        08.05.2012 14:52 
> > >>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> > >>>> 
> > >>>> 
> > >>>> 
> > >>>> Now ojAlgo can get the solution in 6s...
> > >>>> 
> > >>>> I believe I just found a bug that must have been there for years. Get the latest code from CVS, and see if there are any (other) problems with your use cases.
> > >>>> 
> > >>>> /Anders
> > >>>> 
> > >>>> [attachment "ojalgo_solution.txt" deleted by Reda-Jürg MESSIKH/GVA/PCO] 
> > >>>> 
> > >>>> On 8 maj 2012, at 14:15, Reda-Jürg MESSIKH wrote:
> > >>>> 
> > >>>>> Hi Anders, 
> > >>>>> 
> > >>>>> Thanks for your answer. Indeed, 800s is still much longer than matlab. 
> > >>>>> 
> > >>>>> I will try to use other formulations of the problem and, if time permits, I will check the difference of handling between matlab and ojalgo. 
> > >>>>> 
> > >>>>> The  problem is of  Markowitz type, where Q is the covariance matrix and C (all zeros) are the expected returns of the assets. I have two risk linear constraints and an upper bound for the exposure to each asset. 
> > >>>>> Is there an easie/ better formulation in ojAlgo of such problems ? 
> > >>>>> 
> > >>>>> Reda 
> > >>>>> 
> > >>>>> 
> > >>>>> 
> > >>>>> 
> > >>>>> 
> > >>>>> From:        "Anders Peterson" <an...@op...> 
> > >>>>> 
> > >>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> > >>>>> Date:        08.05.2012 12:24 
> > >>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> > >>>>> 
> > >>>>> 
> > >>>>> 
> > >>>>> Don't have a good/quick solution to this problem...
> > >>>>> 
> > >>>>> I have managed to get a solution from ojAlgo that "it" claims to be optimal, but it's not the same solution as you got from Matlab and it took a very long time to get it (800s).
> > >>>>> 
> > >>>>> [attachment "ojalgo_solution.txt" deleted by Reda-Jürg MESSIKH/GVA/PCO] 
> > >>>>> 
> > >>>>> If you want to try the code modifications I made they're in CVS.
> > >>>>> 
> > >>>>> Is it possible for you to experiment with other problem formulations? Can you express the constraints in some alternate way? What are you optimizing - what does "Q" express?
> > >>>>> 
> > >>>>> BTW: If you have matrices on file / as strings and you're going to put the elements in an ExpressionsBasedModel it's better to create BigDecimals directly from the strings (don't use PrimitiveDenseStore as an intermediate type). If you need some sort of intermediate type use BigDenseStore or Array2D<BigDecimal>.
> > >>>>> 
> > >>>>> /Anders
> > >>>>> 
> > >>>>> 
> > >>>>> On 7 maj 2012, at 19:14, Reda-Jürg MESSIKH wrote:
> > >>>>> 
> > >>>>>> Hi Anders, 
> > >>>>>> 
> > >>>>>> Here is the same stuff but with the matrix hardcoded into the code 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> The matlab solution: 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> Thanks again, 
> > >>>>>> Reda 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> From:        "Anders Peterson" <an...@op...> 
> > >>>>>> 
> > >>>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> > >>>>>> Date:        07.05.2012 13:57 
> > >>>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> On 7 maj 2012, at 13:19, Anders Peterson wrote:
> > >>>>>> 
> > >>>>>>> It would be easier to debug if there was only one problem, can you try to get rid of those NumberFormatException for me? 
> > >>>>>> 
> > >>>>>> I'm assuming this works for you so the problem is os/jvm specific. Instead trying to hunt down what goes wrong send me code where the matrices are hardcoded. After you've read the files, copy the matrices to PrimitiveMatrix instances and the call toString() on them. The output can be copy-pasted in to your code.
> > >>>>>> 
> > >>>>>> /Anders
> > >>>>>> 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> 
> > >>>>>> ________________________________________________________________
> > >>>>>> 
> > >>>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> > >>>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> > >>>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
> > >>>>>> ________________________________________________________________
> > >>>>>> 
> > >>>>>> This document should only be read by those persons to whom it is
> > >>>>>> addressed  and  is  not intended to be relied upon by any person
> > >>>>>> without  subsequent written confirmation of its contents. If you
> > >>>>>> have  received  this  e-mail message in error, please destroy it
> > >>>>>> and delete it from your computer.
> > >>>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
> > >>>>>> modification,  distribution  and/or  publication  of this E-mail
> > >>>>>> message is strictly prohibited.
> > >>>>>> ________________________________________________________________
> > >>>>>> 
> > >>>>>> Pictet reserves the right  to  monitor  and  record business and
> > >>>>>> personal communications. By  responding to an email or call from
> > >>>>>> a Pictet  employee  you  are  taken  to  have  consented to such
> > >>>>>> monitoring and recording.
> > >>>>>> ________________________________________________________________
> > >>>>>> 
> > >>>>>> 
> > >>>>>> <MatrixReader.java><QuadraticDemo.java><matlab_solution.txt>
> > >>>>> 
> > >>>>> 
> > >>>>> 
> > >>>>> ________________________________________________________________
> > >>>>> 
> > >>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> > >>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> > >>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
> > >>>>> ________________________________________________________________
> > >>>>> 
> > >>>>> This document should only be read by those persons to whom it is
> > >>>>> addressed  and  is  not intended to be relied upon by any person
> > >>>>> without  subsequent written confirmation of its contents. If you
> > >>>>> have  received  this  e-mail message in error, please destroy it
> > >>>>> and delete it from your computer.
> > >>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
> > >>>>> modification,  distribution  and/or  publication  of this E-mail
> > >>>>> message is strictly prohibited.
> > >>>>> ________________________________________________________________
> > >>>>> 
> > >>>>> Pictet reserves the right  to  monitor  and  record business and
> > >>>>> personal communications. By  responding to an email or call from
> > >>>>> a Pictet  employee  you  are  taken  to  have  consented to such
> > >>>>> monitoring and recording.
> > >>>>> ________________________________________________________________
> > >>>>> 
> > >>>>> 
> > >>>> 
> > >>>> 
> > >>>> 
> > >>>> ________________________________________________________________
> > >>>> 
> > >>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> > >>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> > >>>> CH-1211 GENEVE 73                        http://www.pictet.com/
> > >>>> ________________________________________________________________
> > >>>> 
> > >>>> This document should only be read by those persons to whom it is
> > >>>> addressed  and  is  not intended to be relied upon by any person
> > >>>> without  subsequent written confirmation of its contents. If you
> > >>>> have  received  this  e-mail message in error, please destroy it
> > >>>> and delete it from your computer.
> > >>>> Any  form of  reproduction, dissemination, copying,  disclosure,
> > >>>> modification,  distribution  and/or  publication  of this E-mail
> > >>>> message is strictly prohibited.
> > >>>> ________________________________________________________________
> > >>>> 
> > >>>> Pictet reserves the right  to  monitor  and  record business and
> > >>>> personal communications. By  responding to an email or call from
> > >>>> a Pictet  employee  you  are  taken  to  have  consented to such
> > >>>> monitoring and recording.
> > >>>> ________________________________________________________________
> > >>>> 
> > >>>> 
> > >>> 
> > >>> 
> > >>> 
> > >>> 
> > >>> ________________________________________________________________
> > >>> 
> > >>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> > >>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> > >>> CH-1211 GENEVE 73                        http://www.pictet.com/
> > >>> ________________________________________________________________
> > >>> 
> > >>> This document should only be read by those persons to whom it is
> > >>> addressed  and  is  not intended to be relied upon by any person
> > >>> without  subsequent written confirmation of its contents. If you
> > >>> have  received  this  e-mail message in error, please destroy it
> > >>> and delete it from your computer.
> > >>> Any  form of  reproduction, dissemination, copying,  disclosure,
> > >>> modification,  distribution  and/or  publication  of this E-mail
> > >>> message is strictly prohibited.
> > >>> ________________________________________________________________
> > >>> 
> > >>> Pictet reserves the right  to  monitor  and  record business and
> > >>> personal communications. By  responding to an email or call from
> > >>> a Pictet  employee  you  are  taken  to  have  consented to such
> > >>> monitoring and recording.
> > >>> ________________________________________________________________
> > >>> 
> > >>> 
> > >> 
> > >> 
> > >> 
> > >> 
> > >> ________________________________________________________________
> > >> 
> > >> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> > >> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> > >> CH-1211 GENEVE 73                        http://www.pictet.com/
> > >> ________________________________________________________________
> > >> 
> > >> This document should only be read by those persons to whom it is
> > >> addressed  and  is  not intended to be relied upon by any person
> > >> without  subsequent written confirmation of its contents. If you
> > >> have  received  this  e-mail message in error, please destroy it
> > >> and delete it from your computer.
> > >> Any  form of  reproduction, dissemination, copying,  disclosure,
> > >> modification,  distribution  and/or  publication  of this E-mail
> > >> message is strictly prohibited.
> > >> ________________________________________________________________
> > >> 
> > >> Pictet reserves the right  to  monitor  and  record business and
> > >> personal communications. By  responding to an email or call from
> > >> a Pictet  employee  you  are  taken  to  have  consented to such
> > >> monitoring and recording.
> > >> ________________________________________________________________
> > >> 
> > >> 
> > >> <Q.txt><C.txt><BE.txt><AE.txt><AI.txt><BI.txt>
> > > 
> > 
> > 
> > 
> > 
> > ________________________________________________________________
> > 
> > PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> > 60, route des Acacias                    Fax  +41 (0)58 323 2324
> > CH-1211 GENEVE 73                        http://www.pictet.com/
> > ________________________________________________________________
> > 
> > This document should only be read by those persons to whom it is
> > addressed  and  is  not intended to be relied upon by any person
> > without  subsequent written confirmation of its contents. If you
> > have  received  this  e-mail message in error, please destroy it
> > and delete it from your computer.
> > Any  form of  reproduction, dissemination, copying,  disclosure,
> > modification,  distribution  and/or  publication  of this E-mail
> > message is strictly prohibited.
> > ________________________________________________________________
> > 
> > Pictet reserves the right  to  monitor  and  record business and
> > personal communications. By  responding to an email or call from
> > a Pictet  employee  you  are  taken  to  have  consented to such
> > monitoring and recording.
> > ________________________________________________________________
> > 
> > 
> > <Q.txt><C.txt><BI.txt><BE.txt><AI.txt><AE.txt>
> 
> 
> 
> 
> ________________________________________________________________
> 
> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> CH-1211 GENEVE 73                        http://www.pictet.com/
> ________________________________________________________________
> 
> This document should only be read by those persons to whom it is
> addressed  and  is  not intended to be relied upon by any person
> without  subsequent written confirmation of its contents. If you
> have  received  this  e-mail message in error, please destroy it
> and delete it from your computer.
> Any  form of  reproduction, dissemination, copying,  disclosure,
> modification,  distribution  and/or  publication  of this E-mail
> message is strictly prohibited.
> ________________________________________________________________
> 
> Pictet reserves the right  to  monitor  and  record business and
> personal communications. By  responding to an email or call from
> a Pictet  employee  you  are  taken  to  have  consented to such
> monitoring and recording.
> ________________________________________________________________
> 
> 

Re: [ojAlgo-user] ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution

[Anders P. <an...@op...>]

On 25 maj 2012, at 17:01, Reda-Jürg MESSIKH wrote:

> Hi Anders, 
> 
> I just retested your latest snapshot. A lot of bugs that I had before are not repaired.

I am not aware of these problems. There are no failing unit tests related to problems you (or anyone else) has reported to me. Please supply we with test cases, and descriptions of the problems.


> But, I have found this situation with an odd behaviour: 
> 
> The quadratic solver after a long time says there is a solution and gives a wrong solution. 
> 
> But, actually, this particular problem has infeasible constraints.

An infeasible set of inequality constraints is a bit of a problem for the quadratic solver. I'm not sure how to solve it. Currently you have two alternatives:

1) You can check feasibility with the linear solver. If you build your problems using ExpressionsBasedModel you can feed that model to either/both the linear or/and quadratic solvers.

2) Add slack variables to your "model" so that you do not have to create any inequalities (no AI and BI matrices) and see how the quadratic solver can handle that. Not sure this is a good idea but you can try it.

The solvers have time and iterations limits. If these are reached without finding a feasible solution then the solver should definitely not report "optimal". Is that what appends in your case?


When/if you send me test cases please make things easy for me. Send me complete ready to run test cases. Everything else is too much work on my side.


/Anders


> Regards, 
> Reda 
> 
> From:        "Anders Peterson" <an...@op...> 
> 
> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> Date:        14.05.2012 09:45 
> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> 
> 
> 
> There was a problem with the LInearSolver and constraints with negative valued upper limits. I've released v32.2 with that fixed.
> 
> Also; feeding the QuadraticSolver an initial iteration point (feasible solution) is not always helpful. In a general case you're probably better off not doing this.
> 
> /Anders 
> 
> 
> On 10 maj 2012, at 09:33, Anders Peterson wrote:
> 
>> You're right ojAlgo currently does not have a direct/smart way to detect infeasibility of the inequality constraints in a quadratic program. The LinearSolver should be able to do that for you, and if you get a feasible solution from the linear solver you can feed that as a starting point for the quadratic solver.
>> 
>> /Anders
>> 
>> 
>> On 9 maj 2012, at 18:00, Reda-Jürg MESSIKH wrote:
>> 
>>> H Anders, 
>>> 
>>> I found yet another odd behaviour. 
>>> 
>>> The problem defined with the data below (same notation as the former emails) has no solution (the inequalities are impossible to satisfy). 
>>> 
>>> 
>>> 
>>> Matlab detects this impossibility immediately (less than 3seconds) and exits with the right message. 
>>> 
>>> OjAlgo takes a really long time (more than 1 minute, I have interupted the execution afterwards). 
>>> 
>>> I susepct that there is no preliminary check whether the inequalities are not overly stringent and hence this behaviour. If you agree with this point, wouldn't it be a good thing to add such preliminary checks ? 
>>> 
>>> 
>>> Reda 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> From:        "Anders Peterson" <an...@op...> 
>>> 
>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>>> Date:        09.05.2012 10:11 
>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>>> 
>>> 
>>> 
>>> There is still the issue that ojAlgo and MATLAB gave different solutions. Which is "better"? What does MATLAB say about the ojAlgo solution, is it a feasible solution? What does ojAlgo say about the MATLAB solution...
>>> 
>>> 
>>> On 8 maj 2012, at 17:18, Reda-Jürg MESSIKH wrote:
>>> 
>>>> Hi Anders, thanks for the upload. 
>>>> 
>>>> As per FinanceUtils, I am not sure what I have done wrong before, now all is fine. 
>>>> 
>>>> Reda 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> From:        "Anders Peterson" <an...@op...> 
>>>> 
>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>>>> Date:        08.05.2012 16:42 
>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>>>> 
>>>> 
>>>> 
>>>> Just uploaded a new snapshot to sourceforge.
>>>> 
>>>> What problem do you have with FinanceUtils? I don't see any.
>>>> 
>>>> /Anders
>>>> 
>>>> 
>>>> On 8 maj 2012, at 16:13, Reda-Jürg MESSIKH wrote:
>>>> 
>>>>> Hi Anders, 
>>>>> 
>>>>> I just did a few tests that are conclusive, so would it be possible to do another build ? 
>>>>> 
>>>>> As a side remark, while building your modified source code, we have seen that the financeUtils does not build. 
>>>>> 
>>>>> Reda 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> From:        "Anders Peterson" <an...@op...> 
>>>>> 
>>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>>>>> Date:        08.05.2012 14:52 
>>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>>>>> 
>>>>> 
>>>>> 
>>>>> Now ojAlgo can get the solution in 6s...
>>>>> 
>>>>> I believe I just found a bug that must have been there for years. Get the latest code from CVS, and see if there are any (other) problems with your use cases.
>>>>> 
>>>>> /Anders
>>>>> 
>>>>> [attachment "ojalgo_solution.txt" deleted by Reda-Jürg MESSIKH/GVA/PCO] 
>>>>> 
>>>>> On 8 maj 2012, at 14:15, Reda-Jürg MESSIKH wrote:
>>>>> 
>>>>>> Hi Anders, 
>>>>>> 
>>>>>> Thanks for your answer. Indeed, 800s is still much longer than matlab. 
>>>>>> 
>>>>>> I will try to use other formulations of the problem and, if time permits, I will check the difference of handling between matlab and ojalgo. 
>>>>>> 
>>>>>> The  problem is of  Markowitz type, where Q is the covariance matrix and C (all zeros) are the expected returns of the assets. I have two risk linear constraints and an upper bound for the exposure to each asset. 
>>>>>> Is there an easie/ better formulation in ojAlgo of such problems ? 
>>>>>> 
>>>>>> Reda 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> From:        "Anders Peterson" <an...@op...> 
>>>>>> 
>>>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>>>>>> Date:        08.05.2012 12:24 
>>>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Don't have a good/quick solution to this problem...
>>>>>> 
>>>>>> I have managed to get a solution from ojAlgo that "it" claims to be optimal, but it's not the same solution as you got from Matlab and it took a very long time to get it (800s).
>>>>>> 
>>>>>> [attachment "ojalgo_solution.txt" deleted by Reda-Jürg MESSIKH/GVA/PCO] 
>>>>>> 
>>>>>> If you want to try the code modifications I made they're in CVS.
>>>>>> 
>>>>>> Is it possible for you to experiment with other problem formulations? Can you express the constraints in some alternate way? What are you optimizing - what does "Q" express?
>>>>>> 
>>>>>> BTW: If you have matrices on file / as strings and you're going to put the elements in an ExpressionsBasedModel it's better to create BigDecimals directly from the strings (don't use PrimitiveDenseStore as an intermediate type). If you need some sort of intermediate type use BigDenseStore or Array2D<BigDecimal>.
>>>>>> 
>>>>>> /Anders
>>>>>> 
>>>>>> 
>>>>>> On 7 maj 2012, at 19:14, Reda-Jürg MESSIKH wrote:
>>>>>> 
>>>>>>> Hi Anders, 
>>>>>>> 
>>>>>>> Here is the same stuff but with the matrix hardcoded into the code 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> The matlab solution: 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Thanks again, 
>>>>>>> Reda 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> From:        "Anders Peterson" <an...@op...> 
>>>>>>> 
>>>>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>>>>>>> Date:        07.05.2012 13:57 
>>>>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 7 maj 2012, at 13:19, Anders Peterson wrote:
>>>>>>> 
>>>>>>>> It would be easier to debug if there was only one problem, can you try to get rid of those NumberFormatException for me? 
>>>>>>> 
>>>>>>> I'm assuming this works for you so the problem is os/jvm specific. Instead trying to hunt down what goes wrong send me code where the matrices are hardcoded. After you've read the files, copy the matrices to PrimitiveMatrix instances and the call toString() on them. The output can be copy-pasted in to your code.
>>>>>>> 
>>>>>>> /Anders
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> ________________________________________________________________
>>>>>>> 
>>>>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>>>>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>>>>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
>>>>>>> ________________________________________________________________
>>>>>>> 
>>>>>>> This document should only be read by those persons to whom it is
>>>>>>> addressed  and  is  not intended to be relied upon by any person
>>>>>>> without  subsequent written confirmation of its contents. If you
>>>>>>> have  received  this  e-mail message in error, please destroy it
>>>>>>> and delete it from your computer.
>>>>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
>>>>>>> modification,  distribution  and/or  publication  of this E-mail
>>>>>>> message is strictly prohibited.
>>>>>>> ________________________________________________________________
>>>>>>> 
>>>>>>> Pictet reserves the right  to  monitor  and  record business and
>>>>>>> personal communications. By  responding to an email or call from
>>>>>>> a Pictet  employee  you  are  taken  to  have  consented to such
>>>>>>> monitoring and recording.
>>>>>>> ________________________________________________________________
>>>>>>> 
>>>>>>> 
>>>>>>> <MatrixReader.java><QuadraticDemo.java><matlab_solution.txt>
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> ________________________________________________________________
>>>>>> 
>>>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>>>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>>>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
>>>>>> ________________________________________________________________
>>>>>> 
>>>>>> This document should only be read by those persons to whom it is
>>>>>> addressed  and  is  not intended to be relied upon by any person
>>>>>> without  subsequent written confirmation of its contents. If you
>>>>>> have  received  this  e-mail message in error, please destroy it
>>>>>> and delete it from your computer.
>>>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
>>>>>> modification,  distribution  and/or  publication  of this E-mail
>>>>>> message is strictly prohibited.
>>>>>> ________________________________________________________________
>>>>>> 
>>>>>> Pictet reserves the right  to  monitor  and  record business and
>>>>>> personal communications. By  responding to an email or call from
>>>>>> a Pictet  employee  you  are  taken  to  have  consented to such
>>>>>> monitoring and recording.
>>>>>> ________________________________________________________________
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> ________________________________________________________________
>>>>> 
>>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
>>>>> ________________________________________________________________
>>>>> 
>>>>> This document should only be read by those persons to whom it is
>>>>> addressed  and  is  not intended to be relied upon by any person
>>>>> without  subsequent written confirmation of its contents. If you
>>>>> have  received  this  e-mail message in error, please destroy it
>>>>> and delete it from your computer.
>>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
>>>>> modification,  distribution  and/or  publication  of this E-mail
>>>>> message is strictly prohibited.
>>>>> ________________________________________________________________
>>>>> 
>>>>> Pictet reserves the right  to  monitor  and  record business and
>>>>> personal communications. By  responding to an email or call from
>>>>> a Pictet  employee  you  are  taken  to  have  consented to such
>>>>> monitoring and recording.
>>>>> ________________________________________________________________
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> ________________________________________________________________
>>>> 
>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
>>>> ________________________________________________________________
>>>> 
>>>> This document should only be read by those persons to whom it is
>>>> addressed  and  is  not intended to be relied upon by any person
>>>> without  subsequent written confirmation of its contents. If you
>>>> have  received  this  e-mail message in error, please destroy it
>>>> and delete it from your computer.
>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
>>>> modification,  distribution  and/or  publication  of this E-mail
>>>> message is strictly prohibited.
>>>> ________________________________________________________________
>>>> 
>>>> Pictet reserves the right  to  monitor  and  record business and
>>>> personal communications. By  responding to an email or call from
>>>> a Pictet  employee  you  are  taken  to  have  consented to such
>>>> monitoring and recording.
>>>> ________________________________________________________________
>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> 
>>> ________________________________________________________________
>>> 
>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>>> CH-1211 GENEVE 73                        http://www.pictet.com/
>>> ________________________________________________________________
>>> 
>>> This document should only be read by those persons to whom it is
>>> addressed  and  is  not intended to be relied upon by any person
>>> without  subsequent written confirmation of its contents. If you
>>> have  received  this  e-mail message in error, please destroy it
>>> and delete it from your computer.
>>> Any  form of  reproduction, dissemination, copying,  disclosure,
>>> modification,  distribution  and/or  publication  of this E-mail
>>> message is strictly prohibited.
>>> ________________________________________________________________
>>> 
>>> Pictet reserves the right  to  monitor  and  record business and
>>> personal communications. By  responding to an email or call from
>>> a Pictet  employee  you  are  taken  to  have  consented to such
>>> monitoring and recording.
>>> ________________________________________________________________
>>> 
>>> 
>>> <Q.txt><C.txt><BE.txt><AE.txt><AI.txt><BI.txt>
>> 
> 
> 
> 
> 
> ________________________________________________________________
> 
> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> CH-1211 GENEVE 73                        http://www.pictet.com/
> ________________________________________________________________
> 
> This document should only be read by those persons to whom it is
> addressed  and  is  not intended to be relied upon by any person
> without  subsequent written confirmation of its contents. If you
> have  received  this  e-mail message in error, please destroy it
> and delete it from your computer.
> Any  form of  reproduction, dissemination, copying,  disclosure,
> modification,  distribution  and/or  publication  of this E-mail
> message is strictly prohibited.
> ________________________________________________________________
> 
> Pictet reserves the right  to  monitor  and  record business and
> personal communications. By  responding to an email or call from
> a Pictet  employee  you  are  taken  to  have  consented to such
> monitoring and recording.
> ________________________________________________________________
> 
> 
> <Q.txt><C.txt><BI.txt><BE.txt><AI.txt><AE.txt>

Re: [ojAlgo-user] Removing Equalities and Inequalities methods

[Anders P. <an...@op...>]

If you've been building directly from source during the last couple of weeks you may have seen all sorts of errors. Now, since I released v32.4, it should be ok.

org.ojalgo.optimisation.OptimisationSolver => org.ojalgo.optimisation.Optimisation.Solver

but probably you should reference org.ojalgo.optimisation.GenericSolver instead. Your IDE should be able to help you get it right.

If you open a Type Hierarchy for org.ojalgo.optimisation.Optimisation you will see how everything is connected.

/Anders


On 22 maj 2012, at 17:58, Andy Berzins wrote:

> Anders, My apologies, I had a few mistakes. 
> 
> First, I didn't realize the Bold and Underlines show up as asterisks in posts (or don't show up at all) so you couldn't see the 3 errors I was referring to. 
> 
> The first error I had was a Method not Found, for the equalities method, such as in:
> 
> 
> .equalities(AE_ms,BE_ms), or in your reply:
>  
> 
> Second, it turns out the above error (plus the others) were entirely my fault. I had multiple Workspaces open in Eclipse and when I compiled the code the Ant Build went fine, 
> 
> created the JAR files, and reported a successful build.  I didn't see the compile errors listed in another (Markers) Window in Eclipse (i looked at the wrong workspace I am guessing).
> 
> I also didn't realize that the Ant Build can report a successful build in the Console window and create JAR files, but actually have compile errors for the code.
> 
> Learned something new for the day :-).
> 
> 
> So, I redid the download and compiled V32.5 successfully (based on the build.properties file) with no errors (but with some warnings about deprecated code, etc). 
> However, I still have one (new)issue, my code can no longer find the import (in v32.5):
> 
> 
> import org.ojalgo.optimisation.OptimisationSolver;
> 
> To 'check' my compile, I downloaded your compiled JAR file ojalgo-32.0.jar and my code COULD then find the above import and runs fine.  
> Has the OptimisationSolver class been permanently removed post v32.0? 
> 
> I searched my workspace for 'OptimisationSolver' and came up with 3 occurrences, 2 commented out and 1 in a comment.
> 
> 
> I will try the solution you mentioned below regardless, but could you confirm that OptimisationSolver has been deprecated?
> 
> 
> Thanks Much, Andy
> 
> 
>  
> 
> 
> 
> >You don't say what kind of errors you get. Does your program compile?
> >
> >Doing this:
> >
> >new QuadraticSolver.Builder(tmpQ, tmpC).equalities(tmpAE, tmpBE).inequalities(tmpAI, tmpBI).build().solve();
> >
> >has not been deprecated, and currently there are no plans to do that. I have several unit tests that build the model precisely that way, and as far as I can see they all still work.>
> >
> >/Anders
> 
> 
> 
> <snip>
> 
> just left a small snippet of the original post
> can't find the OptimisationSolver anymore referenced below....
> 
> 
>                 QuadraticSolver.Builder tmpBuilder = new QuadraticSolver.Builder(Q_ms, C_ms).equalities(AE_ms, BE_ms).inequalities(AI_ms, BI_ms);
>                 QuadraticSolver tmpSolver = tmpBuilder.build();
>                 tmpSolver.options.timeLimit = Long.MAX_VALUE; 
>                 OptimisationSolver.Result tmpResult = tmpSolver.solve();
>                 BasicMatrix tmpSolution = tmpResult.getSolution();
>                 PhysicalStore<Double> result = tmpSolution.toPrimitiveStore();
> 
> 
> <snip>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user

Re: [ojAlgo-user] Removing Equalities and Inequalities methods

[Andy B. <ber...@gm...>]

Anders, My apologies, I had a few mistakes.

First, I didn't realize the Bold and Underlines show up as asterisks
in posts (or don't show up at all) so you couldn't see the 3 errors I
was referring to.
The first error I had was a Method not Found, for the equalities
method, such as in:

.equalities(AE_ms,BE_ms), or in your reply:


Second, it turns out the above error (plus the others) were entirely
my fault. I had multiple Workspaces open in Eclipse and when I
compiled the code the Ant Build went fine,
created the JAR files, and reported a successful build.  I didn't see
the compile errors listed in another (Markers) Window in Eclipse (i
looked at the wrong workspace I am guessing).
I also didn't realize that the Ant Build can report a successful build
in the Console window and create JAR files, but actually have compile
errors for the code.
Learned something new for the day :-).

So, I redid the download and compiled V32.5 successfully (based on the
build.properties file) with no errors (but with some warnings about
deprecated code, etc).
However, I still have one (new)issue, my code can no longer find the
import (in v32.5):

import org.ojalgo.optimisation.OptimisationSolver;

To 'check' my compile, I downloaded your compiled JAR file
ojalgo-32.0.jar and my code COULD then find the above import and runs
fine.
Has the OptimisationSolver class been permanently removed post v32.0?
I searched my workspace for 'OptimisationSolver' and came up with 3
occurrences, 2 commented out and 1 in a comment.

I will try the solution you mentioned below regardless, but could you
confirm that OptimisationSolver has been deprecated?

Thanks Much, Andy







>You don't say what kind of errors you get. Does your program compile?
>
>Doing this:
>
>new QuadraticSolver.Builder(tmpQ, tmpC).equalities(tmpAE, tmpBE).inequalities(tmpAI, tmpBI).build().solve();
>
>has not been deprecated, and currently there are no plans to do that. I have several unit tests that build the model precisely that way, and as far as I can see they all still work.>
>
>/Anders

<snip>

just left a small snippet of the original post
can't find the OptimisationSolver anymore referenced below....



                QuadraticSolver.Builder tmpBuilder = new
QuadraticSolver.Builder(Q_ms, C_ms).*equalities*(AE_ms,
BE_ms).inequalities(AI_ms, BI_ms);
                QuadraticSolver tmpSolver = tmpBuilder.build();
                tmpSolver.*options.*timeLimit = Long.MAX_VALUE;
                OptimisationSolver.Result tmpResult = tmpSolver.*solve*();
                BasicMatrix tmpSolution = tmpResult.getSolution();
                PhysicalStore<Double> result =
tmpSolution.toPrimitiveStore();


<snip>

[ojAlgo-user] v32.4

[Anders P. <an...@op...>]

Just uploaded a new snapshot. It has further "improvements" to the optimisation package. I'm not done yet, but I believe the majority of the api changes are done.

If you have code that currently uses v31/v32 (or older) you will have to modify your code. I'm interested in feedback, and you're welcome to ask questions.

If you are explicit regarding what you want in terms of adding (or restoring) functionality I will consider it.

/Anders

Re: [ojAlgo-user] Removing Equalities and Inequalities methods

[Anders P. <an...@op...>]

You don't say what kind of errors you get. Does your program compile?

Doing this:

new QuadraticSolver.Builder(tmpQ, tmpC).equalities(tmpAE, tmpBE).inequalities(tmpAI, tmpBI).build().solve();

has not been deprecated, and currently there are no plans to do that. I have several unit tests that build the model precisely that way, and as far as I can see they all still work.

/Anders


On 18 maj 2012, at 19:55, Andy Berzins wrote:

> Hi Anders,
>  
> Just an FYI upfront, I am not much of a Matrice/Solver 'coder'. I know how to define the problems well, but like to use simple (to the user) code implementations. That is why I chose your package, as it was really clear to me to do something like:
> 
> QuadraticSolver.Builder tmpBuilder = new QuadraticSolver.Builder(Q_ms, C_ms).equalities(AE_ms, BE_ms).inequalities(AI_ms, BI_ms);
> 
> I couldn't figure out from the JavaDocs how to setup the  Expression Based model, (months ago I spent a full day+ on trying to figure that out). Anyways, to my chagrin, I did an update yesterday when I read that you found an old bug and that the (QP?) Solver was now sped up by a factor of 100x. I compiled the update this morning and found out that the update broke my code. Any chance you could bring back the equalities and inequalities methods? From my limited knowledge, those seem to be very basic constraints that almost everybody would use.
> 
> Or perhaps somebody could help me out on a code snippet to fix my "direct access to the solver" code (it is for a simple efficient frontier calc)... I really did spend a lot of time trying to get the expressions based model setup.
> 
> Below is what I had, which no longer works. The 3 bolded and underlined methods under '-------- Solve the Problem --------" are what breaks...
> 
>             /** 
>              * We are solving the problem:
>              *        min 1/2 [X]'[Q][X] - [C]'[X]
>              *            where     [AE][X] == [BE]
>              *                    [AI][X] <= [BI]
>              */
> 
>             // required matrices and arrays
>             MatrixStore Q_ms = null;
>             MatrixStore C_ms = null;
>             MatrixStore AE_ms = null; 
>             MatrixStore BE_ms = null; 
>             MatrixStore AI_ms = null; 
>             MatrixStore BI_ms = null;
>             double[][] c_temp = null;
>             double[][] ae = null;
>             double[][] be = null;
>             double[][] ai = null;
>             double[][] bi = null;
> 
>             .............. I did not include Populating the matrices code, various different constraints depending if Fixed Returns are passed in or not, etc..
> 
>             // create our actual matrix stores to be used in the calcs given problem setup
>             Q_ms = PrimitiveDenseStore.FACTORY.rows(arrCovar);
>             C_ms = PrimitiveDenseStore.FACTORY.rows(c_temp);
>             AE_ms = PrimitiveDenseStore.FACTORY.rows(ae);
>             BE_ms = PrimitiveDenseStore.FACTORY.rows(be); 
>             AI_ms = PrimitiveDenseStore.FACTORY.rows(ai);
>             BI_ms = PrimitiveDenseStore.FACTORY.rows(bi);
> 
> 
>             /** ------------------------------- Solve the Problem --------------------------------------*/
>             try {
>                 QuadraticSolver.Builder tmpBuilder = new QuadraticSolver.Builder(Q_ms, C_ms).equalities(AE_ms, BE_ms).inequalities(AI_ms, BI_ms);
>                 QuadraticSolver tmpSolver = tmpBuilder.build();
>                 tmpSolver.options.timeLimit = Long.MAX_VALUE; 
>                 OptimisationSolver.Result tmpResult = tmpSolver.solve();
>                 BasicMatrix tmpSolution = tmpResult.getSolution();
>                 PhysicalStore<Double> result = tmpSolution.toPrimitiveStore();
> 
>                 arrVariance[0] = result.transpose().multiplyRight(Q_ms).multiplyRight(result).get(0, 0);
> 
>                ...... <snip>
> 
> 
> Finally, if the bug fix you did for speeding up the calcs don't affect my above methodology I could just stay on my last compiled library. However, if the fix did affect the QP solver, is there a version that has the bug fix in it, but has not yet removed the deprecated methods? As a last resort, that version would be an acceptable fix for me (though it would be nice to understand the Expression Model ultimately) ...
>  
> Thanks to everybody in  advance, 
> 
> Andy or Andis (depending on which side of the Ocean you are on :-) )
> 
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Sun, 13 May 2012 17:31:46 +0200
> >From: Anders Peterson <an...@op...>
> Subject: Re: [ojAlgo-user] Future changes to the optimisation package
> To: oja...@li...
> Message-ID: <2D5...@op...>
> Content-Type: text/plain; charset=iso-8859-1
> 
> I deprecated (and removed) all those variants of addEpression(?) because I realized there's an infinite  number of methods that could be useful, and instead of adding more variants I removed all but the most generic ones. I still think that was a good decision.
> 
> I've added a little bit to the on-line javadoc for Expressuon and ExpressionsBasedModel.
> 
> 
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user

Re: [ojAlgo-user] Removing Equalities and Inequalities methods

[Andy B. <ber...@gm...>]

Hi Anders,

Just an FYI upfront, I am not much of a Matrice/Solver 'coder'. I know how
to define the problems well, but like to use simple (to the user) code
implementations. That is why I chose your package, as it was really clear
to me to do something like:

QuadraticSolver.Builder tmpBuilder = new QuadraticSolver.Builder(Q_ms,
C_ms).equalities(AE_ms, BE_ms).inequalities(AI_ms, BI_ms);

I couldn't figure out from the JavaDocs how to setup the  Expression Based
model, (months ago I spent a full day+ on trying to figure that out).
Anyways, to my chagrin, I did an update yesterday when I read that you
found an old bug and that the (QP?) Solver was now sped up by a factor of
100x. I compiled the update this morning and found out that the update
broke my code. Any chance you could bring back the equalities and
inequalities methods? From my limited knowledge, those seem to be very
basic constraints that almost everybody would use.

Or perhaps somebody could help me out on a code snippet to fix my "direct
access to the solver" code (it is for a simple efficient frontier calc)...
I really did spend a lot of time trying to get the expressions based model
setup.

Below is what I had, which no longer works. The 3 bolded and underlined
methods under '-------- Solve the Problem --------" are what breaks...

            /**
             * We are solving the problem:
             *        min 1/2 [X]'[Q][X] - [C]'[X]
             *            where     [AE][X] == [BE]
             *                    [AI][X] <= [BI]
             */

            // required matrices and arrays
            MatrixStore Q_ms = null;
            MatrixStore C_ms = null;
            MatrixStore AE_ms = null;
            MatrixStore BE_ms = null;
            MatrixStore AI_ms = null;
            MatrixStore BI_ms = null;
            double[][] c_temp = null;
            double[][] ae = null;
            double[][] be = null;
            double[][] ai = null;
            double[][] bi = null;

            .............. I did not include Populating the matrices code,
various different constraints depending if Fixed Returns are passed in or
not, etc..

            // create our actual matrix stores to be used in the calcs
given problem setup
            Q_ms = PrimitiveDenseStore.FACTORY.rows(arrCovar);
            C_ms = PrimitiveDenseStore.FACTORY.rows(c_temp);
            AE_ms = PrimitiveDenseStore.FACTORY.rows(ae);
            BE_ms = PrimitiveDenseStore.FACTORY.rows(be);
            AI_ms = PrimitiveDenseStore.FACTORY.rows(ai);
            BI_ms = PrimitiveDenseStore.FACTORY.rows(bi);


            /** ------------------------------- Solve the Problem
--------------------------------------*/
            try {
                QuadraticSolver.Builder tmpBuilder = new
QuadraticSolver.Builder(Q_ms, C_ms).*equalities*(AE_ms,
BE_ms).inequalities(AI_ms, BI_ms);
                QuadraticSolver tmpSolver = tmpBuilder.build();
                tmpSolver.*options.*timeLimit = Long.MAX_VALUE;
                OptimisationSolver.Result tmpResult = tmpSolver.*solve*();
                BasicMatrix tmpSolution = tmpResult.getSolution();
                PhysicalStore<Double> result =
tmpSolution.toPrimitiveStore();

                arrVariance[0] =
result.transpose().multiplyRight(Q_ms).multiplyRight(result).get(0, 0);

               ...... <snip>


Finally, if the bug fix you did for speeding up the calcs don't affect my
above methodology I could just stay on my last compiled library. However,
if the fix did affect the QP solver, is there a version that has the bug
fix in it, but has not yet removed the deprecated methods? As a last
resort, that version would be an acceptable fix for me (though it would be
nice to understand the Expression Model ultimately) ...

Thanks to everybody in  advance,

Andy or Andis (depending on which side of the Ocean you are on :-) )



> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 13 May 2012 17:31:46 +0200
> From: Anders Peterson <an...@op...>
> Subject: Re: [ojAlgo-user] Future changes to the optimisation package
> To: oja...@li...
> Message-ID: <2D5...@op...>
> Content-Type: text/plain; charset=iso-8859-1
>
> I deprecated (and removed) all those variants of addEpression(?) because I
> realized there's an infinite  number of methods that could be useful, and
> instead of adding more variants I removed all but the most generic ones. I
> still think that was a good decision.
>
> I've added a little bit to the on-line javadoc for Expressuon and
> ExpressionsBasedModel.
>
>

Re: [ojAlgo-user] ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution

[Anders P. <an...@op...>]

On 14 maj 2012, at 14:58, Reda-Jürg MESSIKH wrote:

> On the same example than last time: 
> 
> v32.2 gives me null for QuadraticSolver.Result. 

That was probably me that screwed up the build. There is a new v32.3 that I hope works.


> For LinearSolver, I have another problem. Present in v32.1 and v32.2:  The code 
> 
>     LinearSolver.Builder builder = new LinearSolver.Builder(Ct).equalities(AE, BE).inequalities(AI, BI);     
>         
>     LinearSolver solver = builder.build();     
>         
>     LinearSolver.Result result=solver.solve(); 
>         
>     System.out.println(result.getState()); 
>     
>     System.out.println(result.getSolution());     
> 
> It should tell me that there is no solution but gives me the wrong answer 

Not sure which case you're referring to. If you think you have found a problem please send me code that show the error. Make it as compact as possible. One static method with as little code as possible and everything hardcoded works very well for me - make matrices/datasets as small as possible. I'll probably turn what you send me into a junit test.


/Anders



> 
> OPTIMAL 
> class org.ojalgo.matrix.PrimitiveMatrix 
> {{1.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}, 
> {0.0}} 
> 
> 
> 
> 
> 
> 
> Reda Jürg MESSIKH
> SRI & Active Quants - GVA
> tel. +41 58 323 3184 
> 
> 
> 
> From:        "Anders Peterson" <an...@op...> 
>                 
> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> Date:        14.05.2012 09:45 
> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> 
> 
> 
> There was a problem with the LInearSolver and constraints with negative valued upper limits. I've released v32.2 with that fixed.
> 
> Also; feeding the QuadraticSolver an initial iteration point (feasible solution) is not always helpful. In a general case you're probably better off not doing this.
> 
> /Anders 
> 
> 
> On 10 maj 2012, at 09:33, Anders Peterson wrote:
> 
> > You're right ojAlgo currently does not have a direct/smart way to detect infeasibility of the inequality constraints in a quadratic program. The LinearSolver should be able to do that for you, and if you get a feasible solution from the linear solver you can feed that as a starting point for the quadratic solver.
> > 
> > /Anders
> > 
> > 
> > On 9 maj 2012, at 18:00, Reda-Jürg MESSIKH wrote:
> > 
> >> H Anders, 
> >> 
> >> I found yet another odd behaviour. 
> >> 
> >> The problem defined with the data below (same notation as the former emails) has no solution (the inequalities are impossible to satisfy). 
> >> 
> >> 
> >> 
> >> Matlab detects this impossibility immediately (less than 3seconds) and exits with the right message. 
> >> 
> >> OjAlgo takes a really long time (more than 1 minute, I have interupted the execution afterwards). 
> >> 
> >> I susepct that there is no preliminary check whether the inequalities are not overly stringent and hence this behaviour. If you agree with this point, wouldn't it be a good thing to add such preliminary checks ? 
> >> 
> >> 
> >> Reda 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> 
> >> From:        "Anders Peterson" <an...@op...> 
> >> 
> >> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> >> Date:        09.05.2012 10:11 
> >> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> >> 
> >> 
> >> 
> >> There is still the issue that ojAlgo and MATLAB gave different solutions. Which is "better"? What does MATLAB say about the ojAlgo solution, is it a feasible solution? What does ojAlgo say about the MATLAB solution...
> >> 
> >> 
> >> On 8 maj 2012, at 17:18, Reda-Jürg MESSIKH wrote:
> >> 
> >>> Hi Anders, thanks for the upload. 
> >>> 
> >>> As per FinanceUtils, I am not sure what I have done wrong before, now all is fine. 
> >>> 
> >>> Reda 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> From:        "Anders Peterson" <an...@op...> 
> >>> 
> >>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> >>> Date:        08.05.2012 16:42 
> >>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> >>> 
> >>> 
> >>> 
> >>> Just uploaded a new snapshot to sourceforge.
> >>> 
> >>> What problem do you have with FinanceUtils? I don't see any.
> >>> 
> >>> /Anders
> >>> 
> >>> 
> >>> On 8 maj 2012, at 16:13, Reda-Jürg MESSIKH wrote:
> >>> 
> >>>> Hi Anders, 
> >>>> 
> >>>> I just did a few tests that are conclusive, so would it be possible to do another build ? 
> >>>> 
> >>>> As a side remark, while building your modified source code, we have seen that the financeUtils does not build. 
> >>>> 
> >>>> Reda 
> >>>> 
> >>>> 
> >>>> 
> >>>> 
> >>>> 
> >>>> From:        "Anders Peterson" <an...@op...> 
> >>>> 
> >>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> >>>> Date:        08.05.2012 14:52 
> >>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> >>>> 
> >>>> 
> >>>> 
> >>>> Now ojAlgo can get the solution in 6s...
> >>>> 
> >>>> I believe I just found a bug that must have been there for years. Get the latest code from CVS, and see if there are any (other) problems with your use cases.
> >>>> 
> >>>> /Anders
> >>>> 
> >>>> [attachment "ojalgo_solution.txt" deleted by Reda-Jürg MESSIKH/GVA/PCO] 
> >>>> 
> >>>> On 8 maj 2012, at 14:15, Reda-Jürg MESSIKH wrote:
> >>>> 
> >>>>> Hi Anders, 
> >>>>> 
> >>>>> Thanks for your answer. Indeed, 800s is still much longer than matlab. 
> >>>>> 
> >>>>> I will try to use other formulations of the problem and, if time permits, I will check the difference of handling between matlab and ojalgo. 
> >>>>> 
> >>>>> The  problem is of  Markowitz type, where Q is the covariance matrix and C (all zeros) are the expected returns of the assets. I have two risk linear constraints and an upper bound for the exposure to each asset. 
> >>>>> Is there an easie/ better formulation in ojAlgo of such problems ? 
> >>>>> 
> >>>>> Reda 
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> From:        "Anders Peterson" <an...@op...> 
> >>>>> 
> >>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> >>>>> Date:        08.05.2012 12:24 
> >>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> Don't have a good/quick solution to this problem...
> >>>>> 
> >>>>> I have managed to get a solution from ojAlgo that "it" claims to be optimal, but it's not the same solution as you got from Matlab and it took a very long time to get it (800s).
> >>>>> 
> >>>>> [attachment "ojalgo_solution.txt" deleted by Reda-Jürg MESSIKH/GVA/PCO] 
> >>>>> 
> >>>>> If you want to try the code modifications I made they're in CVS.
> >>>>> 
> >>>>> Is it possible for you to experiment with other problem formulations? Can you express the constraints in some alternate way? What are you optimizing - what does "Q" express?
> >>>>> 
> >>>>> BTW: If you have matrices on file / as strings and you're going to put the elements in an ExpressionsBasedModel it's better to create BigDecimals directly from the strings (don't use PrimitiveDenseStore as an intermediate type). If you need some sort of intermediate type use BigDenseStore or Array2D<BigDecimal>.
> >>>>> 
> >>>>> /Anders
> >>>>> 
> >>>>> 
> >>>>> On 7 maj 2012, at 19:14, Reda-Jürg MESSIKH wrote:
> >>>>> 
> >>>>>> Hi Anders, 
> >>>>>> 
> >>>>>> Here is the same stuff but with the matrix hardcoded into the code 
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> The matlab solution: 
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> Thanks again, 
> >>>>>> Reda 
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> From:        "Anders Peterson" <an...@op...> 
> >>>>>> 
> >>>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
> >>>>>> Date:        07.05.2012 13:57 
> >>>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> On 7 maj 2012, at 13:19, Anders Peterson wrote:
> >>>>>> 
> >>>>>>> It would be easier to debug if there was only one problem, can you try to get rid of those NumberFormatException for me? 
> >>>>>> 
> >>>>>> I'm assuming this works for you so the problem is os/jvm specific. Instead trying to hunt down what goes wrong send me code where the matrices are hardcoded. After you've read the files, copy the matrices to PrimitiveMatrix instances and the call toString() on them. The output can be copy-pasted in to your code.
> >>>>>> 
> >>>>>> /Anders
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> ________________________________________________________________
> >>>>>> 
> >>>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> >>>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> >>>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
> >>>>>> ________________________________________________________________
> >>>>>> 
> >>>>>> This document should only be read by those persons to whom it is
> >>>>>> addressed  and  is  not intended to be relied upon by any person
> >>>>>> without  subsequent written confirmation of its contents. If you
> >>>>>> have  received  this  e-mail message in error, please destroy it
> >>>>>> and delete it from your computer.
> >>>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
> >>>>>> modification,  distribution  and/or  publication  of this E-mail
> >>>>>> message is strictly prohibited.
> >>>>>> ________________________________________________________________
> >>>>>> 
> >>>>>> Pictet reserves the right  to  monitor  and  record business and
> >>>>>> personal communications. By  responding to an email or call from
> >>>>>> a Pictet  employee  you  are  taken  to  have  consented to such
> >>>>>> monitoring and recording.
> >>>>>> ________________________________________________________________
> >>>>>> 
> >>>>>> 
> >>>>>> <MatrixReader.java><QuadraticDemo.java><matlab_solution.txt>
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> ________________________________________________________________
> >>>>> 
> >>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> >>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> >>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
> >>>>> ________________________________________________________________
> >>>>> 
> >>>>> This document should only be read by those persons to whom it is
> >>>>> addressed  and  is  not intended to be relied upon by any person
> >>>>> without  subsequent written confirmation of its contents. If you
> >>>>> have  received  this  e-mail message in error, please destroy it
> >>>>> and delete it from your computer.
> >>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
> >>>>> modification,  distribution  and/or  publication  of this E-mail
> >>>>> message is strictly prohibited.
> >>>>> ________________________________________________________________
> >>>>> 
> >>>>> Pictet reserves the right  to  monitor  and  record business and
> >>>>> personal communications. By  responding to an email or call from
> >>>>> a Pictet  employee  you  are  taken  to  have  consented to such
> >>>>> monitoring and recording.
> >>>>> ________________________________________________________________
> >>>>> 
> >>>>> 
> >>>> 
> >>>> 
> >>>> 
> >>>> ________________________________________________________________
> >>>> 
> >>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> >>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> >>>> CH-1211 GENEVE 73                        http://www.pictet.com/
> >>>> ________________________________________________________________
> >>>> 
> >>>> This document should only be read by those persons to whom it is
> >>>> addressed  and  is  not intended to be relied upon by any person
> >>>> without  subsequent written confirmation of its contents. If you
> >>>> have  received  this  e-mail message in error, please destroy it
> >>>> and delete it from your computer.
> >>>> Any  form of  reproduction, dissemination, copying,  disclosure,
> >>>> modification,  distribution  and/or  publication  of this E-mail
> >>>> message is strictly prohibited.
> >>>> ________________________________________________________________
> >>>> 
> >>>> Pictet reserves the right  to  monitor  and  record business and
> >>>> personal communications. By  responding to an email or call from
> >>>> a Pictet  employee  you  are  taken  to  have  consented to such
> >>>> monitoring and recording.
> >>>> ________________________________________________________________
> >>>> 
> >>>> 
> >>> 
> >>> 
> >>> 
> >>> 
> >>> ________________________________________________________________
> >>> 
> >>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> >>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> >>> CH-1211 GENEVE 73                        http://www.pictet.com/
> >>> ________________________________________________________________
> >>> 
> >>> This document should only be read by those persons to whom it is
> >>> addressed  and  is  not intended to be relied upon by any person
> >>> without  subsequent written confirmation of its contents. If you
> >>> have  received  this  e-mail message in error, please destroy it
> >>> and delete it from your computer.
> >>> Any  form of  reproduction, dissemination, copying,  disclosure,
> >>> modification,  distribution  and/or  publication  of this E-mail
> >>> message is strictly prohibited.
> >>> ________________________________________________________________
> >>> 
> >>> Pictet reserves the right  to  monitor  and  record business and
> >>> personal communications. By  responding to an email or call from
> >>> a Pictet  employee  you  are  taken  to  have  consented to such
> >>> monitoring and recording.
> >>> ________________________________________________________________
> >>> 
> >>> 
> >> 
> >> 
> >> 
> >> 
> >> ________________________________________________________________
> >> 
> >> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> >> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> >> CH-1211 GENEVE 73                        http://www.pictet.com/
> >> ________________________________________________________________
> >> 
> >> This document should only be read by those persons to whom it is
> >> addressed  and  is  not intended to be relied upon by any person
> >> without  subsequent written confirmation of its contents. If you
> >> have  received  this  e-mail message in error, please destroy it
> >> and delete it from your computer.
> >> Any  form of  reproduction, dissemination, copying,  disclosure,
> >> modification,  distribution  and/or  publication  of this E-mail
> >> message is strictly prohibited.
> >> ________________________________________________________________
> >> 
> >> Pictet reserves the right  to  monitor  and  record business and
> >> personal communications. By  responding to an email or call from
> >> a Pictet  employee  you  are  taken  to  have  consented to such
> >> monitoring and recording.
> >> ________________________________________________________________
> >> 
> >> 
> >> <Q.txt><C.txt><BE.txt><AE.txt><AI.txt><BI.txt>
> > 
> 
> 
> 
> 
> ________________________________________________________________
> 
> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
> 60, route des Acacias                    Fax  +41 (0)58 323 2324
> CH-1211 GENEVE 73                        http://www.pictet.com/
> ________________________________________________________________
> 
> This document should only be read by those persons to whom it is
> addressed  and  is  not intended to be relied upon by any person
> without  subsequent written confirmation of its contents. If you
> have  received  this  e-mail message in error, please destroy it
> and delete it from your computer.
> Any  form of  reproduction, dissemination, copying,  disclosure,
> modification,  distribution  and/or  publication  of this E-mail
> message is strictly prohibited.
> ________________________________________________________________
> 
> Pictet reserves the right  to  monitor  and  record business and
> personal communications. By  responding to an email or call from
> a Pictet  employee  you  are  taken  to  have  consented to such
> monitoring and recording.
> ________________________________________________________________
> 
> 
> <Q.txt><AE.txt><AI.txt><BE.txt><BI.txt><C.txt>

Re: [ojAlgo-user] ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution

[Anders P. <an...@op...>]

There was a problem with the LInearSolver and constraints with negative valued upper limits. I've released v32.2 with that fixed.

Also; feeding the QuadraticSolver an initial iteration point (feasible solution) is not always helpful. In a general case you're probably better off not doing this.

/Anders 


On 10 maj 2012, at 09:33, Anders Peterson wrote:

> You're right ojAlgo currently does not have a direct/smart way to detect infeasibility of the inequality constraints in a quadratic program. The LinearSolver should be able to do that for you, and if you get a feasible solution from the linear solver you can feed that as a starting point for the quadratic solver.
> 
> /Anders
> 
> 
> On 9 maj 2012, at 18:00, Reda-Jürg MESSIKH wrote:
> 
>> H Anders, 
>> 
>> I found yet another odd behaviour. 
>> 
>> The problem defined with the data below (same notation as the former emails) has no solution (the inequalities are impossible to satisfy). 
>> 
>> 
>> 
>> Matlab detects this impossibility immediately (less than 3seconds) and exits with the right message. 
>> 
>> OjAlgo takes a really long time (more than 1 minute, I have interupted the execution afterwards). 
>> 
>> I susepct that there is no preliminary check whether the inequalities are not overly stringent and hence this behaviour. If you agree with this point, wouldn't it be a good thing to add such preliminary checks ? 
>> 
>> 
>> Reda 
>> 
>> 
>> 
>> 
>> 
>> 
>> From:        "Anders Peterson" <an...@op...> 
>> 
>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>> Date:        09.05.2012 10:11 
>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>> 
>> 
>> 
>> There is still the issue that ojAlgo and MATLAB gave different solutions. Which is "better"? What does MATLAB say about the ojAlgo solution, is it a feasible solution? What does ojAlgo say about the MATLAB solution...
>> 
>> 
>> On 8 maj 2012, at 17:18, Reda-Jürg MESSIKH wrote:
>> 
>>> Hi Anders, thanks for the upload. 
>>> 
>>> As per FinanceUtils, I am not sure what I have done wrong before, now all is fine. 
>>> 
>>> Reda 
>>> 
>>> 
>>> 
>>> 
>>> From:        "Anders Peterson" <an...@op...> 
>>> 
>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>>> Date:        08.05.2012 16:42 
>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>>> 
>>> 
>>> 
>>> Just uploaded a new snapshot to sourceforge.
>>> 
>>> What problem do you have with FinanceUtils? I don't see any.
>>> 
>>> /Anders
>>> 
>>> 
>>> On 8 maj 2012, at 16:13, Reda-Jürg MESSIKH wrote:
>>> 
>>>> Hi Anders, 
>>>> 
>>>> I just did a few tests that are conclusive, so would it be possible to do another build ? 
>>>> 
>>>> As a side remark, while building your modified source code, we have seen that the financeUtils does not build. 
>>>> 
>>>> Reda 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> From:        "Anders Peterson" <an...@op...> 
>>>> 
>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>>>> Date:        08.05.2012 14:52 
>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>>>> 
>>>> 
>>>> 
>>>> Now ojAlgo can get the solution in 6s...
>>>> 
>>>> I believe I just found a bug that must have been there for years. Get the latest code from CVS, and see if there are any (other) problems with your use cases.
>>>> 
>>>> /Anders
>>>> 
>>>> [attachment "ojalgo_solution.txt" deleted by Reda-Jürg MESSIKH/GVA/PCO] 
>>>> 
>>>> On 8 maj 2012, at 14:15, Reda-Jürg MESSIKH wrote:
>>>> 
>>>>> Hi Anders, 
>>>>> 
>>>>> Thanks for your answer. Indeed, 800s is still much longer than matlab. 
>>>>> 
>>>>> I will try to use other formulations of the problem and, if time permits, I will check the difference of handling between matlab and ojalgo. 
>>>>> 
>>>>> The  problem is of  Markowitz type, where Q is the covariance matrix and C (all zeros) are the expected returns of the assets. I have two risk linear constraints and an upper bound for the exposure to each asset. 
>>>>> Is there an easie/ better formulation in ojAlgo of such problems ? 
>>>>> 
>>>>> Reda 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> From:        "Anders Peterson" <an...@op...> 
>>>>> 
>>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>>>>> Date:        08.05.2012 12:24 
>>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>>>>> 
>>>>> 
>>>>> 
>>>>> Don't have a good/quick solution to this problem...
>>>>> 
>>>>> I have managed to get a solution from ojAlgo that "it" claims to be optimal, but it's not the same solution as you got from Matlab and it took a very long time to get it (800s).
>>>>> 
>>>>> [attachment "ojalgo_solution.txt" deleted by Reda-Jürg MESSIKH/GVA/PCO] 
>>>>> 
>>>>> If you want to try the code modifications I made they're in CVS.
>>>>> 
>>>>> Is it possible for you to experiment with other problem formulations? Can you express the constraints in some alternate way? What are you optimizing - what does "Q" express?
>>>>> 
>>>>> BTW: If you have matrices on file / as strings and you're going to put the elements in an ExpressionsBasedModel it's better to create BigDecimals directly from the strings (don't use PrimitiveDenseStore as an intermediate type). If you need some sort of intermediate type use BigDenseStore or Array2D<BigDecimal>.
>>>>> 
>>>>> /Anders
>>>>> 
>>>>> 
>>>>> On 7 maj 2012, at 19:14, Reda-Jürg MESSIKH wrote:
>>>>> 
>>>>>> Hi Anders, 
>>>>>> 
>>>>>> Here is the same stuff but with the matrix hardcoded into the code 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> The matlab solution: 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Thanks again, 
>>>>>> Reda 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> From:        "Anders Peterson" <an...@op...> 
>>>>>> 
>>>>>> To:        Reda-Jürg MESSIKH <rme...@pi...> 
>>>>>> Date:        07.05.2012 13:57 
>>>>>> Subject:        Re: ojalgo v3.2: quadratic optimization, trouble when running an optimization with "tight" upper bounds for the solution 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 7 maj 2012, at 13:19, Anders Peterson wrote:
>>>>>> 
>>>>>>> It would be easier to debug if there was only one problem, can you try to get rid of those NumberFormatException for me? 
>>>>>> 
>>>>>> I'm assuming this works for you so the problem is os/jvm specific. Instead trying to hunt down what goes wrong send me code where the matrices are hardcoded. After you've read the files, copy the matrices to PrimitiveMatrix instances and the call toString() on them. The output can be copy-pasted in to your code.
>>>>>> 
>>>>>> /Anders
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> ________________________________________________________________
>>>>>> 
>>>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>>>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>>>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
>>>>>> ________________________________________________________________
>>>>>> 
>>>>>> This document should only be read by those persons to whom it is
>>>>>> addressed  and  is  not intended to be relied upon by any person
>>>>>> without  subsequent written confirmation of its contents. If you
>>>>>> have  received  this  e-mail message in error, please destroy it
>>>>>> and delete it from your computer.
>>>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
>>>>>> modification,  distribution  and/or  publication  of this E-mail
>>>>>> message is strictly prohibited.
>>>>>> ________________________________________________________________
>>>>>> 
>>>>>> Pictet reserves the right  to  monitor  and  record business and
>>>>>> personal communications. By  responding to an email or call from
>>>>>> a Pictet  employee  you  are  taken  to  have  consented to such
>>>>>> monitoring and recording.
>>>>>> ________________________________________________________________
>>>>>> 
>>>>>> 
>>>>>> <MatrixReader.java><QuadraticDemo.java><matlab_solution.txt>
>>>>> 
>>>>> 
>>>>> 
>>>>> ________________________________________________________________
>>>>> 
>>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
>>>>> ________________________________________________________________
>>>>> 
>>>>> This document should only be read by those persons to whom it is
>>>>> addressed  and  is  not intended to be relied upon by any person
>>>>> without  subsequent written confirmation of its contents. If you
>>>>> have  received  this  e-mail message in error, please destroy it
>>>>> and delete it from your computer.
>>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
>>>>> modification,  distribution  and/or  publication  of this E-mail
>>>>> message is strictly prohibited.
>>>>> ________________________________________________________________
>>>>> 
>>>>> Pictet reserves the right  to  monitor  and  record business and
>>>>> personal communications. By  responding to an email or call from
>>>>> a Pictet  employee  you  are  taken  to  have  consented to such
>>>>> monitoring and recording.
>>>>> ________________________________________________________________
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> ________________________________________________________________
>>>> 
>>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>>>> CH-1211 GENEVE 73                        http://www.pictet.com/
>>>> ________________________________________________________________
>>>> 
>>>> This document should only be read by those persons to whom it is
>>>> addressed  and  is  not intended to be relied upon by any person
>>>> without  subsequent written confirmation of its contents. If you
>>>> have  received  this  e-mail message in error, please destroy it
>>>> and delete it from your computer.
>>>> Any  form of  reproduction, dissemination, copying,  disclosure,
>>>> modification,  distribution  and/or  publication  of this E-mail
>>>> message is strictly prohibited.
>>>> ________________________________________________________________
>>>> 
>>>> Pictet reserves the right  to  monitor  and  record business and
>>>> personal communications. By  responding to an email or call from
>>>> a Pictet  employee  you  are  taken  to  have  consented to such
>>>> monitoring and recording.
>>>> ________________________________________________________________
>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> 
>>> ________________________________________________________________
>>> 
>>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>>> CH-1211 GENEVE 73                        http://www.pictet.com/
>>> ________________________________________________________________
>>> 
>>> This document should only be read by those persons to whom it is
>>> addressed  and  is  not intended to be relied upon by any person
>>> without  subsequent written confirmation of its contents. If you
>>> have  received  this  e-mail message in error, please destroy it
>>> and delete it from your computer.
>>> Any  form of  reproduction, dissemination, copying,  disclosure,
>>> modification,  distribution  and/or  publication  of this E-mail
>>> message is strictly prohibited.
>>> ________________________________________________________________
>>> 
>>> Pictet reserves the right  to  monitor  and  record business and
>>> personal communications. By  responding to an email or call from
>>> a Pictet  employee  you  are  taken  to  have  consented to such
>>> monitoring and recording.
>>> ________________________________________________________________
>>> 
>>> 
>> 
>> 
>> 
>> 
>> ________________________________________________________________
>> 
>> PAM S.A Geneva                           Tel. +41 (0)58 323 3333
>> 60, route des Acacias                    Fax  +41 (0)58 323 2324
>> CH-1211 GENEVE 73                        http://www.pictet.com/
>> ________________________________________________________________
>> 
>> This document should only be read by those persons to whom it is
>> addressed  and  is  not intended to be relied upon by any person
>> without  subsequent written confirmation of its contents. If you
>> have  received  this  e-mail message in error, please destroy it
>> and delete it from your computer.
>> Any  form of  reproduction, dissemination, copying,  disclosure,
>> modification,  distribution  and/or  publication  of this E-mail
>> message is strictly prohibited.
>> ________________________________________________________________
>> 
>> Pictet reserves the right  to  monitor  and  record business and
>> personal communications. By  responding to an email or call from
>> a Pictet  employee  you  are  taken  to  have  consented to such
>> monitoring and recording.
>> ________________________________________________________________
>> 
>> 
>> <Q.txt><C.txt><BE.txt><AE.txt><AI.txt><BI.txt>
> 

Re: [ojAlgo-user] Future changes to the optimisation package

[Anders P. <an...@op...>]

Regarding improvements (or bringing back deprecated methods) you have to be explicit about what you want so I know what to consider. Not sure what you mean by "min. square objective function"...

I deprecated (and removed) all those variants of addEpression(?) because I realized there's an infinite  number of methods that could be useful, and instead of adding more variants I removed all but the most generic ones. I still think that was a good decision.

I've added a little bit to the on-line javadoc for Expressuon and ExpressionsBasedModel. What it is you find difficult? I can imagine two things:

1) What to set the contribution weight to. If you don't know what else to set it to then set it to 1.0 or leave it as null.

2) People (think they) know which solver to use, and they look at the documentation for that solver, and then try build matching "matrices" using the expressions  - even setting contribution weights based on parameters/signs they've seen there. This is the wrong way! Documentation for any specific solver is irrelevant when building a model. The only thing you have to remember is that there currently is no solver that can handle quadratic constraints.

/Anders


On 11 maj 2012, at 14:12, Petru AUSLAENDER wrote:

> Anders, 
> 
> That's fine, but then could you improve the ExpressionBasedModel API? Currently it is a bit cumbersome and error-prone for "novices", e.g. it is not easy to know when you're adding the quadratic objective function, the linear constraints, or / and the variable lower/upper limits. One possibility would be to have specific QuadraticExpression and LinearExpression classes. 
> 
> Also, some methods that are now deprecated were quiet usefull in the past - e.g. min. square objective function. 
> 
> Kind regards, 
> Pierre 
> 
> 
> 
> From:        "Anders Peterson" <an...@op...> 
>                 
> To:        "ojAlgo ojAlgo" <oja...@li...> 
> Date:        11.05.2012 12:38 
> Subject:        [ojAlgo-user] Future changes to the optimisation package 
> 
> 
> 
> There are a number of changes/fixes/additions that are long overdue for the optimisation package. Not sure when I'll have time to do them all, but right now I feel a little inspired.
> 
> Things (api) will change... Using the ExpressionsBasedModel to feed the solvers is, as far as I can tell, the future-safe way. Don't create matrices - create Expressions. Feeding the linear and quadratic solvers matrices directly will most likely still be supported in the future, but the api for how to do that will change.
> 
> /Anders
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> 
> 
> _____________________________________________________________________
> 
> Message électronique privé - les sociétés du Groupe Pictet n'assument 
> aucune responsabilité s'agissant du contenu de ce message;   celle-ci 
> incombe exclusivement à son auteur et / ou expéditeur.
> _____________________________________________________________________
> 
> Personal e-mail - Only the author and / or sender of this message and 
> not any of the Pictet Group entities may be held liable for the 
> content herein.
> _____________________________________________________________________
> 
> Pictet reserves the right to monitor and record business and personal 
> communications. By  responding  to  an  email  or  call from a Pictet
> employee  you  are  taken  to  have  consented to such monitoring and 
> recording.
> _____________________________________________________________________
> 
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user

Re: [ojAlgo-user] Future changes to the optimisation package

[Petru A. <pau...@pi...>]

Anders,

That's fine, but then could you improve the ExpressionBasedModel API? 
Currently it is a bit cumbersome and error-prone for "novices", e.g. it is 
not easy to know when you're adding the quadratic objective function, the 
linear constraints, or / and the variable lower/upper limits. One 
possibility would be to have specific QuadraticExpression and 
LinearExpression classes.

Also, some methods that are now deprecated were quiet usefull in the past 
- e.g. min. square objective function.

Kind regards,
Pierre



From:   "Anders Peterson" <an...@op...>
 
To:     "ojAlgo ojAlgo" <oja...@li...>
Date:   11.05.2012 12:38
Subject:        [ojAlgo-user] Future changes to the optimisation package



There are a number of changes/fixes/additions that are long overdue for 
the optimisation package. Not sure when I'll have time to do them all, but 
right now I feel a little inspired.

Things (api) will change... Using the ExpressionsBasedModel to feed the 
solvers is, as far as I can tell, the future-safe way. Don't create 
matrices - create Expressions. Feeding the linear and quadratic solvers 
matrices directly will most likely still be supported in the future, but 
the api for how to do that will change.

/Anders
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
ojAlgo-user mailing list
ojA...@li...
https://lists.sourceforge.net/lists/listinfo/ojalgo-user



_____________________________________________________________________

Message électronique privé - les sociétés du Groupe Pictet n'assument 
aucune responsabilité s'agissant du contenu de ce message;   celle-ci 
incombe exclusivement à son auteur et / ou expéditeur.
_____________________________________________________________________

Personal e-mail - Only the author and / or sender of this message and 
not any of the Pictet Group entities may be held liable for the 
content herein.
_____________________________________________________________________

Pictet reserves the right to monitor and record business and personal 
communications. By  responding  to  an  email  or  call from a Pictet
employee  you  are  taken  to  have  consented to such monitoring and 
recording.
_____________________________________________________________________

[ojAlgo-user] Future changes to the optimisation package

[Anders P. <an...@op...>]

There are a number of changes/fixes/additions that are long overdue for the optimisation package. Not sure when I'll have time to do them all, but right now I feel a little inspired.

Things (api) will change... Using the ExpressionsBasedModel to feed the solvers is, as far as I can tell, the future-safe way. Don't create matrices - create Expressions. Feeding the linear and quadratic solvers matrices directly will most likely still be supported in the future, but the api for how to do that will change.

/Anders

Re: [ojAlgo-user] Maven support for v32

[Anders P. <an...@op...>]

I have no idea who did it.

I did try to contact the repository admins asking about how to upload a new version, but I've not received an answer.

/Anders


On 6 maj 2012, at 18:39, Tamás Sallai wrote:

> Publishing as a maven artifact should be straightforward. It just
> needs a repository to publish the resources to, and a minimal pom.xml,
> but it is already present for the v31. Publishing to the central maven
> repo needs special privileges as not everybody is able to push there.
> The guy who published the v31 should push the v32 too, for him, it's
> just a matter of minutes.
> 
> Do you know who did it?
> 
> Thanks,
> sashee
> 
> On Thu, May 3, 2012 at 8:19 PM, Anders Peterson <an...@op...> wrote:
>> I'm not a Maven user myself, and I'm not the one who added v31 to that repository.
>> 
>> Exactly what is required to be maven compatible, or maven friendly?
>> 
>> /Anders
>> 
>> 
>> On 3 maj 2012, at 15:44, Tamás Sallai wrote:
>> 
>>> Hello,
>>> 
>>> Maven central has v31
>>> (http://mvnrepository.com/artifact/org.ojalgo/ojalgo), but still no
>>> v32. Is there another repository where I can depend on for the v32, or
>>> do you plan to release it the central?
>>> 
>>> Thanks,
>>> sashee
>>> 
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> ojAlgo-user mailing list
>>> ojA...@li...
>>> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
>>> 
>>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> ojAlgo-user mailing list
>> ojA...@li...
>> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> 

Re: [ojAlgo-user] Maven support for v32

[Tamás S. <gs...@gm...>]

Publishing as a maven artifact should be straightforward. It just
needs a repository to publish the resources to, and a minimal pom.xml,
but it is already present for the v31. Publishing to the central maven
repo needs special privileges as not everybody is able to push there.
The guy who published the v31 should push the v32 too, for him, it's
just a matter of minutes.

Do you know who did it?

Thanks,
sashee

On Thu, May 3, 2012 at 8:19 PM, Anders Peterson <an...@op...> wrote:
> I'm not a Maven user myself, and I'm not the one who added v31 to that repository.
>
> Exactly what is required to be maven compatible, or maven friendly?
>
> /Anders
>
>
> On 3 maj 2012, at 15:44, Tamás Sallai wrote:
>
>> Hello,
>>
>> Maven central has v31
>> (http://mvnrepository.com/artifact/org.ojalgo/ojalgo), but still no
>> v32. Is there another repository where I can depend on for the v32, or
>> do you plan to release it the central?
>>
>> Thanks,
>> sashee
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> ojAlgo-user mailing list
>> ojA...@li...
>> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
>>
>>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user

Re: [ojAlgo-user] Maven support for v32

[Anders P. <an...@op...>]

I'm not a Maven user myself, and I'm not the one who added v31 to that repository.

Exactly what is required to be maven compatible, or maven friendly?

/Anders


On 3 maj 2012, at 15:44, Tamás Sallai wrote:

> Hello,
> 
> Maven central has v31
> (http://mvnrepository.com/artifact/org.ojalgo/ojalgo), but still no
> v32. Is there another repository where I can depend on for the v32, or
> do you plan to release it the central?
> 
> Thanks,
> sashee
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> 

[ojAlgo-user] Maven support for v32

[Tamás S. <gs...@gm...>]

Hello,

Maven central has v31
(http://mvnrepository.com/artifact/org.ojalgo/ojalgo), but still no
v32. Is there another repository where I can depend on for the v32, or
do you plan to release it the central?

Thanks,
sashee

[ojAlgo-user] Does the 16GB array size limitation affect you?

[Anders P. <an...@op...>]

Currently ojAlgo has an array size limit of 2^31 - 1 which is the limit java has for arrays and collections. Even when you work with 2-dimensional arrays/matrixes or any-dimensional arrays ojAlgo has a limit of 2^31 - 1 elements, instead of having that many rows, columns, planes, cubes...

Compared to other libraries ojAlgo performs best when the arrays/matrices are a bit larger. I always felt that because of that it's a bit ironic that ojAlgo has a size limitation that some other libraries don't have.

An array of double with length 2^31 - 1 uses 16GB memory. For a 2-dimensional square matrix the size limit is 46340x46340. Personally I've never been close to this limit with real world ojAlgo usage.

Currently I'm thinking about adding support for huge "arrays" in ojAlgo, but I hesitate, and it would only be supported with Array1D, Array2D and ArrayAnyD - not with the various MatrixStore implementations. That would be too much of a change.

The implementation would of course require that some sort of array abstraction is introduced, and that will decrease performance (only when you actually use those huge arrays).

Anyone have an opinion? If I add this; will you, or anyone you know, be able to do things you cannot do now?

/Anders

Re: [ojAlgo-user] Interface analogous to Numpy?

[Stefan R. <Ste...@gm...>]

Hello together,
for simplicity, I will just comment in my opinion about your statements, regardless to the author...

> I thought learning about what would be possible was "the project". That
> SVN module is there if can be useful to you.

Agree ;-)


> Can't answer any of your questions regarding the importance of fully
> supporting various MumPy-capabilities. Why not just try to port some small
> application to Java/Jython?

Agree again. I think, Applications, which suffer from those java restrictions are so special-purpouse, that they should not be done via java anyway (and in my opinion also not in python) or should be outsourced into specialized native code.


> Don't know about such a benchmark. For simple operations like scaling, or
> adding to, all the aliments of an array I would expect little or no
> difference between Java and C/Fortran. For more complex operations like matrix
> decompositions I'd guess that C/Fortran is several times faster. Then again
> there can be be quite large differences between C/Fortran implementations...

I once thought about building kind of jumpy on top of JMT http://code.google.com/p/matrix-toolkits-java/ because it can use native lapack and blas bakcends for those "complex" operations. Problem with it is, that it does not support this any-dim-array-stuff as far as I know.

Maybe it would be a nice plan for future ojalgo (absolutely apart of this jumpy/numpy/jython/python thing) to support netlib-java (http://code.google.com/p/netlib-java/) or even JMT as a backend. So it could benefit from any native, high-efficiency blas or lapack implementation.


> > - unsigned long ist not supported. BigInteger as a replacement would be
> much
> >  too slow because of the many boxed heap object allocations (mentioned,
> >  for example, in Bloch, Effective Java, Item 49: Prefer primitive types
> to
> >  boxed primitives). How often are unsigned longs used in Numpy?

I think, this is a general jython-issue anyway, which we can't aim to cure here. I could imagine, that pure integer processing is a rather seldom used feature anyway.
If we could set up a jumpy capable of efficient floating point processing and a consistent and compatible but slow (and slow only when using longs) integer processing with relatively small effort, I would be completely satisfied for now.
Regarding to a project like jumpy, there is a very significant distance between "works with some restrictions" and "is not available at all".


> > - Java arrays (and, sadly, Collections as well) have a maximum length of
> >  Integer.MAX_VALUE which is specified as 2^31 - 1. Numpy arrays, on the
> other
> >  hand, can be much larger. If we use a 1-dimensional array with strides
> in
> >  Jumpy, this would limit our quadratic matrices to only 46340 rows in
> Java.
> >  But the design of Numpy is seriously tied to this underlying
> 1-dimensional
> >  array implementation, and applications might allocate larger ndarrays
> than we
> >  can provide.

Same as for unsigned long/BigInteger-issue. This could only resolved by my entire idea to adapt original numpy to jython by JNI, making the complete nd-array handling native. But this idea is not ojalgo-related anyway, so this is the wrong place to discuss it.



> > - We cannot support the Array interface (__array_interface__,
> >  __array_struct__) for interfacing C libraries and the ctypes foreign
> >  function interface of Python. How often are they used in Numpy
> applications,
> >  in particular Machine Learning applications?

I think, bringing numpy to jython would already be a huge benefit. Accessing other native libraries can't be the scope of this project in any case.



> > - I'm not sure whether Numpy applications sometimes need to perform
> casts of the
> >  contained data array to a different kind of array without copying, e.g.
> from
> >  int[] to byte[] with 4 times the size or vice versa. This could not be
> >  supported in Java.

It would be easy, to write some native methods for this. JNI allows to access java-arrays from c directly without copying (JNI can give you a memory pointer, which points to the actual array data inside the jvm in many cases - however this functionality is not guaranteed, but promised to work quite often).
I'm not sure, whether it would work, because I didn't try yet. But I think, there is a fair chance, that one could cast that c pointer and construct a new java-array from it via JNI. One would have to compile this for a sufficient set of platforms, of course.

Another possibility might be to use stuff from javax.nio. I'm not familiar with it, but I think, it was designed to do things like that. However, the ojalgo-class providing nd-array features would probably need to be reimplemented to use javax.nio, which would burst the effort needed for jumpy. 


> >> I created a module ExtNumPy in the already existing SVN repository.
> (Actually tried switching to Git but couldn't get it working.) You'll have to
> give me your Google user names if you want commit rights to that
> repository.

I have no google-account yet - will see to get one during the next days.



> >>>> I'm not sure, how jython deals with it. Can it map field-access in
> python-code to getter/setter methods on java-side? If not, numpy-python code
> would run only with an additional preprocessing.
> >>>> If you don't know, I would offer to check this out.
> >>> 
> >>> You can always use properties in Python. It's explained here, for
> example:
> >>>
> http://bytebaker.com/2009/03/31/python-properties-vs-java-access-modifiers/

I agree, it will probably work. Only thing I worried about is that java does not have this feature and whether this will go all through with jython.


> >>> 
> >>>> The rest of your post sounds good to me after a first view, although
> I must admit, that I only used rather basic numpy-stuff yet. For instance I
> very rarely used reshaping and never used striding, so I might lack enough
> experience to see problems in your approach.
> >>> 
> >>> Reshaping and striding are seldom used explicitly by the users, but
> very often
> >>> implicitly.

Never doubted that. Just wanted to say, that I might lack the experience needed to judge your ideas with a sufficient competence.



All in all I think, the project is still worth the effort and we should not worry about the arising restrictions in first place. We should aim to adopt as much numpy-functionality as is possible with a sensible large effort. Everything else could be done in future versions or on feature-request basis once a proof of concept is done.



Cheers

Stefan





> >>> Stefan Richthofer wrote:
> >>>>> There are many fields which are all public. We could replace them by
> set-
> >>>>> and
> >>>>> get methods in the adapter.
> >>>> 
> >>>> I'm not sure, how jython deals with it. Can it map field-access in
> python-code to getter/setter methods on java-side? If not, numpy-python code
> would run only with an additional preprocessing.
> >>>> If you don't know, I would offer to check this out.
> >>>> 
> >>>> The rest of your post sounds good to me after a first view, although
> I must admit, that I only used rather basic numpy-stuff yet. For instance I
> very rarely used reshaping and never used striding, so I might lack enough
> experience to see problems in your approach.
> >>>> 
> >>>> cheers
> >>>> 
> >>>> Stefan
> >>>> -- 
> >>>> Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
> >>>> belohnen Sie mit bis zu 50,- Euro!
> https://freundschaftswerbung.gmx.de
> >>>> 
> >>>>
> ------------------------------------------------------------------------------
> >>>> For Developers, A Lot Can Happen In A Second.
> >>>> Boundary is the first to Know...and Tell You.
> >>>> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
> >>>> http://p.sf.net/sfu/Boundary-d2dvs2
> >>>> _______________________________________________
> >>>> ojAlgo-user mailing list
> >>>> ojA...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> >>> 
> >>>
> ------------------------------------------------------------------------------
> >>> For Developers, A Lot Can Happen In A Second.
> >>> Boundary is the first to Know...and Tell You.
> >>> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
> >>> http://p.sf.net/sfu/Boundary-d2dvs2
> >>> _______________________________________________
> >>> ojAlgo-user mailing list
> >>> ojA...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> >>> 
> >>> 
> >> 
> >> 
> >>
> ------------------------------------------------------------------------------
> >> For Developers, A Lot Can Happen In A Second.
> >> Boundary is the first to Know...and Tell You.
> >> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
> >> http://p.sf.net/sfu/Boundary-d2dvs2
> >> _______________________________________________
> >> ojAlgo-user mailing list
> >> ojA...@li...
> >> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> > 
> >
> ------------------------------------------------------------------------------
> > For Developers, A Lot Can Happen In A Second.
> > Boundary is the first to Know...and Tell You.
> > Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
> > http://p.sf.net/sfu/Boundary-d2dvs2
> > _______________________________________________
> > ojAlgo-user mailing list
> > ojA...@li...
> > https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> > 
> > 
> 
> 
> ------------------------------------------------------------------------------
> For Developers, A Lot Can Happen In A Second.
> Boundary is the first to Know...and Tell You.
> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
> http://p.sf.net/sfu/Boundary-d2dvs2
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user

-- 
NEU: FreePhone 3-fach-Flat mit kostenlosem Smartphone!                                  
Jetzt informieren: http://mobile.1und1.de/?ac=OM.PW.PW003K20328T7073a

Re: [ojAlgo-user] Interface analogous to Numpy?

[Anders P. <an...@op...>]

I thought learning about what would be possible was "the project". That SVN module is there if can be useful to you.

Can't answer any of your questions regarding the importance of fully supporting various MumPy-capabilities. Why not just try to port some small application to Java/Jython?

Don't know about such a benchmark. For simple operations like scaling, or adding to, all the aliments of an array I would expect little or no difference between Java and C/Fortran. For more complex operations like matrix decompositions I'd guess that C/Fortran is several times faster. Then again there can be be quite large differences between C/Fortran implementations...

/Anders


On 21 apr 2012, at 13:01, Bruno Daniel wrote:

> Dear Anders Peterson and Stefan Richthofer,
> 
> let's first establish the feasibility of Jumpy before really starting the
> project. The problems I've seen so far -- thanks also to Stefan -- are:
> 
> - unsigned long ist not supported. BigInteger as a replacement would be much
>  too slow because of the many boxed heap object allocations (mentioned,
>  for example, in Bloch, Effective Java, Item 49: Prefer primitive types to
>  boxed primitives). How often are unsigned longs used in Numpy?
> - Java arrays (and, sadly, Collections as well) have a maximum length of
>  Integer.MAX_VALUE which is specified as 2^31 - 1. Numpy arrays, on the other
>  hand, can be much larger. If we use a 1-dimensional array with strides in
>  Jumpy, this would limit our quadratic matrices to only 46340 rows in Java.
>  But the design of Numpy is seriously tied to this underlying 1-dimensional
>  array implementation, and applications might allocate larger ndarrays than we
>  can provide.
> - We cannot support the Array interface (__array_interface__,
>  __array_struct__) for interfacing C libraries and the ctypes foreign
>  function interface of Python. How often are they used in Numpy applications,
>  in particular Machine Learning applications?
> - I'm not sure whether Numpy applications sometimes need to perform casts of the
>  contained data array to a different kind of array without copying, e.g. from
>  int[] to byte[] with 4 times the size or vice versa. This could not be
>  supported in Java.
> 
> And please forgive me for asking this question: Are there any benchmarks
> comparing ojAlgo to C/Fortran/C++ Linear Algebra libraries, e.g. Eigen (see
> http://eigen.tuxfamily.org/index.php?title=Benchmark )?
> 
> Best regards
>  Bruno Daniel
> 
> Anders Peterson wrote:
>> I created a module ExtNumPy in the already existing SVN repository. (Actually tried switching to Git but couldn't get it working.) You'll have to give me your Google user names if you want commit rights to that repository. /Anders
>> 
>> On 20 apr 2012, at 16:32, Bruno Daniel wrote:
>> 
>>> Dear Stefan,
>>> 
>>>> I'm not sure, how jython deals with it. Can it map field-access in python-code to getter/setter methods on java-side? If not, numpy-python code would run only with an additional preprocessing.
>>>> If you don't know, I would offer to check this out.
>>> 
>>> You can always use properties in Python. It's explained here, for example:
>>> http://bytebaker.com/2009/03/31/python-properties-vs-java-access-modifiers/
>>> 
>>>> The rest of your post sounds good to me after a first view, although I must admit, that I only used rather basic numpy-stuff yet. For instance I very rarely used reshaping and never used striding, so I might lack enough experience to see problems in your approach.
>>> 
>>> Reshaping and striding are seldom used explicitly by the users, but very often
>>> implicitly.
>>> 
>>> Best regards
>>> Bruno Daniel
>>> 
>>> Stefan Richthofer wrote:
>>>>> There are many fields which are all public. We could replace them by set-
>>>>> and
>>>>> get methods in the adapter.
>>>> 
>>>> I'm not sure, how jython deals with it. Can it map field-access in python-code to getter/setter methods on java-side? If not, numpy-python code would run only with an additional preprocessing.
>>>> If you don't know, I would offer to check this out.
>>>> 
>>>> The rest of your post sounds good to me after a first view, although I must admit, that I only used rather basic numpy-stuff yet. For instance I very rarely used reshaping and never used striding, so I might lack enough experience to see problems in your approach.
>>>> 
>>>> cheers
>>>> 
>>>> Stefan
>>>> -- 
>>>> Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir
>>>> belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
>>>> 
>>>> ------------------------------------------------------------------------------
>>>> For Developers, A Lot Can Happen In A Second.
>>>> Boundary is the first to Know...and Tell You.
>>>> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
>>>> http://p.sf.net/sfu/Boundary-d2dvs2
>>>> _______________________________________________
>>>> ojAlgo-user mailing list
>>>> ojA...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
>>> 
>>> ------------------------------------------------------------------------------
>>> For Developers, A Lot Can Happen In A Second.
>>> Boundary is the first to Know...and Tell You.
>>> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
>>> http://p.sf.net/sfu/Boundary-d2dvs2
>>> _______________________________________________
>>> ojAlgo-user mailing list
>>> ojA...@li...
>>> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
>>> 
>>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> For Developers, A Lot Can Happen In A Second.
>> Boundary is the first to Know...and Tell You.
>> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
>> http://p.sf.net/sfu/Boundary-d2dvs2
>> _______________________________________________
>> ojAlgo-user mailing list
>> ojA...@li...
>> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
> ------------------------------------------------------------------------------
> For Developers, A Lot Can Happen In A Second.
> Boundary is the first to Know...and Tell You.
> Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
> http://p.sf.net/sfu/Boundary-d2dvs2
> _______________________________________________
> ojAlgo-user mailing list
> ojA...@li...
> https://lists.sourceforge.net/lists/listinfo/ojalgo-user
> 
>