Re: [gplab-users] GPLab Performance Questions

[Jonathan S. <jon...@gm...>]

Well, I started down the path of trying to do a quick hack (which
ended up being a few hours) to link in the Matlab parallel computing
toolbox.  I think I nailed down where the focus should go.  Being new
to the toolkit, I may be misreading the code, but here it goes.

The calcpopfitness.m file has what I believe is a master loop to
iterate through all the individuals and calculate the fitness by
calling calcfitness.m.  I think the primary problem is that the
calcfitness function returns both the pop and status return values.
The state variable is updated with every individual update (as so it
should).  Most parallel functions can't have a return value for each
spawn - because you'll be overwriting the values with each return.

The first thought was to try and replace for with parfor, but because
of the return values, that won't work.

I then tried to just call calcfitness from calcpopfitness by just
passing the entire population, and then putting in a big loop in
calcfitness to replicate the parfor, but I think somewhere else in the
code recursively calls back to calcfitness.

The best way to do this, I believe, would be to find a way to split up
the individuals of a population (and their status), so that parallel
jobs using the local scheduler could perform the fitness evaluations
in parallel.  The return could be the fitness and status parameters of
the subset, and then when all the parallel tasks are complete for a
generation, the data could be combined in the master variable.

Again, being fairly new to the code (more the data structures) - it
might take some time to disect to try and add this feature.

Regards,
Jonathan

On Tue, Jan 5, 2010 at 7:53 AM, Sara Silva <sa...@de...> wrote:
> Hi Jonathan,
>
> Look for the parameter "drawperspin" in the user's manual, and see if it
> helps. I don't have a lot of experience with GPLAB in Linux, but I know
> Windows is not good at using the RAM.
>
> Please let me know if this helps,
> Sara
>
>
>
> Jonathan Schreiter wrote:
>>
>> Hello everyone,
>> I've just started using GPLab, and it works really well.  I have two
>> questions related to performance (matlab 7.8.0).
>>
>> Has anybody been able to improve CPU efficiency with the evaluations?
>> I have a multi-core (and multi-processor) computer, and only one of
>> the cores seems to be used for any of the calculations.  I have a
>> parallel computing toolbox, but changing the evaluator code is
>> probably required to improve this.  If anybody has any suggestions,
>> maybe I'll give it a try.
>>
>> Second, I have tried it on both a 64bit windows (16 GB) and linux
>> system (4GB).  I am experimenting with a modified regression demo, and
>> put the max depth at 1000 (although it never hit that) and 2000
>> individuals.  The memory sits around 450MB for Matlab.  When I try to
>> bump up to 3000 or greater individuals, it maxes out the memory on the
>> linux (over 16 GB on the Windows - and well in to my 100 GB swap).
>> The jump from 1000 to 2000 doesn't seem to bad (100 MB or so) - but
>> for some reason anything over this kills both systems.  Any thoughts?
>>
>> Many thanks,
>> Jonathan
>>
>>
>> ------------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Verizon Developer Community
>> Take advantage of Verizon's best-in-class app development support
>> A streamlined, 14 day to market process makes app distribution fast and
>> easy
>> Join now and get one step closer to millions of Verizon customers
>> http://p.sf.net/sfu/verizon-dev2dev
>> _______________________________________________
>> gplab-users mailing list
>> gpl...@li...
>> https://lists.sourceforge.net/lists/listinfo/gplab-users
>>
>