Re: [Numpy-discussion] Numpy x Matlab: some synthetic benchmarks

[eric j. <er...@en...>]

In scipy, we talked about having a benchmark_xyz methods that could be 
added to the test classes.  These weren't run during unit tests 
(scipy.test()) but would could be run using scipy.benchmark() or 
something like that.  I can't remember if Pearu got the machinery in 
place, but it seems to me it wouldn't be so hard.  You would have to add 
guards around benchmarks that compare to 3rd party tools, obviously, so 
that people without them could still run the benchmark suite.  Adding a 
regression process that checks against results from previous builds to 
flag potential problems when a slow down is noted would be good -- that 
is more work.

Anyway, something flagging these "tests" as benchmarks instead of 
standard correctness tests seems like a good idea.

eric



Fernando Perez wrote:
> Matthew Brett wrote:
>> Hi,
>>
>>
>>> Travis asked me to benchmark numpy versus matlab in some basic linear
>>> algebra operations. Here are the resuts for matrices/vectors of
>>> dimensions 5, 50 and 500:
>>
>>
>> This is really excellent, thanks.  Is there any chance we can make
>> these and other benchmarks part of the pre-release testing?  Apart
>> from testing for bottlenecks, if we could show that we were in the
>> ballpark of matlab for speed for each release, this would be very
>> helpful for those us trying to persuade our matlab colleagues to
>> switch.
>
> +1
>
> It might not be part of test(1), but at test(10) these tests could be 
> automatically run, activating each line as each package is found (or 
> not) in the system (Numeric, numarray, matlab).  This way, people who 
> have matlab on their box can even get a real-time check of how their 
> fresh-off-svn numpy fares against matlab that day.
>
> cheers,
>
> f
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log 
> files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion