From: Todd <tod...@gm...> - 2013-10-10 11:33:38
|
I have been implementing some new plot types, with tests. This code passes all existing tests. I have also expanded the tests on some existing plot types and mlab functions. These tests run fine on their own. The problem is that, when I run the code with the new tests, I get a lot of out of memory errors. Further, the errors do not occur in the new tests, but rather in other, unrelated tests. Further, the tests that fail work fine when run on their own, they only fail when run as part of the complete test suite. Even stranger, when I run the tests in parallel (even with only one process) and enable "--process-restartworker", the tests run fine (with a large enough timeout). But "--process-restartworker" doesn't help if parallel tests are not turned on. So I am not sure exactly what to do here. Even if I leave out my own tests, I may be running into some limit or memory leak that may very well result in problems for other people down the road. A solution might be to force tests to run in parallel with "--process-restartworker", but of course it would be better to find out where the leak is. |
From: Michael D. <md...@st...> - 2013-10-10 13:22:54
|
Are your tests including the "@cleanup" decorator? (The @cleanup decorator is run implicitly with the @image_comparison decorator, so you really only need one or the other). Beyond that wild guess, I'm not sure what could be going on. You could file a pull request with your new code, even if it's not fully ready, so we could try it out and poke at it. Or just point us to your git branch so we could check it out. Mike On 10/10/2013 07:33 AM, Todd wrote: > I have been implementing some new plot types, with tests. This code > passes all existing tests. I have also expanded the tests on some > existing plot types and mlab functions. These tests run fine on their > own. > > The problem is that, when I run the code with the new tests, I get a > lot of out of memory errors. Further, the errors do not occur in the > new tests, but rather in other, unrelated tests. Further, the tests > that fail work fine when run on their own, they only fail when run as > part of the complete test suite. > > Even stranger, when I run the tests in parallel (even with only one > process) and enable "--process-restartworker", the tests run fine > (with a large enough timeout). But "--process-restartworker" doesn't > help if parallel tests are not turned on. > > So I am not sure exactly what to do here. Even if I leave out my own > tests, I may be running into some limit or memory leak that may very > well result in problems for other people down the road. > > A solution might be to force tests to run in parallel with > "--process-restartworker", but of course it would be better to find > out where the leak is. > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk > > > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel -- _ |\/|o _|_ _. _ | | \.__ __|__|_|_ _ _ ._ _ | ||(_| |(_|(/_| |_/|(_)(/_|_ |_|_)(_)(_)| | | http://www.droettboom.com |
From: Todd <tod...@gm...> - 2013-10-10 21:02:55
|
The branch is matplotlib/toddrjen spectral: https://github.com/toddrjen/matplotlib/tree/spectral Specifically the tests that are causing the problem are in test_mlab.py. I tried reorganizing the tests into subclasses and implementing a cleanup class (that is the current HEAD), but the problem exists even without that commit. You can cherry-pick 50c90102a929af5d534e551fd624abffeb9470b8 and 7c1b4db8b2d04826e267781c0de1bcc622f0fdb5. On Thu, Oct 10, 2013 at 3:22 PM, Michael Droettboom <md...@st...> wrote: > Are your tests including the "@cleanup" decorator? (The @cleanup > decorator is run implicitly with the @image_comparison decorator, so you > really only need one or the other). > > Beyond that wild guess, I'm not sure what could be going on. You could > file a pull request with your new code, even if it's not fully ready, so we > could try it out and poke at it. Or just point us to your git branch so we > could check it out. > > Mike > > > On 10/10/2013 07:33 AM, Todd wrote: > > I have been implementing some new plot types, with tests. This code > passes all existing tests. I have also expanded the tests on some existing > plot types and mlab functions. These tests run fine on their own. > > The problem is that, when I run the code with the new tests, I get a lot > of out of memory errors. Further, the errors do not occur in the new > tests, but rather in other, unrelated tests. Further, the tests that fail > work fine when run on their own, they only fail when run as part of the > complete test suite. > > Even stranger, when I run the tests in parallel (even with only one > process) and enable "--process-restartworker", the tests run fine (with a > large enough timeout). But "--process-restartworker" doesn't help if > parallel tests are not turned on. > > So I am not sure exactly what to do here. Even if I leave out my own > tests, I may be running into some limit or memory leak that may very well > result in problems for other people down the road. > > A solution might be to force tests to run in parallel with > "--process-restartworker", but of course it would be better to find out > where the leak is. > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register >http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Matplotlib-devel mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > > > > -- > _ > |\/|o _|_ _. _ | | \.__ __|__|_|_ _ _ ._ _ > | ||(_| |(_|(/_| |_/|(_)(/_|_ |_|_)(_)(_)| | | > http://www.droettboom.com > > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk > _______________________________________________ > Matplotlib-devel mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel > > |