Hmm, interesting observation. There is very little in mpl that limits your ability to produce elements for plotting (which is probably why you were getting shrugs from the mailing list...). However, ipython has various "tricks" for caching mpl elements, and can sometimes be a bit excessive in memory usage. This could lead to some issues.
Now that you have identified ipython as the culprit, I would suggest taking this to the ipython list and seeing if they can better identify the problem for you.