Re: [Rdkit-discuss] Using SDMolSupplier

[Greg L. <gre...@gm...>]

Marshall,

I just checked in some changes that should fix the problems. I need to
put in some more testing code before I'm completely happy that it's
ok, but things should be better behaved now.

Note: one consequence of the change I made is that some molecules that
previously failed in either embedding or optimization may not fail any
more. This isn't because new parameters magically appeared, just
because I now no longer try to add terms to the force field where one
or more of the atoms don't have parameters. This is not necessarily a
good thing. I'm going to add a call to check to see if a molecule is
fully parameterized to catch these cases.

-greg


On Sun, Apr 12, 2009 at 6:46 PM, Greg Landrum <gre...@gm...> wrote:
> Hi Marshall,
>
> First an administrivia thing: I'm going to reject the message from the
> list since the attachments are larger than sourceforge likes. I still
> have the whole message, so no worries.
>
> My general assumption with python and memory leaks is that if I'm
> using an extension module (as you are with the RDKit), the problem is
> in the extension module, not in Python itself. It's not easy to make
> python leak core.
>
> I should have some time tomorrow to look into the problem you're
> having and see if I can fix it. Thanks for sending the test case; that
> really helps.
>
> -greg
>
>
> On Sat, Apr 11, 2009 at 5:26 AM, Marshall Levesque
> <mar...@em...> wrote:
>> Greg-
>>
>> This memory leak has now become a real hurdle when processing lots and
>> lots of compounds through SDMolSupplier (>25k cmpds with multiple
>> conformers) via EmbedMultipleConfs, followed by UFFOptimizedMolecule.
>>
>> I have tried different methods of iterating through the supplier (for
>> i,mol in enumerate(suppl):  or  for mol in suppl: , etc) and also
>> tried clearing the variables that are being used to hold the molecules
>> for processing via "del mol".
>>
>> Hours of looking through webpages on people dealing memory leaks, has
>> made me distrust everything about Python. :-)
>>
>> If you can have time to and try to process the attached file with the
>> attached script, you might be able to quickly see what's wrong with my
>> script or find some other cause.
>>
>> usage of memleak.py:
>>
>> $ python memoryleak.py mem_test.sdf
>>
>> Thanks!
>>
>> -Marshall
>>
>> On Mon, Mar 30, 2009 at 9:31 PM, Greg Landrum <gre...@gm...> wrote:
>>> On Tue, Mar 31, 2009 at 6:26 AM, Marshall Levesque
>>> <mar...@em...> wrote:
>>>> I actually do see a gradual increase in the memory footprint (a leak) that
>>>> went from around ~5MB to about 400MB while processing a 187MB SD file
>>>> (containing 100,000 cmpds).  I was using the script I have provided
>>>> previously.
>>>
>>> Ok, I will look into it and see if I can find the leak.
>>>
>>> Thanks for the report.
>>> -greg
>>>
>>
>