RDKit

Hi Andrew,

On Mon, Jan 25, 2010 at 4:45 AM, Andrew Dalke <dalke@...> wrote:
> I looked but found nothing in RDKit which gives a version string, like Q32009.

Yeah, that's an oversight. I'll add the function "version()" to
$RDBASE/rdkit/__init__.py for the release:
[1]>>> import rdkit

[2]>>> rdkit.version()
Out[2] 'Q42009_1'

Now I just need to remember to update that for each release...

A question: I'm tempted to rearrange that output to the somewhat less
human readable, but correctly sortable "2009Q4_1". What do you think?

-greg

On Jan 25, 2010, at 6:29 AM, Greg Landrum wrote:
> Yeah, that's an oversight. I'll add the function "version()" to
> $RDBASE/rdkit/__init__.py for the release:
> [1]>>> import rdkit
> 
> [2]>>> rdkit.version()
> Out[2] 'Q42009_1'

Thanks. I'll add that to my code. Done.

> Now I just need to remember to update that for each release...

Add it to the "HowToReleaseRDKit.txt" file? Metaphorically speaking.

> A question: I'm tempted to rearrange that output to the somewhat less
> human readable, but correctly sortable "2009Q4_1". What do you think?

I think I live in Sweden, where many things are dated YYYY-MM-DD
and this would be better. I think having the quarter number and the
year not next to each other is nice. So I would do it.

However, nothing I do is affected by the data ordering in the string.


What would be affected is distutils code. There are two versioning
styles it supports, Loose and Strict.

  http://pydoc.org/2.5.1/distutils.version.html

The RDKit number is "Loose", and those are sorted lexically.

Another thing which is sorted is directory listings.

For those reasons I would also switch to lexical ordering.


				Andrew
				dalke@...

On Mon, Jan 25, 2010 at 2:06 PM, Andrew Dalke <dalke@...> wrote:
> On Jan 25, 2010, at 6:29 AM, Greg Landrum wrote:
>> Yeah, that's an oversight. I'll add the function "version()" to
>> $RDBASE/rdkit/__init__.py for the release:
>> [1]>>> import rdkit
>>
>> [2]>>> rdkit.version()
>> Out[2] 'Q42009_1'
>
> Thanks. I'll add that to my code. Done.
>
>> Now I just need to remember to update that for each release...
>
> Add it to the "HowToReleaseRDKit.txt" file? Metaphorically speaking.

ha... if only I could do metaphorical releases.

Improving the release process is on my list of things that ought to be
done. The cmake-based build system should make that easier, but it's
still going to take some time.

>> A question: I'm tempted to rearrange that output to the somewhat less
>> human readable, but correctly sortable "2009Q4_1". What do you think?
>
> I think I live in Sweden, where many things are dated YYYY-MM-DD
> and this would be better. I think having the quarter number and the
> year not next to each other is nice. So I would do it.
>
> However, nothing I do is affected by the data ordering in the string.
>
>
> What would be affected is distutils code. There are two versioning
> styles it supports, Loose and Strict.
>
>  http://pydoc.org/2.5.1/distutils.version.html
>
> The RDKit number is "Loose", and those are sorted lexically.
>
> Another thing which is sorted is directory listings.
>
> For those reasons I would also switch to lexical ordering.

I just checked in the change.

-greg

On Jan 25, 2010, at 5:05 PM, Greg Landrum wrote:
> ha... if only I could do metaphorical releases.

> Improving the release process is on my list of things that ought to be
> done. The cmake-based build system should make that easier, but it's
> still going to take some time.

Just remember, metaphorical releases are not like a piece of cake, they *are* a piece of cake. Since otherwise it would be a simile.


> I just checked in the change.

Okay. Now in a related question, I read


    ******  Release_Q32009_1 *******
    (Changes relative to Release_Q22009_1)
    
    !!!!!! IMPORTANT !!!!!!
      - Due to bug fixes in the boost random-number generator, RDK
        fingerprints generated with boost 1.40 are not backwards
        compatible with those from earlier versions.


Is there any way to get this version information as well? Ideally I would like a short, identifier which uniquely identifies the fingerprint scheme, but I'm happy enough to synthesize my own if I have base information. This might be the boost version, or a bit of code which acts differently depending on the underlying operations.

Cheer,

				Andrew
				dalke@...

On Mon, Jan 25, 2010 at 6:29 AM, Greg Landrum <greg.landrum@...> wrote:
> Hi Andrew,
>
> On Mon, Jan 25, 2010 at 4:45 AM, Andrew Dalke <dalke@...> wrote:
>> I looked but found nothing in RDKit which gives a version string, like Q32009.
>
> Yeah, that's an oversight. I'll add the function "version()" to
> $RDBASE/rdkit/__init__.py for the release:
> [1]>>> import rdkit
>
> [2]>>> rdkit.version()
> Out[2] 'Q42009_1'
>

FYI: I just changed the way this is done in order to provide info
about boost version and have the same information available from C++.

I removed the function rdkit.version() and added two constants to the
module rdkit.rdBase:

[1]>>> from rdkit import rdBase

[2]>>> rdBase.rdkitVersion
Out[2] '2009Q4_1'

[3]>>> rdBase.boostVersion
Out[3] '1_41'


-greg

On Mon, Jan 25, 2010 at 6:07 PM, Andrew Dalke <dalke@...> wrote:
> On Jan 25, 2010, at 5:05 PM, Greg Landrum wrote:
>> ha... if only I could do metaphorical releases.
>
>> Improving the release process is on my list of things that ought to be
>> done. The cmake-based build system should make that easier, but it's
>> still going to take some time.
>
> Just remember, metaphorical releases are not like a piece of cake, they *are* a piece of cake. Since otherwise it would be a simile.

:-)

>> I just checked in the change.
>
> Okay. Now in a related question, I read
>
>
>    ******  Release_Q32009_1 *******
>    (Changes relative to Release_Q22009_1)
>
>    !!!!!! IMPORTANT !!!!!!
>      - Due to bug fixes in the boost random-number generator, RDK
>        fingerprints generated with boost 1.40 are not backwards
>        compatible with those from earlier versions.
>
>
> Is there any way to get this version information as well? Ideally I would like a short, identifier which uniquely identifies the fingerprint scheme, but I'm happy enough to synthesize my own if I have base information. This might be the boost version, or a bit of code which acts differently depending on the underlying operations.
>

The fingerprinting functions already have version numbers assigned to
them. I exposed this information to python this morning:
[4]>>> from rdkit.Chem import rdmolops

[5]>>> rdmolops.__LayeredFingerprint_version__
Out[5] '0.2.1'

[6]>>> rdmolops.__RDKFingerprint_version__
Out[6] '1.0.0'

[7]>>> from rdkit.Chem import rdMolDescriptors

[8]>>> rdMolDescriptors.__MorganFingerprint_version__
Out[8] '0.1.0'

That's part of it. The next part is the boost version, which is also
now available (see my previous post from this morning):

[10]>>> from rdkit import rdBase

[11]>>> rdBase.boostVersion
Out[11] '1_41'

I guess the RDKit version (rdBase.rdkitVersion) could also be
important, though it encompasses much more variability than just the
fingerprint versions.

But all these pieces of information don't really address the
compatibility problem: fingerprints between different versions of the
RDKit or different boost versions are often, but not always,
compatible with each other.

For example, the following combinations of versions should give
compatible fingerprints:
 - RDKit 2009Q3_1+boost 1.40; RDKit 2009Q4_1+boost 1.40; RDKit
2009Q3_1+boost 1.41; RDKit 2009Q4_1+boost 1.41
These combinations should also be compatible with each other, but not
with those above:
 - RDKit 2009Q3_1+boost 1.39; RDKit 2009Q4_1+boost 1.39; RDKit
2009Q3_1+boost 1.38; RDKit 2009Q4_1+boost 1.38

It would be nice to have versioning information associated with the
fingerprints that somehow captures this information, but I haven't
figured out a reasonable way of doing this... any ideas?

-greg

On Jan 26, 2010, at 6:55 AM, Greg Landrum wrote:
> The fingerprinting functions already have version numbers assigned to
> them. I exposed this information to python this morning:
> [4]>>> from rdkit.Chem import rdmolops
> 
> [5]>>> rdmolops.__LayeredFingerprint_version__
> Out[5] '0.2.1'

Just a note that leading and trailing "__"s are reserved
for the Python implementation.

Removing the trailing "__" should be fine, but why even
have the leading "__"?


> That's part of it. The next part is the boost version, which is also
> now available (see my previous post from this morning):
> 
> [10]>>> from rdkit import rdBase

Oh! Another note: the docstring from rdbase says

    No user-serviceable parts inside.

;)


> But all these pieces of information don't really address the
> compatibility problem: fingerprints between different versions of the
> RDKit or different boost versions are often, but not always,
> compatible with each other.
> 
> For example, the following combinations of versions should give
> compatible fingerprints:
> - RDKit 2009Q3_1+boost 1.40; RDKit 2009Q4_1+boost 1.40; RDKit
> 2009Q3_1+boost 1.41; RDKit 2009Q4_1+boost 1.41
> These combinations should also be compatible with each other, but not
> with those above:
> - RDKit 2009Q3_1+boost 1.39; RDKit 2009Q4_1+boost 1.39; RDKit
> 2009Q3_1+boost 1.38; RDKit 2009Q4_1+boost 1.38

Ooof. I see your point. But isn't that something to push down
into the __LayeredFingerprint_version__ string? That is, what's
the purpose of the version number that's there, if it can't be
used as a way to test if two fingerprints can be compared?

Perhaps there could be an ugly set of #defines, to get a
unique number based on the relevant factors?

#if BOOST_VERSION >= 140
  LayeredFingerprint_version = "1.1"
#else
  LayeredFingerprint_version = "1.0"
#endif

That's doable, but it's going to be ugly and a bit fragile.


The only other solution I really see would be to
distribute all the components (like the boost RNG code)
as part of the distribution.

This is definitely a place where binary-only distributions
have it easier.


				Andrew
				dalke@...

On Wed, Jan 27, 2010 at 3:58 AM, Andrew Dalke <dalke@...> wrote:
> On Jan 26, 2010, at 6:55 AM, Greg Landrum wrote:
>> The fingerprinting functions already have version numbers assigned to
>> them. I exposed this information to python this morning:
>> [4]>>> from rdkit.Chem import rdmolops
>>
>> [5]>>> rdmolops.__LayeredFingerprint_version__
>> Out[5] '0.2.1'
>
> Just a note that leading and trailing "__"s are reserved
> for the Python implementation.

For some reason, despite the fact that I've been using Python for more
than 10 years, I cannot retain the rules/conventions for underscores
in my head for more than a week at a time. I need to have the relevant
portions of PEP8 tattooed on my forearm. ;-)

> Removing the trailing "__" should be fine, but why even
> have the leading "__"?

I want something there to prevent the version names from polluting the
module namespaces too much. For example, they shouldn't show up as
options in "normal" command completion operations.

I switched all the version names to a single leading underscore:

[2]>>> rdmolops._LayeredFingerprint_version
Out[2] '0.2.1'

etc.

>
>> That's part of it. The next part is the boost version, which is also
>> now available (see my previous post from this morning):
>>
>> [10]>>> from rdkit import rdBase
>
> Oh! Another note: the docstring from rdbase says
>
>    No user-serviceable parts inside.
>
> ;)

I had already removed that. :-)

>> But all these pieces of information don't really address the
>> compatibility problem: fingerprints between different versions of the
>> RDKit or different boost versions are often, but not always,
>> compatible with each other.
>>
>> For example, the following combinations of versions should give
>> compatible fingerprints:
>> - RDKit 2009Q3_1+boost 1.40; RDKit 2009Q4_1+boost 1.40; RDKit
>> 2009Q3_1+boost 1.41; RDKit 2009Q4_1+boost 1.41
>> These combinations should also be compatible with each other, but not
>> with those above:
>> - RDKit 2009Q3_1+boost 1.39; RDKit 2009Q4_1+boost 1.39; RDKit
>> 2009Q3_1+boost 1.38; RDKit 2009Q4_1+boost 1.38
>
> Ooof. I see your point. But isn't that something to push down
> into the __LayeredFingerprint_version__ string? That is, what's
> the purpose of the version number that's there, if it can't be
> used as a way to test if two fingerprints can be compared?
>
> Perhaps there could be an ugly set of #defines, to get a
> unique number based on the relevant factors?
>
> #if BOOST_VERSION >= 140
>  LayeredFingerprint_version = "1.1"
> #else
>  LayeredFingerprint_version = "1.0"
> #endif
>
> That's doable, but it's going to be ugly and a bit fragile.

It's a topic that deserves a bit of thought. It's clearly not going to
be resolved for this release.

-greg