cclib-devel — Developer's mailing list

[cclib-devel] cclib methods/calculations

[Adam T. <a-t...@st...>]

I've been thinking about adapting PyMOlyze to work with cclib's  
parsers. I believe the mocoeffs format is slightly different than  
what PyMOlyze currently parses too, so I need to update the code that  
does population analyses. I figure since I'm making those  
modifications anyways, what if I just committed those changes to the  
cclib tree under a methods of calculations subdirectory? Then cclib  
would provide an interface to parsing files and using that data to do  
population analysis.

The methods I propose are:

1. C-squared PA (not as famous as mulliken, but can be done without  
aooverlap matrices)
2. Mulliken PA
3. Overlap PA
4. Mayer bond orders
5. Density and/or spin-density matrices
6. Charge density analysis (ooh, hard since it depends on multiple  
files!)

I don't think these functions would result in any data other than  
their result, so that makes me think they shouldn't have classes  
associated with them. Unless we want to keep referring to their data  
after the fact (c.result, or something like that).

What do you think? (Yeah, yeah, I know I should be working on the  
parsers, but I also need to update PyMOlyze at some point, and I  
figure it makes sense to start thinking about it now.)

Adam

[cclib-devel] mocoeffs wiki page

[Adam T. <a-t...@st...>]

I just created the mocoeffs wiki page because I was working the  
mocoeffs and realized I didn't understand how they worked. I played  
around a bit, and I think I understand the layout of the Numeric  
arrays. Can you look at the page and make sure my understanding is  
correct?

Adam

Re: [cclib-devel] SCF convergence in ADF

[Adam T. <a-t...@st...>]

> In summary, the situation is weird. Can you first try the keyword  
> Err or
> Err1 (http://www.scm.com/Doc/Doc2005.01/ADF/ADFUsersGuide/ 
> page158.html)
> to see whether it prints out anything more?
>
> Everything is understandable until GeoOpt Cycle 5, where it should  
> have
> terminated at Step 6.

The new output looks useful, although I haven't explored it as  
thoroughly as you did with the previous example. The files have been  
uploaded.

Adam

Re: [cclib-devel] SCF convergence in ADF

[Noel O'B. <no...@ca...>]

On Thu, 2006-04-27 at 18:45 -0700, Adam Tenderholt wrote:
> > On an unrelated note, the geo-opt files don't seem to contain scf
> > convergence info except at the very end of the file. I'm not sure
> > whether I should parse this, or try to find out whether there's some
> > keyword that will cause scf convergence info to be printed in the main
> > body of the file.
> 
> So I looked into this a bit. As I understand it, ADF determines SCF  
> convergence by comparing the Fock matrix and the density matrix (as  
> determined from the Fock matrix). Ideally, the difference would be 0,  
> but since that would take forever, it uses two threshold values:  
> scfcnv and scfcnv2. These can be set, but default to 1e-6 and 1e-3,  
> respectively. These are on lines 1761 in dvb_gopt.adfout. These two  
> values are used depending on whether it is a geometry optimization or  
> not. During the first and last geom cycle, it should use scfcnv;  
> otherwise, scfcnv2. More info is at:
> 
> http://www.scm.com/Doc/Doc2005.01/ADF/ADFUsersGuide/page124.html
> 
> During the SCF cycle, it prints imax (maximum matrix element) and d- 
> Pmat, which I think are the maximum element and norm of the matrix.  
> Unfortunately, the rules for determining whether the value is scfcnv  
> or scfcnv2 don't seem to be followed exactly in this calc. It looks  
> ok in the single point calc.
> 
> Is your understanding after reading the above link similar?

scfcnv = 1e-6 (used for start up and final step in geoopt)
scfcnv2 = 1e-3 (used for intermediate steps in geoopt, but the value
varies, although it will never be less than 1e-3, i.e. it could be
more!)

Looking at dvb_gopt.adfout..
Geo 1
CYCLE    1
CYCLE    2 d-Pmat  mean:  0.00E+00 imax=      1:  0.00E+00
CYCLE    3
Terminated: mean < 10*scfcnv and imax < scfcnv
Geo 2
CYCLE    1
CYCLE    2 d-Pmat  mean:  0.36E-15 imax=      2:  0.13E-14
CYCLE    3
Terminated: mean < 10*scfcnv2 and imax < scfcnv2
Geo 3
CYCLE    1
CYCLE    2 d-Pmat  mean:  0.52E-02 imax=     45: -0.34E-01
CYCLE    3 d-Pmat  mean:  0.53E-02 imax=     40: -0.23E-01
CYCLE    4 d-Pmat  mean:  0.62E-02 imax=     45: -0.35E-01
CYCLE    5 d-Pmat  mean:  0.22E-02 imax=     20:  0.84E-02
CYCLE    6 d-Pmat  mean:  0.29E-03 imax=     20: -0.69E-03
CYCLE    7
Terminated: Only at Cycle 6 is abs(imax) < scfcnv2
Geo 4
CYCLE    1
CYCLE    2 d-Pmat  mean:  0.88E-03 imax=     45:  0.42E-02
CYCLE    3 d-Pmat  mean:  0.15E-02 imax=     59:  0.94E-02 
CYCLE    4 d-Pmat  mean:  0.10E-02 imax=     45:  0.67E-02
CYCLE    5 d-Pmat  mean:  0.30E-03 imax=     15:  0.13E-02
CYCLE    6 d-Pmat  mean:  0.66E-04 imax=     25:  0.20E-03
CYCLE    7
Terminated: Only at Cycle 6 is imax < scfcnv2
Geo 5
CYCLE    1
CYCLE    2 d-Pmat  mean:  0.51E-03 imax=     15:  0.20E-02
CYCLE    3 d-Pmat  mean:  0.69E-03 imax=     59:  0.21E-02
CYCLE    4 d-Pmat  mean:  0.46E-03 imax=     45:  0.24E-02
CYCLE    5 d-Pmat  mean:  0.19E-03 imax=     45: -0.13E-02
CYCLE    6 d-Pmat  mean:  0.36E-04 imax=     45:  0.16E-03
CYCLE    7 d-Pmat  mean:  0.59E-05 imax=     20:  0.18E-04
CYCLE    8
Terminated: Strange!! At Cycle 6 imax < scfcnv2 and mean < 10*scfcnv2
Geo 6
CYCLE    1
CYCLE    2 d-Pmat  mean:  0.57E-03 imax=     45: -0.21E-02
CYCLE    3 d-Pmat  mean:  0.80E-03 imax=     39: -0.22E-02
CYCLE    4 d-Pmat  mean:  0.55E-03 imax=     45: -0.37E-02
CYCLE    5 d-Pmat  mean:  0.25E-03 imax=     45:  0.12E-02
CYCLE    6 d-Pmat  mean:  0.17E-04 imax=     55:  0.65E-04
CYCLE    7
Geo 7
CYCLE    1
CYCLE    2 d-Pmat  mean:  0.41E-03 imax=     45: -0.16E-02
CYCLE    3 d-Pmat  mean:  0.55E-03 imax=     39: -0.16E-02
CYCLE    4 d-Pmat  mean:  0.39E-03 imax=     45: -0.25E-02
CYCLE    5 d-Pmat  mean:  0.18E-03 imax=     45:  0.83E-03
CYCLE    6 d-Pmat  mean:  0.12E-04 imax=     55:  0.46E-04
CYCLE    7
Geo 8
CYCLE    1
CYCLE    2 d-Pmat  mean:  0.21E-03 imax=     45: -0.82E-03
CYCLE    3 d-Pmat  mean:  0.27E-03 imax=     39: -0.76E-03
CYCLE    4 d-Pmat  mean:  0.19E-03imax=     45: -0.12E-02
CYCLE    5 d-Pmat  mean:  0.87E-04 imax=     45:  0.42E-03
CYCLE    6 d-Pmat  mean:  0.53E-05 imax=     55:  0.20E-04
CYCLE    7
Geo 9
CYCLE    1
CYCLE    2 d-Pmat  mean:  0.98E-04 imax=     45: -0.41E-03
CYCLE    3 d-Pmat  mean:  0.13E-03 imax=     39: -0.37E-03
CYCLE    4 d-Pmat  mean:  0.92E-04 imax=     45: -0.58E-03
CYCLE    5 d-Pmat  mean:  0.41E-04 imax=     45:  0.19E-03
CYCLE    6 d-Pmat  mean:  0.30E-05 imax=     55:  0.11E-04
CYCLE    7 d-Pmat  mean:  0.57E-06 imax=     58: -0.16E-05
CYCLE    8 d-Pmat  mean:  0.12E-06 imax=     52: -0.42E-06
CYCLE    9

In summary, the situation is weird. Can you first try the keyword Err or
Err1 (http://www.scm.com/Doc/Doc2005.01/ADF/ADFUsersGuide/page158.html)
to see whether it prints out anything more?

Everything is understandable until GeoOpt Cycle 5, where it should have
terminated at Step 6.

If Err or Err1 doesn't do the trick, I'll post it on the ADF forum.

Regards,
 Noel

Re: [cclib-devel] Update

[Noel O'B. <no...@ca...>]

On Thu, 2006-05-04 at 10:23 -0700, Adam Tenderholt wrote:
> > I still need to look at the ADF SCF criteria...but I will.
> 
> I already added code for ADF. Hopefully you'll agree that it does  
> what we need it to do. I updated the wiki to reflect what's implemented.

Yes - I saw that.

> > I'm going to take a lesson in Jaguar either tomorrow or early next  
> > week,
> > as I need to tweak the settings on the test files. E.g. the geometry
> > optimisation converged immediately (given the G03 output coordinates),
> > so I don't have any information on the output of a geoopt. Also, I  
> > found
> > that virtual orbitals aren't printed unless a particular keyword is
> > given. And I need to figure out how to print the overlap matrix. So,
> > lots to be done there - but the test files need to be created first.
> 
> Ok. You're using Jaguar 4.2, right? 
Yes, indeed.
>  I think someone in my lab uses  
> Jaguar 5, so we should probably get some of those files because I  
> believe the output is different between the two versions...but I may  
> be wrong.
Sounds good.

> > We also need to start adding unittests for some of the other
> > calculations, e.g. the unrestricted calculation.
> 
> Yeah. I'm not as familiar with the unittest framework, but I suppose  
> I can learn. ;o)

Come on, I think you know more than you're saying - I saw that addition
for ADF scfvalues. ;-)

Which reminds me - do you think it's necessary for the entire diff to be
sent to the cclib-svn list. Other projects just use the log message. The
problem is that sometimes the file is too big, and so (as admin of the
list) I just reject it, rather than having the upload log files emailed
to the list. As a result, we miss a few log messages. I guess I don't
mind either way, but what do you think?

> > I'm thinking of making a TODO page on the wiki, which will focus us a
> > bit more on what we need to do before we make a release. How do you  
> > feel
> > things are going? Obviously, it's taking longer that we hoped at  
> > first,
> > but I think we don't really have any technical difficulties left -  
> > it's
> > just a question of ironing out bugs and the unittests should do a
> > reasonable job at that.
> 
> I think a TODO is a great idea.

OK, I'll make it so.

>  I feel like things are going a bit  
> slower than we anticipated, but what project doesn't have that  
> problem. From the way the table on the wiki looks, it appears we are  
> making some progress.

We definitely are.

> > In fact, I'm also thinking about making a preliminary release as  
> > soon as
> > possible - to get all the release stuff working (e.g. uploading to  
> > PyPi
> > and Sourceforge). Also, we can start incorporating cclib into our own
> > projects.
> 
> I think we should make a release fairly soon. I'd be completely happy  
> with an initial release if it just parsed aonames/fonames, mocoeffs,  
> and aooverlap/fooverlap since that's all I need to incorporate it  
> into PyMOlyze.

Yes - let's make that the target. Obviously, I want to continue cclib
devel to have a few more things, but these items are still the core
information. And we can advertise cclib a bit more on the web.

> > I'm very annoyed with Google as it still does not pick up the wiki.  
> > This
> > seems to be a permanent problem although I've tried to do some magic
> > with Google Sitemaps and a php file (off the web) that generates a
> > sitemap for MediaWiki. I think I will contact Geoff at OpenBabel as  
> > they
> > also use a wiki but they don't have this problem.
> 
> I have no idea in this area. I remember it took a while for Google to  
> pick up my other projects. I think it wasn't until after you had  
> linked to it from linux4chemistry. I dunno for sure tho.

Thing is, I've already a link to cclib.sourceforge.net from GaussSum and
it hasn't helped that much.

> Adam
> 
> 
> 
> -------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel

Re: [cclib-devel] Update

[Adam T. <a-t...@st...>]

> I still need to look at the ADF SCF criteria...but I will.

I already added code for ADF. Hopefully you'll agree that it does  
what we need it to do. I updated the wiki to reflect what's implemented.

> I'm going to take a lesson in Jaguar either tomorrow or early next  
> week,
> as I need to tweak the settings on the test files. E.g. the geometry
> optimisation converged immediately (given the G03 output coordinates),
> so I don't have any information on the output of a geoopt. Also, I  
> found
> that virtual orbitals aren't printed unless a particular keyword is
> given. And I need to figure out how to print the overlap matrix. So,
> lots to be done there - but the test files need to be created first.

Ok. You're using Jaguar 4.2, right? I think someone in my lab uses  
Jaguar 5, so we should probably get some of those files because I  
believe the output is different between the two versions...but I may  
be wrong.

> We also need to start adding unittests for some of the other
> calculations, e.g. the unrestricted calculation.

Yeah. I'm not as familiar with the unittest framework, but I suppose  
I can learn. ;o)

> I'm thinking of making a TODO page on the wiki, which will focus us a
> bit more on what we need to do before we make a release. How do you  
> feel
> things are going? Obviously, it's taking longer that we hoped at  
> first,
> but I think we don't really have any technical difficulties left -  
> it's
> just a question of ironing out bugs and the unittests should do a
> reasonable job at that.

I think a TODO is a great idea. I feel like things are going a bit  
slower than we anticipated, but what project doesn't have that  
problem. From the way the table on the wiki looks, it appears we are  
making some progress.

> In fact, I'm also thinking about making a preliminary release as  
> soon as
> possible - to get all the release stuff working (e.g. uploading to  
> PyPi
> and Sourceforge). Also, we can start incorporating cclib into our own
> projects.

I think we should make a release fairly soon. I'd be completely happy  
with an initial release if it just parsed aonames/fonames, mocoeffs,  
and aooverlap/fooverlap since that's all I need to incorporate it  
into PyMOlyze.

> I'm very annoyed with Google as it still does not pick up the wiki.  
> This
> seems to be a permanent problem although I've tried to do some magic
> with Google Sitemaps and a php file (off the web) that generates a
> sitemap for MediaWiki. I think I will contact Geoff at OpenBabel as  
> they
> also use a wiki but they don't have this problem.

I have no idea in this area. I remember it took a while for Google to  
pick up my other projects. I think it wasn't until after you had  
linked to it from linux4chemistry. I dunno for sure tho.

Adam

[cclib-devel] Update

[Noel O'B. <no...@ca...>]

Hello again Adam,

I still need to look at the ADF SCF criteria...but I will.

I'm going to take a lesson in Jaguar either tomorrow or early next week,
as I need to tweak the settings on the test files. E.g. the geometry
optimisation converged immediately (given the G03 output coordinates),
so I don't have any information on the output of a geoopt. Also, I found
that virtual orbitals aren't printed unless a particular keyword is
given. And I need to figure out how to print the overlap matrix. So,
lots to be done there - but the test files need to be created first.

We also need to start adding unittests for some of the other
calculations, e.g. the unrestricted calculation.

I'm thinking of making a TODO page on the wiki, which will focus us a
bit more on what we need to do before we make a release. How do you feel
things are going? Obviously, it's taking longer that we hoped at first,
but I think we don't really have any technical difficulties left - it's
just a question of ironing out bugs and the unittests should do a
reasonable job at that. 

In fact, I'm also thinking about making a preliminary release as soon as
possible - to get all the release stuff working (e.g. uploading to PyPi
and Sourceforge). Also, we can start incorporating cclib into our own
projects.

I'm very annoyed with Google as it still does not pick up the wiki. This
seems to be a permanent problem although I've tried to do some magic
with Google Sitemaps and a php file (off the web) that generates a
sitemap for MediaWiki. I think I will contact Geoff at OpenBabel as they
also use a wiki but they don't have this problem.

Regards,
 Noel

Re: [cclib-devel] SCF convergence in ADF

[Adam T. <a-t...@st...>]

> On an unrelated note, the geo-opt files don't seem to contain scf
> convergence info except at the very end of the file. I'm not sure
> whether I should parse this, or try to find out whether there's some
> keyword that will cause scf convergence info to be printed in the main
> body of the file.

So I looked into this a bit. As I understand it, ADF determines SCF  
convergence by comparing the Fock matrix and the density matrix (as  
determined from the Fock matrix). Ideally, the difference would be 0,  
but since that would take forever, it uses two threshold values:  
scfcnv and scfcnv2. These can be set, but default to 1e-6 and 1e-3,  
respectively. These are on lines 1761 in dvb_gopt.adfout. These two  
values are used depending on whether it is a geometry optimization or  
not. During the first and last geom cycle, it should use scfcnv;  
otherwise, scfcnv2. More info is at:

http://www.scm.com/Doc/Doc2005.01/ADF/ADFUsersGuide/page124.html

During the SCF cycle, it prints imax (maximum matrix element) and d- 
Pmat, which I think are the maximum element and norm of the matrix.  
Unfortunately, the rules for determining whether the value is scfcnv  
or scfcnv2 don't seem to be followed exactly in this calc. It looks  
ok in the single point calc.

Is your understanding after reading the above link similar?

Adam

Re: [cclib-devel] three things

[Noel O'B. <no...@ca...>]

On Mon, 2006-04-24 at 08:22 -0700, Adam Tenderholt wrote:
> > ADF seems to do things differently, so I guess we should too. I had  
> > the
> > same impression that ADF likes fragments. Since we really, in the end,
> > want to calculate Mulliken pops, etc. for fragments ourselves, you're
> > probably right that we should run with ADF on this. In terms of
> > standardising across different parsers, we may wish to move to a
> > different naming scheme, i.e. not 'ao', for the fragment-based  
> > orbitals
> > ('fo'?) that we are going to use in ADF, or it will be misleading to
> > users.
> 
> Calling them fonames makes sense to me. If the SFOs are actual atomic  
> orbitals (ie. no user-defined fragments and no symmetry), should we  
> try to make sure they are called aonames? Perhaps we can have set a  
> flag during parsing, and if it passes parsing everything, we can  
> simply set aonames equal to fonames. Or should there be no aonames in  
> ADF?

Sounds like a good idea. Will you write something along these lines
under aonames and fonames on the wiki, e.g. "ADF specific info"?

On an unrelated note, the geo-opt files don't seem to contain scf
convergence info except at the very end of the file. I'm not sure
whether I should parse this, or try to find out whether there's some
keyword that will cause scf convergence info to be printed in the main
body of the file.

> Adam
> 
> 
> 
> -------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel

Re: [cclib-devel] three things

[Adam T. <a-t...@st...>]

> ADF seems to do things differently, so I guess we should too. I had  
> the
> same impression that ADF likes fragments. Since we really, in the end,
> want to calculate Mulliken pops, etc. for fragments ourselves, you're
> probably right that we should run with ADF on this. In terms of
> standardising across different parsers, we may wish to move to a
> different naming scheme, i.e. not 'ao', for the fragment-based  
> orbitals
> ('fo'?) that we are going to use in ADF, or it will be misleading to
> users.

Calling them fonames makes sense to me. If the SFOs are actual atomic  
orbitals (ie. no user-defined fragments and no symmetry), should we  
try to make sure they are called aonames? Perhaps we can have set a  
flag during parsing, and if it passes parsing everything, we can  
simply set aonames equal to fonames. Or should there be no aonames in  
ADF?

Adam

Re: [cclib-devel] three things

[Noel O'B. <no...@ca...>]

On Sat, 2006-04-22 at 11:20 -0700, Adam Tenderholt wrote:
> > Print Smat and EPrint SCF Eigvec looks useful, and I uploaded  
> > dvb_sp_b.adfout
> > as a test. Read the svn log for a description of challenges we face.
> 
> Awhile ago I added a simple Mo atom with BAS overlap and coeffs, but  
> only commented on it in the svn log. Unfortunately, the presence of  
> frozen core orbitals complicates the orbitals. For instance, look at  
> the BAS eigenvectors for anything. You can see that they are linear  
> combinations of orbitals that include the core basis functions. This  
> means, there isn't a 1 to 1 mapping between BAS functions and  
> aonames. Bummer. I don't think my understanding of BAS -> CFs -> SFOs  
> is good enough to figure out how to create a mapping between BAS and  
> real aonames right now. So...
> 
> We need to discuss how we'll parse aonames for ADF calculations. I  
> see three options:
> 
> 1) Don't parse aonames unless the molecule has nosymmetry (ie. MO  
> labels are are 1A, 2A, etc.)
> 
> 2) aonames become something like 1C_1S+4C_1S or 1C+4C_1S+1S for the  
> "positive" case and 1C_1Px-4C_1Px or 1C+4C_1Px-1Px for the negative  
> case.
> 
> 3) Take time and learn out how to map BAS to aonames. It ought to  
> just involve using the Core functions to linearize the valence  
> functions. This will be complicated with d and f functions because  
> one of the d cartesian function is an "s" orbital, and 3 of the 10 f  
> functions are "p" orbitals (says the ADF manual).
> 
> My preference is number 2. Since ADF allows "fragment" calculations,  
> it would be nice to keep track of the MOs in a basis of the fragment  
> MOs instead of atomic orbitals. I can come up with an example of how  
> this would be useful if you want. I'm pretty sure doing option 3  
> would get rid of this useful info. I'm against option 1 because the  
> symmetry labels are also useful information, although it's not quite  
> like info from the fragments.
> 
> What do you think?

ADF seems to do things differently, so I guess we should too. I had the
same impression that ADF likes fragments. Since we really, in the end,
want to calculate Mulliken pops, etc. for fragments ourselves, you're
probably right that we should run with ADF on this. In terms of
standardising across different parsers, we may wish to move to a
different naming scheme, i.e. not 'ao', for the fragment-based orbitals
('fo'?) that we are going to use in ADF, or it will be misleading to
users. 

> Adam
> 
> 
> 
> -------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel

Re: [cclib-devel] three things

[Adam T. <a-t...@st...>]

> Print Smat and EPrint SCF Eigvec looks useful, and I uploaded  
> dvb_sp_b.adfout
> as a test. Read the svn log for a description of challenges we face.

Awhile ago I added a simple Mo atom with BAS overlap and coeffs, but  
only commented on it in the svn log. Unfortunately, the presence of  
frozen core orbitals complicates the orbitals. For instance, look at  
the BAS eigenvectors for anything. You can see that they are linear  
combinations of orbitals that include the core basis functions. This  
means, there isn't a 1 to 1 mapping between BAS functions and  
aonames. Bummer. I don't think my understanding of BAS -> CFs -> SFOs  
is good enough to figure out how to create a mapping between BAS and  
real aonames right now. So...

We need to discuss how we'll parse aonames for ADF calculations. I  
see three options:

1) Don't parse aonames unless the molecule has nosymmetry (ie. MO  
labels are are 1A, 2A, etc.)

2) aonames become something like 1C_1S+4C_1S or 1C+4C_1S+1S for the  
"positive" case and 1C_1Px-4C_1Px or 1C+4C_1Px-1Px for the negative  
case.

3) Take time and learn out how to map BAS to aonames. It ought to  
just involve using the Core functions to linearize the valence  
functions. This will be complicated with d and f functions because  
one of the d cartesian function is an "s" orbital, and 3 of the 10 f  
functions are "p" orbitals (says the ADF manual).

My preference is number 2. Since ADF allows "fragment" calculations,  
it would be nice to keep track of the MOs in a basis of the fragment  
MOs instead of atomic orbitals. I can come up with an example of how  
this would be useful if you want. I'm pretty sure doing option 3  
would get rid of this useful info. I'm against option 1 because the  
symmetry labels are also useful information, although it's not quite  
like info from the fragments.

What do you think?

Adam

Re: [cclib-devel] scfvalues cannot be an array

[Adam T. <a-t...@st...>]

Sounds reasonable.

Adam

On Apr 18, 2006, at 6:08 AM, Noel O'Boyle wrote:

> I've realised that scfvalues cannot be a Numeric array as the  
> number of
> dimensions varies (that is, different geo opt steps have different
> numbers of scf cycles). We could use a list of Numeric arrays instead.
> If you agree, I'll update the wiki (and the code).
>
> Regards,
>  Noel
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting  
> language
> that extends applications into web and mobile media. Attend the  
> live webcast
> and join the prime developer group breaking into this new coding  
> territory!
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel

[cclib-devel] scfvalues cannot be an array

[Noel O'B. <no...@ca...>]

I've realised that scfvalues cannot be a Numeric array as the number of
dimensions varies (that is, different geo opt steps have different
numbers of scf cycles). We could use a list of Numeric arrays instead.
If you agree, I'll update the wiki (and the code).

Regards,
 Noel

Re: [cclib-devel] Miscellaneous

[Dr N. O'B. <no...@ca...>]

On Apr 14 2006, Adam Tenderholt wrote:

>> (2) It'd be good if you could run testall.py before uploading svn
>> changes as this guarantees that code doesn't break, and that we move in
>> the direction of passing increasing numbers of tests (a bug in the
>> latest revision means that all tests failed for adfparser). And I think
>> it's a pretty neat way of comparing the results from different programs
>> for some sort of parser validation.
>
>In the case of my most recent commit, I think you're right. It was a
>silly bug for the gopt case that I hadn't considered. However, I feel
>this isn't practical sometimes. What if I start work on a file, take a
>few days off before getting it to pass the tests, and then you begin
>working on something I had already started? We'd be duplicating our
>efforts. This may be a good thing because you might have the better
>solution, but may also be a waste of time. Also, what if we start work
>on one computer (work) and then finish it on another computer (home)?
>It's easy to hunt down the changed files and copy them as needed, but
>it's not as trivial as just updating your working copy.
>
>I know these two scenarios are rather unlikely and/or the "hard" fix
>outside of svn is actually quite easy, but I think they should at least
>be mentioned. If you think we should just ignore them, that's fine.
>Otherwise, I have two suggestions:
>
>1) Create unstable branches for those few times when we're going to
>start working on a file, but we can't get it to pass the unit tests
>before we take a few day break. This would be as simple as svn copy to
>create new branch, switch to that branch, and then make changes. After
>work is finished, it can be merged (hopefully automatically) back into
>the main branch.
>
>2) Run the tests before a commit, and if anything breaks, say which part
> in the commit log so it's not a complete surprise to other devs. We'll
>be able to review the changes that caused the break, and revert or fix
>them as necessary.
>
>What do you think?

In fact (and I've been thinking about this question and your response over 
the past few days, as well as reading the SVN book) I'd like to use a 
variation of 1. Since you usually don't know in advance that the tests will 
break just before it's time to leave work/go to bed, etc., it's not too 
difficult to create a new branch in-place and commit your changes to it.

a) Run the tests before commiting b) If tests that previously passed now no 
longer do (when we have a more complete and stable release, this will read 
"If any tests fail"), and you don't have time to fix things before 
commiting, commits your changes to a new branch as follows:
  1) svn copy https://svn.sourceforge.net/svnroot/cclib/trunk
               
https://svn/sourceforge/net/svnroot/cclib/branches/brokenadfparser
              -m "Informative log message about why you're branching"
  2) change directory into the 'top' of your working copy where setup.py is
  3) svn switch 
https://svn.sourceforge.net/svnroot/cclib/branches/brokenadfparser
     (note that this preserves the local modifications, only now these 
modifications are to the branch instead of the trunk)
  4) svn commit (commits the local modifications to the branch)

As soon as the tests that previously passed are passed again, merge the 
changes and remove the branch (should be within a revision or two of the 
branch).
  1) svn log --stop-on-copy (to find the revision when the branching took 
place)
  2) svn switch https://...../mytrunk
  3) svn merge --dry-run -r123:HEAD https://..../mybranch
  4) svn merge -r123:HEAD https://..../mybranch
  5) svn commit -m "Merging 123:HEAD of mybranch into trunk" 
  6) svn remove https://..../mybranch

(BTW, I've never tried these instructions, so I'll put them on the wiki and 
we can edit them into correctness when we actually try them out)

I'd really like to do this. Of course, it won't always be possible but I 
think it's good to have a policy like this. It means that our code is 
always improving, and means that test failures are real, and need to be 
addressed. Of course, it's a bit of overkill for two developers, but hey 
:-) Also, I used to be scared of branches with CVS, but with SVN it's a lot 
simpler, so it'd be good to have some experience messing around with them.

I should say that at the same time I wouldn't like to have long- or even 
medium-term branches for cclib. I think it'd be good to merge and remove a 
branch as soon as the code's not broken.

What do you think of all this? I don't want branching policies to get in 
the way of development.

Regards,
 Noel

Re: [cclib-devel] Miscellaneous

[Adam T. <a-t...@st...>]

> On some miscellaneous notes,
> (1) I've put information on the symmetry labels used by ADF and GAMESS
> on the mosyms page. Have you considered that we should extract all of
> the symmetry labels if we extract all of the evalues? The info on the
> wiki needs to be updated if we agree to go down this route.

I agree that we should extract all the symmetry labels if we extract all
the evalues. I'll update the wiki later.

> (2) It'd be good if you could run testall.py before uploading svn
> changes as this guarantees that code doesn't break, and that we move in
> the direction of passing increasing numbers of tests (a bug in the
> latest revision means that all tests failed for adfparser). And I think
> it's a pretty neat way of comparing the results from different programs
> for some sort of parser validation.

In the case of my most recent commit, I think you're right. It was a
silly bug for the gopt case that I hadn't considered. However, I feel
this isn't practical sometimes. What if I start work on a file, take a
few days off before getting it to pass the tests, and then you begin
working on something I had already started? We'd be duplicating our
efforts. This may be a good thing because you might have the better
solution, but may also be a waste of time. Also, what if we start work
on one computer (work) and then finish it on another computer (home)?
It's easy to hunt down the changed files and copy them as needed, but
it's not as trivial as just updating your working copy.

I know these two scenarios are rather unlikely and/or the "hard" fix
outside of svn is actually quite easy, but I think they should at least
be mentioned. If you think we should just ignore them, that's fine.
Otherwise, I have two suggestions:

1) Create unstable branches for those few times when we're going to
start working on a file, but we can't get it to pass the unit tests
before we take a few day break. This would be as simple as svn copy to
create new branch, switch to that branch, and then make changes. After
work is finished, it can be merged (hopefully automatically) back into
the main branch.

2) Run the tests before a commit, and if anything breaks, say which part
 in the commit log so it's not a complete surprise to other devs. We'll
be able to review the changes that caused the break, and revert or fix
them as necessary.

What do you think?

> (3) With regard to aonames (or anything else), upload any relevant test
> files although try to aim for as small a file as possible. If you could
> add a description to the metadata, it'll make it easier for me to do the
> same (if necessary) for some of the other codes. As you are taking the
> lead on aonames, it'd also be good if you put up some specs/examples on
> the wiki, and I'll make sure other parsers meet the specs.

Ok. It's basically because I'm not sure how frozen core basis functions
with ADF translate into an aoname yet. I figure single atoms will be
sufficient.

Happy Easter!

Adam

[cclib-devel] Miscellaneous

[Dr N. O'B. <no...@ca...>]

On some miscellaneous notes, 

(1) I've put information on the symmetry labels used by ADF and GAMESS on 
the mosyms page. Have you considered that we should extract all of the 
symmetry labels if we extract all of the evalues? The info on the wiki 
needs to be updated if we agree to go down this route.

(2) It'd be good if you could run testall.py before uploading svn changes 
as this guarantees that code doesn't break, and that we move in the 
direction of passing increasing numbers of tests (a bug in the latest 
revision means that all tests failed for adfparser). And I think it's a 
pretty neat way of comparing the results from different programs for some 
sort of parser validation.

(3) With regard to aonames (or anything else), upload any relevant test 
files although try to aim for as small a file as possible. If you could add 
a description to the metadata, it'll make it easier for me to do the same 
(if necessary) for some of the other codes. As you are taking the lead on 
aonames, it'd also be good if you put up some specs/examples on the wiki, 
and I'll make sure other parsers meet the specs.

Happy Easter,
   Noel

[cclib-devel] ADF aonames (SVN r64)

[Adam T. <a-t...@st...>]

The test file for printing BAS info only includes single-zeta info, and only 
handles s/p orbitals right now. We'll have to look at it again once a higher 
quality basis set is added that includes multiple-zeta, polarization, and 
core functions is used.

I propose we test a few single point calculations of a d- and f- block metal 
(TZP basis set). What do you think?

Adam

-- 
Want a web browser that is fully customizable? 

Go to http://www.mozilla.org/products/firefox/

to download the newest version of Firefox and 
its available extentions.

Re: [cclib-devel] three things

[Adam T. <a-t...@st...>]

> Maybe there's an easier solution. I've been looking at the ADF manual
> online. Try "Print Smat". It appears to print an overlap matrix of basis
> functions. If true, I think this may solve our problems.

Print Smat and EPrint SCF Eigvec looks useful, and I uploaded dvb_sp_b.adfout 
as a test. Read the svn log for a description of challenges we face.

Adam

Re: [cclib-devel] Wiki for specifications of attributes

[Noel O'B. <no...@ca...>]

On Mon, 2006-04-10 at 10:28 -0700, Adam Tenderholt wrote:
> > I propose http://cclib.sourceforge.net/wiki/index.php/Parsed_Data as the
> > home of the cclib specification (instead of logfileparser.py),
> > especially as we can put more specific information (for developers, in
> > particular) on hyperlinked 'subpages'.
> 
> Sounds reasonable. Are you going to take charge in setting up a couple example 
> pages so the rest of us can follow your lead?

Well, you know, just random stuff is fine. For example, we discussed
keeping track of the convergence criteria of different programs...we
should paste this in behind scftargets and geotargets.

> > One thing that we should clarify is how we handle geometry optimisation
> > files. When we extract MO symmetries and evalues, should we extract the
> > final MO symmetry and evalues, the first one, all of them? My parsers
> > extract the final one, by rewriting over the previous one every time it
> > comes to a new block, but this is something we need to agree on.
> 
> I think we should keep every evalue for a geometry optimization so that the 
> progress of optimization can be monitored if so desired. I'm not too sure 
> keeping track of symmetry would be useful, so we can add it later if 
> requested.
I'm not sure whether any of this is useful, but I think we need to do
the same for all of the affected attributes. If you think it's useful to
keep all evalues, then we should keep all everything. Otherwise the
interface will be pretty confusing. Users won't know which ones are kept
and which not, etc.

> > Don't despair though - we're getting there! Although, on another note, I
> > have just gotten a copy of GAMESS-UK from the computing officer, and it
> > appears to have completely different input and output compared to the
> > other GAMESSes. Yikes!
> 
> Sounds like fun.

I'll upload the data files as soon as I have them.

> > If we get ADF/GAMESS/Gaussian/Jaguar in a decent state, we can make the
> > first release of cclib, and get people using it (including ourselves
> > regarding GaussSum and PyMOlyze). Then onwards and upwards.
> 
> > BTW, there are some magic SVN commands that will allow you to include a
> > particular version of cclib from SVN (e.g. the tagged 0.1 release) into
> > your own PyMOlyze SVN repository. I think you will find this handy
> > rather than asking users to install cclib separately.
> 
> Are you thinking of svn copy? This is a really great idea, although I'm sure 
> it'll require some setup.py tweaking to make it work.

Um, I forget - looking quickly at the book, I think it's "Externals
definition" in Chap 7.

OK - gotta go...

Noel

Re: [cclib-devel] three things

[Noel O'B. <no...@ca...>]

On Mon, 2006-04-10 at 10:51 -0700, Adam Tenderholt wrote:
> > I guess my question is, is it possible to translate the mocoeffs matrix
> > from being based on the Symmetry Adapted Linear Combinations (or
> > whatever) to being based on the original atomic orbitals. Isn't this
> > what we need to do to make it possible to do our population analyses?
> > We're not interested in how much 1/sqrt(2) (C1s + C2s) contributes to a
> > particular molecular orbital, but in how much C1s contributes.
> 
> This should be possible since the coefficients of each atomic orbital in a 
> SALC orbital is printed. I see three challeges/problems:
> 
> First, there are only two significant figures printed for these aoname/SALC 
> coefficients, which I feel isn't enough since the mocoeffs often have 5 or 
> more sigfigs. I suppose for now, we can use the actual value 1/sqrt(2) when 
> we find 0.71 and address other possibilities as they come up.
> 
> Second, is splitting up the SALC overlap matrix trivial like for the mocoeffs? 
> My intuition says no, but I can't think of an example.
> 
> Third, I can't think of a trivial way to do all of this. We'd probably have to 
> read everything in, do some list/dictionary magic, and at the end, convert 
> the SALC matrices to aoname matrices.

Maybe there's an easier solution. I've been looking at the ADF manual
online. Try "Print Smat". It appears to print an overlap matrix of basis
functions. If true, I think this may solve our problems.

> Any thoughts?
> 
> Adam
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel

Re: [cclib-devel] three things

[Adam T. <a-t...@st...>]

> I guess my question is, is it possible to translate the mocoeffs matrix
> from being based on the Symmetry Adapted Linear Combinations (or
> whatever) to being based on the original atomic orbitals. Isn't this
> what we need to do to make it possible to do our population analyses?
> We're not interested in how much 1/sqrt(2) (C1s + C2s) contributes to a
> particular molecular orbital, but in how much C1s contributes.

This should be possible since the coefficients of each atomic orbital in a 
SALC orbital is printed. I see three challeges/problems:

First, there are only two significant figures printed for these aoname/SALC 
coefficients, which I feel isn't enough since the mocoeffs often have 5 or 
more sigfigs. I suppose for now, we can use the actual value 1/sqrt(2) when 
we find 0.71 and address other possibilities as they come up.

Second, is splitting up the SALC overlap matrix trivial like for the mocoeffs? 
My intuition says no, but I can't think of an example.

Third, I can't think of a trivial way to do all of this. We'd probably have to 
read everything in, do some list/dictionary magic, and at the end, convert 
the SALC matrices to aoname matrices.

Any thoughts?

Adam

Re: [cclib-devel] Wiki for specifications of attributes

[Adam T. <a-t...@st...>]

> I propose http://cclib.sourceforge.net/wiki/index.php/Parsed_Data as the
> home of the cclib specification (instead of logfileparser.py),
> especially as we can put more specific information (for developers, in
> particular) on hyperlinked 'subpages'.

Sounds reasonable. Are you going to take charge in setting up a couple example 
pages so the rest of us can follow your lead?

> One thing that we should clarify is how we handle geometry optimisation
> files. When we extract MO symmetries and evalues, should we extract the
> final MO symmetry and evalues, the first one, all of them? My parsers
> extract the final one, by rewriting over the previous one every time it
> comes to a new block, but this is something we need to agree on.

I think we should keep every evalue for a geometry optimization so that the 
progress of optimization can be monitored if so desired. I'm not too sure 
keeping track of symmetry would be useful, so we can add it later if 
requested.

> Don't despair though - we're getting there! Although, on another note, I
> have just gotten a copy of GAMESS-UK from the computing officer, and it
> appears to have completely different input and output compared to the
> other GAMESSes. Yikes!

Sounds like fun.

> If we get ADF/GAMESS/Gaussian/Jaguar in a decent state, we can make the
> first release of cclib, and get people using it (including ourselves
> regarding GaussSum and PyMOlyze). Then onwards and upwards.

Glad to hear. I'll try to work on cclib a bit this week. We still need to 
address ADF aonames, but I think that is best discussed in the other thread.

> BTW, there are some magic SVN commands that will allow you to include a
> particular version of cclib from SVN (e.g. the tagged 0.1 release) into
> your own PyMOlyze SVN repository. I think you will find this handy
> rather than asking users to install cclib separately.

Are you thinking of svn copy? This is a really great idea, although I'm sure 
it'll require some setup.py tweaking to make it work.

Adam

[cclib-devel] Wiki for specifications of attributes

[Noel O'B. <no...@ca...>]

I propose http://cclib.sourceforge.net/wiki/index.php/Parsed_Data as the
home of the cclib specification (instead of logfileparser.py),
especially as we can put more specific information (for developers, in
particular) on hyperlinked 'subpages'.

One thing that we should clarify is how we handle geometry optimisation
files. When we extract MO symmetries and evalues, should we extract the
final MO symmetry and evalues, the first one, all of them? My parsers
extract the final one, by rewriting over the previous one every time it
comes to a new block, but this is something we need to agree on.

Don't despair though - we're getting there! Although, on another note, I
have just gotten a copy of GAMESS-UK from the computing officer, and it
appears to have completely different input and output compared to the
other GAMESSes. Yikes!

If we get ADF/GAMESS/Gaussian/Jaguar in a decent state, we can make the
first release of cclib, and get people using it (including ourselves
regarding GaussSum and PyMOlyze). Then onwards and upwards.

BTW, there are some magic SVN commands that will allow you to include a
particular version of cclib from SVN (e.g. the tagged 0.1 release) into
your own PyMOlyze SVN repository. I think you will find this handy
rather than asking users to install cclib separately.

Regards,
 Noel

Re: [cclib-devel] three things

[Noel O'B. <no...@ca...>]

On Thu, 2006-04-06 at 09:06 -0700, Adam Tenderholt wrote:
> > I've been thinking some more about this and now I think that we should
> > avoid Classes altogether, as this has been our method until now (e.g.
> > for molecular orbitals we have symmetries and coefficients as separate
> > attributes, for vibrations, we have raman intensity, and symmetry as
> > separate items, etc.). We can uses atomcoords, atomnos, atomXXXX, etc.
> > Or reduce 'atom' to 'at', or just 'a'.
> 
> Sounds reasonable.
> 
> > For the moment, I propose atomcoords (as we already have atomnos). If
> > you agree, we can add this to our specification (in the docstring of
> > logfileparser.py).
> 
> So atomcoords will be a list of array[2] (with dimensions natom x 3)?
> The list will be length 1 for single point calcs, and however long for
> geometry optimizations?

Yes, it'll be as you say if I understand you correctly. Remember that we
will be using Numeric arrays, and that atomcoords.shape should give
(n,m,3) where n is 1 for an sp (>=1 for a geoopt), m is the number of
atoms. Since it's not efficient/easy to extend Numeric arrays when
reading through a file, I find it better to just convert it from lists
to an array at the end of the file (see the GAMESS/Gaussian parsers).

> Adam
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email is sponsored by xPML, a groundbreaking scripting language
> that extends applications into web and mobile media. Attend the live webcast
> and join the prime developer group breaking into this new coding territory!
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel