cclib-devel — Developer's mailing list

Re: [cclib-devel] parser-refactoring

[Adam T. <a-t...@st...>]

Hi Karol,

That fixed it.

However, using PyMOlyze this morning exposed another bug. ADF parsers  
should have frags and fragnames attributes because ADF sometimes  
alters the atom order so that there isn't a direct mapping between  
fonames (ie. C1_3s) and atomcoords. Look in the svn log and ADF test  
files for more details.

Adam

On Feb 6, 2007, at 6:24 AM, Karol Langner wrote:

> That would affect any logfile - I cut out the definition of nstep  
> without
> checking if it is used (and forgot the tests don't check progress).  
> Revision
> 510 is fixed for this.
>
> Thanks for the testing,
> Karol
>
> On Monday 05 of February 2007 19:07, Adam Tenderholt wrote:
>> Minor problem with Rev. 509 and ADF files:
>>>>> from cclib.parser import ccopen
>>>>> from cclib.progress import TextProgress
>>>>> progress = TextProgress()
>>>>> import logging
>>>>> parser = ccopen("dvb_sp.adfout", progress, logging.ERROR)
>>>>> parser.parse()
>>
>> [=========-]  97%    Unsupported informationTraceback (most recent
>> call last):
>>    File "<stdin>", line 1, in ?
>>    File "/Library/Frameworks/Python.framework/Versions/2.4/lib/
>> python2.4/site-packages/cclib/parser/logfileparser.py", line 223, in
>> parse
>>      self.progress.update(nstep, "Done")
>> NameError: global name 'nstep' is not defined
>>
>> I haven't checked if it affects other calculation files yet. Just
>> noticed it when I was using PyMOlyze this morning on an ADF  
>> file... ;o)
>>
>> Adam
>
> -- 
> written by Karol Langner
> Tue Feb  6 15:22:22 CET 2007
>
> ---------------------------------------------------------------------- 
> ---
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your  
> job easier.
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel

Re: [cclib-devel] parser-refactoring

[Karol L. <kar...@kn...>]

That would affect any logfile - I cut out the definition of nstep without 
checking if it is used (and forgot the tests don't check progress). Revision 
510 is fixed for this.

Thanks for the testing,
Karol

On Monday 05 of February 2007 19:07, Adam Tenderholt wrote:
> Minor problem with Rev. 509 and ADF files:
>  >>> from cclib.parser import ccopen
>  >>> from cclib.progress import TextProgress
>  >>> progress = TextProgress()
>  >>> import logging
>  >>> parser = ccopen("dvb_sp.adfout", progress, logging.ERROR)
>  >>> parser.parse()
>
> [=========-]  97%    Unsupported informationTraceback (most recent
> call last):
>    File "<stdin>", line 1, in ?
>    File "/Library/Frameworks/Python.framework/Versions/2.4/lib/
> python2.4/site-packages/cclib/parser/logfileparser.py", line 223, in
> parse
>      self.progress.update(nstep, "Done")
> NameError: global name 'nstep' is not defined
>
> I haven't checked if it affects other calculation files yet. Just
> noticed it when I was using PyMOlyze this morning on an ADF file... ;o)
>
> Adam

-- 
written by Karol Langner
Tue Feb  6 15:22:22 CET 2007

Re: [cclib-devel] parser-refactoring

[Adam T. <a-t...@st...>]

Minor problem with Rev. 509 and ADF files:

 >>> from cclib.parser import ccopen
 >>> from cclib.progress import TextProgress
 >>> progress = TextProgress()
 >>> import logging
 >>> parser = ccopen("dvb_sp.adfout", progress, logging.ERROR)
 >>> parser.parse()
[=========-]  97%    Unsupported informationTraceback (most recent  
call last):
   File "<stdin>", line 1, in ?
   File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ 
python2.4/site-packages/cclib/parser/logfileparser.py", line 223, in  
parse
     self.progress.update(nstep, "Done")
NameError: global name 'nstep' is not defined

I haven't checked if it affects other calculation files yet. Just  
noticed it when I was using PyMOlyze this morning on an ADF file... ;o)

Adam


On Feb 4, 2007, at 3:25 AM, Karol Langner wrote:

> On Sunday 04 of February 2007 00:24, Adam Tenderholt wrote:
>> Hi Karol,
>>
>> I found a couple of problems:
>>
>> 1) Gaussian object has no attribute optfinished (line 77 in _extract)
>> when I run PyMOlyze (and ccget) on trunk/data/Gaussian/Gaussian03/
>> Mo4OSibdt2-opt.log.bz2.
>
> That's strange, I don't get that error:
>
>>>> import cclib
>>>> cclib.parser.gaussianparser.__revision__
> '$Revision: 509 $'
>>>> g = cclib.parser.ccopen("Mo4OSibdt2-opt.log.bz2")
>>>> g.parse()
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute atomcoords[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute atomnos[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute natom: 62
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute gbasis[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute nbasis: 581
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute nmo: 581
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute scftargets[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute scfvalues[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute  
> scfenergies[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute mosyms[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute homos[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute moenergies[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute aonames[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute mocoeffs[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute geovalues[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute geotargets[]
> [Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute  
> coreelectrons[]
>
> Everything looks fine here. Are you sure you're using revision 509  
> of the
> Gaussian parser? The attribute optfinished is set to False at the  
> beginning.
> Using ccget with the file also works for me, and all the regression  
> tests
> finish fine.
>
>> 2) nstep isn't defined when I try to open trunk/data/Gaussian/
>> basicGaussian03/dvb_gopt.out with PyMOlyze. Presumably, this isn't
>> seen in ccget because it doesn't use a progress class. This appears
>> to be a problem with a lot of the other dvb_gopt/sp files from ADF
>> and GAMESS calcs.
>
>> This is after completely removing cclib from python's site-packages
>> (remove all traces of trunk) and reinstalling cclib from revision 508
>> of your new branch. There's also a bunch of errors that I don't
>> recall seeing from trunk/test/regression.py.
>
> That's what I get for not reading mails to the end :) Could you try  
> out
> revision 509 from the branch? That's when I added minor fixes for  
> these
> attributes
>
> Karol
>
> -- 
> written by Karol Langner
> Sun Feb  4 12:14:55 CET 2007
>
> ---------------------------------------------------------------------- 
> ---
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your  
> job easier.
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel

Re: [cclib-devel] parser-refactoring

[Karol L. <kar...@kn...>]

On Sunday 04 of February 2007 00:24, Adam Tenderholt wrote:
> Hi Karol,
>
> I found a couple of problems:
>
> 1) Gaussian object has no attribute optfinished (line 77 in _extract)
> when I run PyMOlyze (and ccget) on trunk/data/Gaussian/Gaussian03/
> Mo4OSibdt2-opt.log.bz2.

That's strange, I don't get that error:

>>> import cclib
>>> cclib.parser.gaussianparser.__revision__
'$Revision: 509 $'
>>> g = cclib.parser.ccopen("Mo4OSibdt2-opt.log.bz2")
>>> g.parse()
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute atomcoords[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute atomnos[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute natom: 62
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute gbasis[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute nbasis: 581
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute nmo: 581
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute scftargets[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute scfvalues[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute scfenergies[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute mosyms[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute homos[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute moenergies[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute aonames[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute mocoeffs[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute geovalues[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute geotargets[]
[Gaussian Mo4OSibdt2-opt.log.bz2 INFO] Creating attribute coreelectrons[]

Everything looks fine here. Are you sure you're using revision 509 of the 
Gaussian parser? The attribute optfinished is set to False at the beginning. 
Using ccget with the file also works for me, and all the regression tests 
finish fine.

> 2) nstep isn't defined when I try to open trunk/data/Gaussian/
> basicGaussian03/dvb_gopt.out with PyMOlyze. Presumably, this isn't
> seen in ccget because it doesn't use a progress class. This appears
> to be a problem with a lot of the other dvb_gopt/sp files from ADF
> and GAMESS calcs.

> This is after completely removing cclib from python's site-packages
> (remove all traces of trunk) and reinstalling cclib from revision 508
> of your new branch. There's also a bunch of errors that I don't
> recall seeing from trunk/test/regression.py.

That's what I get for not reading mails to the end :) Could you try out 
revision 509 from the branch? That's when I added minor fixes for these 
attributes

Karol

-- 
written by Karol Langner
Sun Feb  4 12:14:55 CET 2007

Re: [cclib-devel] parser-refactoring

[Adam T. <a-t...@st...>]

Hi Karol,

I found a couple of problems:

1) Gaussian object has no attribute optfinished (line 77 in _extract)  
when I run PyMOlyze (and ccget) on trunk/data/Gaussian/Gaussian03/ 
Mo4OSibdt2-opt.log.bz2.

2) nstep isn't defined when I try to open trunk/data/Gaussian/ 
basicGaussian03/dvb_gopt.out with PyMOlyze. Presumably, this isn't  
seen in ccget because it doesn't use a progress class. This appears  
to be a problem with a lot of the other dvb_gopt/sp files from ADF  
and GAMESS calcs.

This is after completely removing cclib from python's site-packages  
(remove all traces of trunk) and reinstalling cclib from revision 508  
of your new branch. There's also a bunch of errors that I don't  
recall seeing from trunk/test/regression.py.

Adam

On Feb 3, 2007, at 2:50 PM, Karol Langner wrote:

> I finished refactoring the parsers in the new branch ("parser- 
> refactoring").
>
> Besides moving the loop over to the base class, most of the changes  
> were
> connected with moving bits and pieces around and turning variables  
> into class
> attributes when they're used across calls of _extract().
>
> All tests now finish like in the trunk, and the user side shouldn't  
> be any
> different than it was. Something might go wrong, though, in real-life
> situations, so I would be grateful for testing the lateset revision  
> in this
> branch with some normal usage (GassSum and, PyMOlyze?). I'm not  
> going to add
> anything new here for now.
>
> As to making _extract a dictionary of functions - from here on   
> this is
> simple, since the function is already divided into clear parts. It  
> won't
> simplify the code anymore, though, so I would leave this for the  
> future. Also
> - where would the many extracting funtions be defined? It's not a  
> good idea,
> for example, to populate the parser object with so many new  
> attrbiutes.
>
> Also changed/added:
>
> 1) Updategprogress is still needed... because alot of the parsing  
> code updates
> progress inside loops and such.
> 2) I wrote a new __setattr__ method for LogFile - it updates the  
> logger, so no
> more worrying about this in the parsers!
> 3) I also added a loop after parsing that deletes all attributes  
> that are not
> in self._nodelete (which is set utomatically), so that all the  
> temporary
> attributes used across calls of _extract() during parsing go away.
> 4) Notice that now in order for an attribute to be parsed it HAS TO  
> be added
> to LogFile._attrlist, otherwise it will get deleted, even if set by
> _extract().
>
> Looking forward to your feedback,
> Karol
>
> -- 
> written by Karol Langner
> Sat Feb  3 17:38:31 CET 2007
>
> ---------------------------------------------------------------------- 
> ---
> Using Tomcat but need to do more? Need to support web services,  
> security?
> Get stuff done quickly with pre-integrated technology to make your  
> job easier.
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  
> Geronimo
> http://sel.as-us.falkag.net/sel? 
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel

[cclib-devel] parser-refactoring

[Karol L. <kar...@kn...>]

I finished refactoring the parsers in the new branch ("parser-refactoring").

Besides moving the loop over to the base class, most of the changes were 
connected with moving bits and pieces around and turning variables into class 
attributes when they're used across calls of _extract(). 

All tests now finish like in the trunk, and the user side shouldn't be any 
different than it was. Something might go wrong, though, in real-life 
situations, so I would be grateful for testing the lateset revision in this 
branch with some normal usage (GassSum and, PyMOlyze?). I'm not going to add 
anything new here for now.

As to making _extract a dictionary of functions - from here on  this is 
simple, since the function is already divided into clear parts. It won't 
simplify the code anymore, though, so I would leave this for the future. Also 
- where would the many extracting funtions be defined? It's not a good idea, 
for example, to populate the parser object with so many new attrbiutes.

Also changed/added:

1) Updategprogress is still needed... because alot of the parsing code updates 
progress inside loops and such.
2) I wrote a new __setattr__ method for LogFile - it updates the logger, so no 
more worrying about this in the parsers!
3) I also added a loop after parsing that deletes all attributes that are not 
in self._nodelete (which is set utomatically), so that all the temporary 
attributes used across calls of _extract() during parsing go away.
4) Notice that now in order for an attribute to be parsed it HAS TO be added 
to LogFile._attrlist, otherwise it will get deleted, even if set by 
_extract().

Looking forward to your feedback,
Karol

-- 
written by Karol Langner
Sat Feb  3 17:38:31 CET 2007

Re: [cclib-devel] Refactoring

[Karol L. <kar...@kn...>]

On Saturday 03 of February 2007 11:05, you wrote:
> Since you are moving at a faster pace than we can make decisions it's
> probably better for you to make a branch for this refactoring, so that
> you can continue without losing momentum and can experiment a bit.

OK, done.

> I think the refactoring changes up till now in trunk have been
> relatively minor, but I'm happy to do as Adam suggests and and make a
> cclib-0.7 release branch from before the refactoring. All future
> development will be done on the trunk anyway and stable changes or
> completion of agreed features will be merged to the release branch.

OK, So I haven't reverted the changes up till now, and started off fro mteh 
current revision.

Karol

-- 
written by Karol Langner
Sat Feb  3 11:17:11 CET 2007

Re: [cclib-devel] Refactoring

[Noel O'B. <bao...@gm...>]

Since you are moving at a faster pace than we can make decisions it's
probably better for you to make a branch for this refactoring, so that
you can continue without losing momentum and can experiment a bit.

I think the refactoring changes up till now in trunk have been
relatively minor, but I'm happy to do as Adam suggests and and make a
cclib-0.7 release branch from before the refactoring. All future
development will be done on the trunk anyway and stable changes or
completion of agreed features will be merged to the release branch.

Noel

On 03/02/07, Karol Langner <kar...@kn...> wrote:
> On Friday 02 of February 2007 20:47, Adam Tenderholt wrote:
> > > Sounds like this is starting to turn into a major refactoring, and I'm
> > > a bit wary of having this on the trunk, especially as it will be
> > > experimental for the first while. Can you put this on a branch instead
> > > and just test with a single parser for the moment?
> >
> > Karol, it does sound like you are doing some interesting work, but I
> > agree with Noel that this should be developed someplace other than
> > where our current changes are. Especially since my most recent
> > version of PyMOlyze uses trunk as that's where the CDA and
> > FragmentAnalysis code is.
> >
> > What are the current features we are hoping to have in cclib 0.7? If
> > there aren't going to be many changes (esp to parsers), I think it
> > makes sense to merge trunk (from before the refactoring) into a
> > cclib-0.7 branch. If we decide we like the refactoring changes, we
> > can merge those into cclib-0.7 once its stable. What do y'all think?
>
> OK, so tell me which way to go - branch off with the refactoring or create a
> 0.7 branch? I would opt for the first, becuase I was really hoping to get the
> refactoring over with quickly and get on to some "real" additions :)
>
> Conceptually, the parsing function structure I proposed is not far from a dict
> of functions, so maybe that would be a good first step to take.
>
> Karol
>
> --
> written by Karol Langner
> Sat Feb  3 10:26:45 CET 2007
>

Re: [cclib-devel] Refactoring

[Karol L. <kar...@kn...>]

On Friday 02 of February 2007 20:47, Adam Tenderholt wrote:
> > Sounds like this is starting to turn into a major refactoring, and I'm
> > a bit wary of having this on the trunk, especially as it will be
> > experimental for the first while. Can you put this on a branch instead
> > and just test with a single parser for the moment?
>
> Karol, it does sound like you are doing some interesting work, but I
> agree with Noel that this should be developed someplace other than
> where our current changes are. Especially since my most recent
> version of PyMOlyze uses trunk as that's where the CDA and
> FragmentAnalysis code is.
>
> What are the current features we are hoping to have in cclib 0.7? If
> there aren't going to be many changes (esp to parsers), I think it
> makes sense to merge trunk (from before the refactoring) into a
> cclib-0.7 branch. If we decide we like the refactoring changes, we
> can merge those into cclib-0.7 once its stable. What do y'all think?

OK, so tell me which way to go - branch off with the refactoring or create a 
0.7 branch? I would opt for the first, becuase I was really hoping to get the 
refactoring over with quickly and get on to some "real" additions :)

Conceptually, the parsing function structure I proposed is not far from a dict 
of functions, so maybe that would be a good first step to take.

Karol

-- 
written by Karol Langner
Sat Feb  3 10:26:45 CET 2007

Re: [cclib-devel] Refactoring

[Karol L. <kar...@kn...>]

On Friday 02 of February 2007 20:35, Noel O'Boyle wrote:
> Just been reading today that instead of a massive switch statement
> it's better to use a dictionary of functions. It's not exactly the
> same, but it makes me think of something like...
>
> for line in file:
>     for keyphrase,blockname in keyphrases:
>         if line.find(keyphrase)>=0:
>              extract[blockname]()
>
> e.g. keyphrase("Number of MOs") and corresponding blockname is 'MO section'

I thought about this also - extracting a new attribute is then just adding a 
new entry to that dict, very conscise - but this would take some more work 
from where we are (which I'm willign to do). One question remains, where 
would the functions be defined? As methods of the class? As locals in the 
module? Directly inside the dictionary?

There's one more problem, the "keyphrase conditions" are various, some 
conditions check more than just the current line - some have more than one 
possible keyphrase, some have additional conditions. So there would have to 
be another dict of functions that check the conditions specific to the parser 
and attribute. So I thought that that was too much change at one time.

> I'm not sure why you think it should return a tuple though (this makes
> things more difficult). Surely it can just set the attributes from
> instead the extract function.

Not necessarily a tuple, but subscriptable, because some conditional blocks 
extract more than one attribute at a time. Now that I look at it, seems 
superfluous, since the type checks can go after the extracting, and progress 
isn't ever updated twice in the same block.

Karol

-- 
written by Karol Langner
Sat Feb  3 10:08:29 CET 2007

Re: [cclib-devel] Refactoring

[Adam T. <a-t...@st...>]

> Sounds like this is starting to turn into a major refactoring, and I'm
> a bit wary of having this on the trunk, especially as it will be
> experimental for the first while. Can you put this on a branch instead
> and just test with a single parser for the moment?

Karol, it does sound like you are doing some interesting work, but I  
agree with Noel that this should be developed someplace other than  
where our current changes are. Especially since my most recent  
version of PyMOlyze uses trunk as that's where the CDA and  
FragmentAnalysis code is.

What are the current features we are hoping to have in cclib 0.7? If  
there aren't going to be many changes (esp to parsers), I think it  
makes sense to merge trunk (from before the refactoring) into a  
cclib-0.7 branch. If we decide we like the refactoring changes, we  
can merge those into cclib-0.7 once its stable. What do y'all think?

Adam

Re: [cclib-devel] Refactoring

[Noel O'B. <bao...@gm...>]

Sounds interesting. Anything that you feel makes it easier for people
to write parsers is ok with me. :-)

Just been reading today that instead of a massive switch statement
it's better to use a dictionary of functions. It's not exactly the
same, but it makes me think of something like...

for line in file:
    for keyphrase,blockname in keyphrases:
        if line.find(keyphrase)>=0:
             extract[blockname]()

e.g. keyphrase("Number of MOs") and corresponding blockname is 'MO section'

I'm not sure why you think it should return a tuple though (this makes
things more difficult). Surely it can just set the attributes from
instead the extract function.

Sounds like this is starting to turn into a major refactoring, and I'm
a bit wary of having this on the trunk, especially as it will be
experimental for the first while. Can you put this on a branch instead
and just test with a single parser for the moment?

On 02/02/07, Karol Langner <kar...@kn...> wrote:
> I have a more radical idea for refactoring the code for the parsers, which
> could structure them better and make them more 'standardized'. It still won't
> change anything in the user-side, but I really think something like this will
> give alot of long-term benifits in terms of writing new parsers and
> regulating common aspects (like LogFile.progress and attribute types).
>
> The attached file contains my fast scetch of the new parse function (so it
> probaly won't work).
>
> Besides parse() itself:
> 1) the extract methods of all parsers would be revised. This is pretty
> straightforward and I would do it with a script - the loop is cut out, and
> each conditional block  for line content needs to returns a tuple with the
> attributes that were set. If the "_" prefix were to denote private
> attributes, extract should probably be renamed to _extract at this point.
> 2) a few new dictionaries are needed for common information, as used in the
> attached file: _attrtypes (attribute types we want for each attribute),
> _fupdate (for the progress update), _msg (messages for progress). The last
> two should logically be attributes of the progress object itself), and could
> be set to default values by cclib if they don't exist (for compatibility).
>
> And that would be about it! If this all looks like it might work :) and you
> think it's a good idea, I'll go straight ahead and work on the details. I
> don't think it would take alot of work, and I have some spare time this
> weekend.
>
> Karol
>
> --
> written by Karol Langner
> Fri Feb  2 19:00:35 CET 2007
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier.
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>
>
>

Re: [cclib-devel] Refactoring

[Karol L. <kar...@kn...>]

I have a more radical idea for refactoring the code for the parsers, which 
could structure them better and make them more 'standardized'. It still won't 
change anything in the user-side, but I really think something like this will 
give alot of long-term benifits in terms of writing new parsers and 
regulating common aspects (like LogFile.progress and attribute types).

The attached file contains my fast scetch of the new parse function (so it 
probaly won't work).

Besides parse() itself:
1) the extract methods of all parsers would be revised. This is pretty 
straightforward and I would do it with a script - the loop is cut out, and 
each conditional block  for line content needs to returns a tuple with the 
attributes that were set. If the "_" prefix were to denote private 
attributes, extract should probably be renamed to _extract at this point.
2) a few new dictionaries are needed for common information, as used in the 
attached file: _attrtypes (attribute types we want for each attribute), 
_fupdate (for the progress update), _msg (messages for progress). The last 
two should logically be attributes of the progress object itself), and could 
be set to default values by cclib if they don't exist (for compatibility).

And that would be about it! If this all looks like it might work :) and you 
think it's a good idea, I'll go straight ahead and work on the details. I 
don't think it would take alot of work, and I have some spare time this 
weekend.

Karol

-- 
written by Karol Langner
Fri Feb  2 19:00:35 CET 2007

Re: [cclib-devel] Refactoring

[Karol L. <kar...@kn...>]

On Friday 02 of February 2007 12:57, Noel O'Boyle wrote:
> Do they have to be attributes of the class? Can they be local
> variables in parse() which are passed into extract() (or maybe this is
> too messy)? Alternatively, they can be 'weak' attributes, with names
> like self._nstep. AFAIK, this is a convention to indicate 'private'
> variables of a class, although it doesn't do much else.

A made nstep an attribute of LogFile.progress, hope that doesn't conflict with 
your code. After all, it does fit in as an attribute of LogFile.progress.

Karol

-- 
written by Karol Langner
Fri Feb  2 18:47:42 CET 2007

Re: [cclib-devel] Refactoring

[Karol L. <kar...@kn...>]

On Friday 02 of February 2007 17:06, you wrote:
> > Is there a difference in the meaning of the numbers cupdate and
> > fupdate,
> > beyond what I can see in the code?
>
> I think I originally intended them as course and fine update values,
> although I honestly don't recall how they are actually used. I think
> that the progress while parsing mocoeffs didn't update frequently
> enough with cupdate, so I added fupdate---but I may be wrong.
>
> Adam

The function I added to LogFile that does this now takes an argument xupdate, 
so a parser can give any value now (I retained the cupdate/fupdate values 
that were in all parsers. I see that this was used moslty for the Gaussian 
parser up till now.

Karol

-- 
written by Karol Langner
Fri Feb  2 18:23:14 CET 2007

Re: [cclib-devel] Refactoring

[Adam T. <a-t...@st...>]

> Is there a difference in the meaning of the numbers cupdate and  
> fupdate,
> beyond what I can see in the code?

I think I originally intended them as course and fine update values,  
although I honestly don't recall how they are actually used. I think  
that the progress while parsing mocoeffs didn't update frequently  
enough with cupdate, so I added fupdate---but I may be wrong.

Adam

Re: [cclib-devel] Refactoring

[Karol L. <kar...@kn...>]

On Friday 02 of February 2007 12:57, you wrote:
> On 02/02/07, Karol Langner <kar...@kn...> wrote:
> > Thanks... it might be worthwhile to move some more things to
> > LogFile.pares, but I need your input on these.
> >
> > 1)  The stuff before the loop (inputfile.seek, initialize self.progress,
> > etc.). For this, inputfile, nstep, and oldstep would need to be
> > attributes of the class. However, since they are temporary (parsing is
> > never aborted before it's finished, I think), this may not be a good
> > idea.
>
> Do they have to be attributes of the class? Can they be local
> variables in parse() which are passed into extract() (or maybe this is
> too messy)? Alternatively, they can be 'weak' attributes, with names
> like self._nstep. AFAIK, this is a convention to indicate 'private'
> variables of a class, although it doesn't do much else.

Yes, probably better to pass them as arguments, for now at least.

> > 2) The code that uses cupdate, fupdate, nstep, and oldstep (condition "if
> > self.progress and random.random() < fupdate") is repeated multiple times,
> > but the same question arises as the above.
>
> Well, I think that this code can be moved into a function.

Is there a difference in the meaning of the numbers cupdate and fupdate, 
beyond what I can see in the code?

> > Could you perhaps explain the full purpose of LogFile.progress?
>
> It's main purpose is for GUI applications to be able to display a
> progress bar showing how near to completion the parsing is. This is
> because parsing can take more than 20 seconds for large files
> containing population data (which is what both myself and Adam were
> interested in). You should try out PyMOlyze to see this in action.
> There is some cost in seconds in including the progress code, but
> blindingly fast parsing is not the main goal of cclib. For instance,
> it is possible to rewrite our multiple 'if' statements in other forms
> that would parse large files quicker. If you care about this, it might
> be worth thinking about, bearing in mind that "premature optimization
> is the root of all evil" or something.

I don't care about optimization a bit, at least at this point. I do hope, 
though, that some extent of refactoring will make writing new parsers and 
extending the present ones a little easier.

I'm going to try out pyMOlyze and GaussSum soon, when I find a bit more free 
time.

> > Maybe there
> > would be some advantages in making the logfile file object an attribute
> > of LogFile (as in self.intputfile)?
>
> IMO the fewer attributes the better. Since it's a 'derived attribute'
> (i.e. it can be recreated from self.filename), and is only used within
> extract(), I don't see how it can be useful. If you can think of a
> useful reason for doing this though, go ahead.

No, I don't :)

Karol

-- 
written by Karol Langner
Fri Feb  2 16:29:18 CET 2007

Re: [cclib-devel] Refactoring

[Noel O'B. <bao...@gm...>]

On 02/02/07, Karol Langner <kar...@kn...> wrote:
> Thanks... it might be worthwhile to move some more things to LogFile.pares,
> but I need your input on these.
>
> 1)  The stuff before the loop (inputfile.seek, initialize self.progress,
> etc.). For this, inputfile, nstep, and oldstep would need to be attributes of
> the class. However, since they are temporary (parsing is never aborted before
> it's finished, I think), this may not be a good idea.

Do they have to be attributes of the class? Can they be local
variables in parse() which are passed into extract() (or maybe this is
too messy)? Alternatively, they can be 'weak' attributes, with names
like self._nstep. AFAIK, this is a convention to indicate 'private'
variables of a class, although it doesn't do much else.

> 2) The code that uses cupdate, fupdate, nstep, and oldstep (condition "if
> self.progress and random.random() < fupdate") is repeated multiple times, but
> the same question arises as the above.
Well, I think that this code can be moved into a function.

> Could you perhaps explain the full purpose of LogFile.progress?
It's main purpose is for GUI applications to be able to display a
progress bar showing how near to completion the parsing is. This is
because parsing can take more than 20 seconds for large files
containing population data (which is what both myself and Adam were
interested in). You should try out PyMOlyze to see this in action.
There is some cost in seconds in including the progress code, but
blindingly fast parsing is not the main goal of cclib. For instance,
it is possible to rewrite our multiple 'if' statements in other forms
that would parse large files quicker. If you care about this, it might
be worth thinking about, bearing in mind that "premature optimization
is the root of all evil" or something.

> Maybe there
> would be some advantages in making the logfile file object an attribute of
> LogFile (as in self.intputfile)?
IMO the fewer attributes the better. Since it's a 'derived attribute'
(i.e. it can be recreated from self.filename), and is only used within
extract(), I don't see how it can be useful. If you can think of a
useful reason for doing this though, go ahead.

> Karol
>
> --
> written by Karol Langner
> Fri Feb  2 12:21:24 CET 2007
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier.
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>

Re: [cclib-devel] Refactoring

[Karol L. <kar...@kn...>]

On Thursday 01 of February 2007 17:55, Noel O'Boyle wrote:
> Nice work on the refactoring, Karol.

Thanks... it might be worthwhile to move some more things to LogFile.pares, 
but I need your input on these.

1)  The stuff before the loop (inputfile.seek, initialize self.progress, 
etc.). For this, inputfile, nstep, and oldstep would need to be attributes of 
the class. However, since they are temporary (parsing is never aborted before 
it's finished, I think), this may not be a good idea.

2) The code that uses cupdate, fupdate, nstep, and oldstep (condition "if 
self.progress and random.random() < fupdate") is repeated multiple times, but 
the same question arises as the above.

Could you perhaps explain the full purpose of LogFile.progress? Maybe there 
would be some advantages in making the logfile file object an attribute of 
LogFile (as in self.intputfile)?

Karol

-- 
written by Karol Langner
Fri Feb  2 12:21:24 CET 2007

[cclib-devel] Refactoring

[Noel O'B. <bao...@gm...>]

Nice work on the refactoring, Karol.

Re: [cclib-devel] pre/post-parsing function for LogFile

[Karol L. <kar...@kn...>]

On Tuesday 30 of January 2007 17:39, you wrote:
>  I also think ccopen and others
> should check to see if the logfile exists, and if not, print an error/
> warning about it.

I added and IOError exception to the call to openlogfile() in ccopen, which 
returns None.

Karol

-- 
written by Karol Langner
Thu Feb  1 14:35:35 CET 2007

Re: [cclib-devel] Jaguar basis sets

[Adam T. <a-t...@st...>]

Which Jaguar version are you looking at? I think my colleague just  
used the basis functions she normally uses for her calcs when she ran  
the Jaguar 6.0 calcs.

Also, as I recall, the MO coeffs aren't printed for the Jaguar 4.2  
files, but I may be mistaken.

Adam

On Jan 31, 2007, at 6:35 AM, Karol Langner wrote:

> The only thing I can imagine right now is that there is an additional
> contraction, or that the linear combination is summed somewhat  
> differently
> than normal. I don't use the program myself, so I don't have the  
> means to ask
> the Jaguar people, as I understand you must have a liscence to do so.
>
> Karol
>
> On Wednesday 31 of January 2007 15:28, you wrote:
>> Yes, there are problems with gbasis for Jaguar. First I want to  
>> finish
>> volume.py, though. I'll come back to this though and we can talk it
>> through. If you have any ideas in the meantime I'd be happy to hear
>> them.
>>
>> On 31/01/07, Karol Langner <kar...@kn...> wrote:
>>> Hi,
>>>
>>>  I'm wondering about the basis sets output by Jaguar... they have me
>>> stumped, since the contraction coefficients for the SP gaussian
>>> primitives are totally different than those given by GAMESS and  
>>> Gaussian.
>>>
>>>  Compare the STO-3G function for carbon that I found in the cclib  
>>> test
>>> logfiles.
>>>
>>> Gaussian:
>>>  S   3 1.00       0.000000000000
>>>       0.7161683735D+02  0.1543289673D+00
>>>       0.1304509632D+02  0.5353281423D+00
>>>       0.3530512160D+01  0.4446345422D+00
>>>  SP   3 1.00       0.000000000000
>>>       0.2941249355D+01 -0.9996722919D-01  0.1559162750D+00
>>>       0.6834830964D+00  0.3995128261D+00  0.6076837186D+00
>>>       0.2222899159D+00  0.7001154689D+00  0.3919573931D+00
>>>
>>> Jaguar:
>>>              s    j
>>>              h    c  i       n
>>>              e    o  s       f
>>>              l    n  h       s
>>>   atom       l    t  l  l    h          z              coef
>>> rcoef --------    ---  --- -- --  ---     ----------      ----------
>>>  --------- C1          1    3  0  1    0      71.6168373        
>>> 0.1543290
>>>      2.7078144 C1          2   -1  0  1    0      13.0450963
>>> 0.5353281       2.6188802 C1          3   -1  0  1    0        
>>> 3.5305122
>>>     0.4446345       0.8161906 C1          4    2  2  1    1
>>> 2.9412494      -0.2956454      -0.4732386 C1          5   -4  2   
>>> 1    1
>>>     0.6834831       1.1815287       0.6329949 C1          6    2  
>>> -1  2
>>> 2       2.9412494       0.2213487       1.2152952 C1          7    
>>> -6 -1
>>> 2    2       0.6834831       0.8627064       0.7642102  
>>> C1          8    1
>>>  1  1    1       0.2222899       1.0000000       0.2307278  
>>> C1          9
>>>   1 -1  2    2       0.2222899       1.0000000       0.2175654
>>>
>>> The coefficient for the S shells are the same, but the ones for  
>>> SP are
>>> not. Does anybody know how to explain this, and eventually how to
>>> interconvert? I assume that the STO-3G functions used in both  
>>> programs
>>> are identical, so it should be possible. Or, where should I ask  
>>> about
>>> this? Maybe this is the reason why there's no gbasis attribute  
>>> for the
>>> Jaguar parser yet?
>>>
>>> Karol
>
> -- 
> written by Karol Langner
> Wed Jan 31 15:32:18 CET 2007
>
> ---------------------------------------------------------------------- 
> ---
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to  
> share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php? 
> page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel

Re: [cclib-devel] Numeric --> numpy

[Karol L. <kar...@kn...>]

On Friday 26 of January 2007 09:00, Noel O'Boyle wrote:
> Both myself and Adam are also thinking of our own programs, which will
> also need to be adjusted. They will already have to be changed
> slightly to deal with the redefinition of mocoeffs and a couple of
> other attributes.
>
> What this adds up to is that if nobody disagrees, this will be the
> first thing we'll do after the next release.
>
> Noel

One more thing: from Numeric 24.2 onwards, there generally is supposed to be 
no major problems with passing numpy arrays to Numeric. There was a thread 
about this a couple of days ago on the Numpy-discussion mailing list.

I'm not pushing for the conversion to be sooner, just wanted to make sure you 
know that your cclib-dependent programs will probably still work fine after a 
cclib upgrade if they are not upgraded themselves.

Karol

-- 
written by Karol Langner
Wed Jan 31 17:01:12 CET 2007

Re: [cclib-devel] Jaguar basis sets

[Noel O'B. <bao...@gm...>]

The first priority is figuring out the H coefficients...here's the
current situation for STO-3G:

PyQuante

basis_data = [
   14     None,
   15     [#H
   16     ('S',[
   17         (3.425251, 0.154329),
   18         (0.623914, 0.535328),
   19         (0.168855, 0.444635),
   20         ]),
   21     ],

The normalised coefficients for the primitives (each on their own) are:
    0.2769
    0.2678
    0.0835


GAMESS-UK

exponents, then contraction coefficients

 h


                14   1s      19        3.425251       0.276934  (    0.154329  )
                14   1s      20        0.623914       0.267839  (    0.535328  )
                14   1s      21        0.168855       0.083474  (    0.444635  )

Jag 4.2

 H6          1    2  0  1   25       3.4252509       0.2405014       0.4315658
 H6          2   -1  0  1   25       0.6239137       0.8342385       0.4173916
 H6          3    1  0  1   25       0.1688554       1.0000000       0.1877355

Normalised coefficients
 H6          1    S   26       3.425251     0.431566
 H6          2    S   26       0.623914     0.417392
 H6          3    S   26       0.168855     0.187735


GAMESS

H

     13   S      37             3.4252509    0.154328967295
     13   S      38             0.6239137    0.535328142282
     13   S      39             0.1688554    0.444634542185

PC-GAMESS

 THE CONTRACTED PRIMITIVE FUNCTIONS HAVE BEEN UNNORMALIZED
 THE CONTRACTED BASIS FUNCTIONS ARE NOW NORMALIZED TO UNITY

 H

  22   S   31       3.425251    0.276934 (  0.154329)
  22   S   32       0.623914    0.267839 (  0.535328)
  22   S   33       0.168855    0.083474 (  0.444635)


G03

 Atom H7       Shell    12 S   3     bf   27 -    27
-4.468771508880          1.536663462131          0.000000000000
      0.3425250914D+01  0.1543289673D+00
      0.6239137298D+00  0.5353281423D+00
      0.1688554040D+00  0.4446345422D+00

Re: [cclib-devel] Jaguar basis sets

[Karol L. <kar...@kn...>]

The only thing I can imagine right now is that there is an additional 
contraction, or that the linear combination is summed somewhat differently 
than normal. I don't use the program myself, so I don't have the means to ask 
the Jaguar people, as I understand you must have a liscence to do so.

Karol

On Wednesday 31 of January 2007 15:28, you wrote:
> Yes, there are problems with gbasis for Jaguar. First I want to finish
> volume.py, though. I'll come back to this though and we can talk it
> through. If you have any ideas in the meantime I'd be happy to hear
> them.
>
> On 31/01/07, Karol Langner <kar...@kn...> wrote:
> > Hi,
> >
> >  I'm wondering about the basis sets output by Jaguar... they have me
> > stumped, since the contraction coefficients for the SP gaussian
> > primitives are totally different than those given by GAMESS and Gaussian.
> >
> >  Compare the STO-3G function for carbon that I found in the cclib test
> > logfiles.
> >
> > Gaussian:
> >  S   3 1.00       0.000000000000
> >       0.7161683735D+02  0.1543289673D+00
> >       0.1304509632D+02  0.5353281423D+00
> >       0.3530512160D+01  0.4446345422D+00
> >  SP   3 1.00       0.000000000000
> >       0.2941249355D+01 -0.9996722919D-01  0.1559162750D+00
> >       0.6834830964D+00  0.3995128261D+00  0.6076837186D+00
> >       0.2222899159D+00  0.7001154689D+00  0.3919573931D+00
> >
> > Jaguar:
> >              s    j
> >              h    c  i       n
> >              e    o  s       f
> >              l    n  h       s
> >   atom       l    t  l  l    h          z              coef           
> > rcoef --------    ---  --- -- --  ---     ----------      ----------     
> >  --------- C1          1    3  0  1    0      71.6168373       0.1543290 
> >      2.7078144 C1          2   -1  0  1    0      13.0450963      
> > 0.5353281       2.6188802 C1          3   -1  0  1    0       3.5305122  
> >     0.4446345       0.8161906 C1          4    2  2  1    1      
> > 2.9412494      -0.2956454      -0.4732386 C1          5   -4  2  1    1  
> >     0.6834831       1.1815287       0.6329949 C1          6    2 -1  2   
> > 2       2.9412494       0.2213487       1.2152952 C1          7   -6 -1 
> > 2    2       0.6834831       0.8627064       0.7642102 C1          8    1
> >  1  1    1       0.2222899       1.0000000       0.2307278 C1          9 
> >   1 -1  2    2       0.2222899       1.0000000       0.2175654
> >
> > The coefficient for the S shells are the same, but the ones for SP are
> > not. Does anybody know how to explain this, and eventually how to
> > interconvert? I assume that the STO-3G functions used in both programs
> > are identical, so it should be possible. Or, where should I ask about
> > this? Maybe this is the reason why there's no gbasis attribute for the
> > Jaguar parser yet?
> >
> > Karol

-- 
written by Karol Langner
Wed Jan 31 15:32:18 CET 2007