Thread: [cclib-devel] problem with utils.py revision 336

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi,

There is apparently something that got messed up with the file 

src/cclib/parser/utils.py

since the last revision 336

Windows ? ;-)

Mehdi

I guess it's something that has a dependency on qt. It'll probably need a
protected 'import' statement, as in progress/__init__.py. I'll look into it
as soon as I have a chance. You can locally switch to the earlier revision
with something like 'svn update -r335' which will allow you to continue for
the moment

Noel.

On 14/09/06, Mehdi Bounouar <meh...@ch...> wrote:
>
> Hi,
>
> There is apparently something that got messed up with the file
>
> src/cclib/parser/utils.py
>
> since the last revision 336
>
> Windows ? ;-)
>
> Mehdi
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>

What exactly is the problem you've been having? I haven't found any
problems: I've installed the latest SVN on WinXP, and 147 tests pass when I
run testall.py (the failures indicate 'things to do') and 58 tests in
regression.py.

On 14/09/06, Noel O'Boyle <bao...@gm...> wrote:
>
> I guess it's something that has a dependency on qt. It'll probably need a
> protected 'import' statement, as in progress/__init__.py. I'll look into it
> as soon as I have a chance. You can locally switch to the earlier revision
> with something like 'svn update -r335' which will allow you to continue for
> the moment
>
> Noel.
>
>
> On 14/09/06, Mehdi Bounouar <meh...@ch...> wrote:
> >
> > Hi,
> >
> > There is apparently something that got messed up with the file
> >
> > src/cclib/parser/utils.py
> >
> > since the last revision 336
> >
> > Windows ? ;-)
> >
> > Mehdi
> >
> > -------------------------------------------------------------------------
> >
> > Using Tomcat but need to do more? Need to support web services,
> > security?
> > Get stuff done quickly with pre-integrated technology to make your job
> > easier
> > Download IBM WebSphere Application Server v.1.0.1 based on Apache
> > Geronimo
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > cclib-devel mailing list
> > ccl...@li...
> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
> >
>
>

> I guess it's something that has a dependency on qt. It'll probably  
> need a protected 'import' statement, as in progress/__init__.py.  
> I'll look into it as soon as I have a chance. You can locally  
> switch to the earlier revision with something like 'svn update - 
> r335' which will allow you to continue for the moment

The current version of __init__.py *should* only load the qt-based  
modules if qt or PyQt4 has already been loaded. I have yet to test on  
a machine without qt/PyQt4.

As far as utils.py goes, I modified ccopen to allow optional progress  
classes and logging stuff to it in addition to the filename. I ran  
testall.py, and I didn't get any extra errors.

Adam

It might be something to do with SVN. In Windows, SVN adds ^M to everything
(i.e. DOS format). In Linux, it doesn't. But, if you're using cygwin's SVN
on Windows, and then editing with IDLE, it may get weird depending on the
combinations of settings you use.

?

On 14/09/06, Mehdi Bounouar <meh...@ch...> wrote:
>
> On Thu, Sep 14, 2006 at 09:55:30AM +0100, Noel O'Boyle wrote:
> > What exactly is the problem you've been having? I haven't found any
> > problems: I've installed the latest SVN on WinXP, and 147 tests pass
> when I
> > run testall.py (the failures indicate 'things to do') and 58 tests in
> > regression.py.
>
> Hum, then I must have done something. The thing is that the file was in
> DOS format with these "^M" carriage returns. So I don't know what
> happened.
>
> BTW I hope to upload as soon as possible the Molpro parser :-) it is
> written and works (at least for what I in general need) I just have to
> get more confident using svn and generate the data examples.
>
> Mehdi
>

Hello Mehdi,

Sorry for the delay in looking at your files. I didn't realise you had so
much done! It looks very good. Can you get all of this onto subversion as
soon as possible? This will make it easier for us to help you, and to get
the tests set up and so on. Just do a 'svn add' for every file you want to
upload, and then either do a single 'svn commit' for all of the files, or
individual svn commits for every file. I can do it for you if you like, if
you send me the latest copy of your code.

Once these are uploaded, I will modify the tests to include Molpro so that
we can work on making them all pass.

I'm not sure whether we want to keep the .molden files though. These files
duplicate the information in the .log files, isn't this correct?

Regards,
   Noel

On 14/09/06, Mehdi Bounouar <meh...@ch...> wrote:
>
> Hi,
>
> Before doing any major .... up with subversion I just send you the what
> I did for Molpro parser, also to have an opinion.
>
> I did not extend the tests but at least cclib
> is not broken :-)
>
> Mehdi
>
>
>

Hi,

> Sorry for the delay in looking at your files.

I'm sorry too for not submitting yet, as did have much time to do that,
plus I had several problems making these tests examples.

> Can you get all of this onto subversion as soon as possible? This will
> make it easier for us to help you, and to get the tests set up and so
> on. Just do a 'svn add' for every file you want to upload, and then
> either do a single 'svn commit' for all of the files, or individual
> svn commits for every file. I can do it for you if you like, if you
> send me the latest copy of your code.

I hope to dot it myself still today but latter, as I finally managed to
get Molpro run the test job on divinyl-benzene.

> I'm not sure whether we want to keep the .molden files though. These
> files duplicate the information in the .log files, isn't this correct?

More or less, I don't know how familiar you are with Molpro but it is
almost a script language by itself which means that one have a lot of
control (I not advertising for them) to essentially do everything what
one would like to do with the data but on the other hand hides a lot of
info unless specified, making it thus difficult to parse, so dumping a
molden file is a way to check things, and harmless, untill they get too
big to be opened by Molden :-), which happens often as Molpro puts
almost everything you did'nt need in.

In the same line of thought I wonder why nobody included the .dat files
for the Gamess-US jobs, they also could be parsed and more easily.

Regards,

Mehdi

>
>
> > I'm not sure whether we want to keep the .molden files though. These
> > files duplicate the information in the .log files, isn't this correct?
>
> More or less, I don't know how familiar you are with Molpro but it is
> almost a script language by itself which means that one have a lot of
> control (I not advertising for them) to essentially do everything what
> one would like to do with the data but on the other hand hides a lot of
> info unless specified, making it thus difficult to parse, so dumping a
> molden file is a way to check things, and harmless, untill they get too
> big to be opened by Molden :-), which happens often as Molpro puts
> almost everything you did'nt need in.

I am not at all familiar with Molpro, but for all of the other programs we
have a similar problem. You need to include various extra keywords to get
the information written to the logfile. Please try to parse everything from
the logfile. Whatever extra keywords are required should be entered on the
wiki.

In the same line of thought I wonder why nobody included the .dat files
> for the Gamess-US jobs, they also could be parsed and more easily.

Or the Gaussian checkpoint file...

This is what cclib does; it parses the free-format output files. Typically
more documentation is available for the free-format output files, more
information is contained in them (although additional keywords may be
required), and most people only retain these as they are smaller. I take
your point, but this will require two parsers for every computational
chemistry program. While this is not a bad idea, I would prefer to
concentrate on getting more programs in cclib and writing more algorithms.
You are welcome to write two parsers, but I would really like you to focus
on the free-format output file as being the more useful (though more
difficult).

It could be that I have a bias towards the free-format output files as the
formatted output files typically do not contain information on the progress
of geometry optimisations or the convergence of the SCF. I find this
information very useful for large systems to assess whether the system is
likely to converge with the chosen convergence method (this is used by
GaussSum).

Regards,
 Noel

I've see you've just imported the Molpro parser and associated data. That's
great.

I've added it to testGeoOpt. I'm about to change one thing: "au_to_cm-1"
should be "hartree_to_cm-1", I think, to be consistent with the other
conversions.

Could you update the wiki with information on the hessian, so that we can
add it to the other parsers. We will also (at some point) need to write a
unit test (similar to testgbasis.py) for the hessian, so that all of the
parsers agree.

Regards,
  Noel

On 25/09/06, Noel O'Boyle <bao...@gm...> wrote:
>
>
> > > I'm not sure whether we want to keep the .molden files though. These
> > > files duplicate the information in the .log files, isn't this correct?
> >
> >
> > More or less, I don't know how familiar you are with Molpro but it is
> > almost a script language by itself which means that one have a lot of
> > control (I not advertising for them) to essentially do everything what
> > one would like to do with the data but on the other hand hides a lot of
> > info unless specified, making it thus difficult to parse, so dumping a
> > molden file is a way to check things, and harmless, untill they get too
> > big to be opened by Molden :-), which happens often as Molpro puts
> > almost everything you did'nt need in.
>
>
>
> I am not at all familiar with Molpro, but for all of the other programs we
> have a similar problem. You need to include various extra keywords to get
> the information written to the logfile. Please try to parse everything from
> the logfile. Whatever extra keywords are required should be entered on the
> wiki.
>
> In the same line of thought I wonder why nobody included the .dat files
> > for the Gamess-US jobs, they also could be parsed and more easily.
>
>
> Or the Gaussian checkpoint file...
>
> This is what cclib does; it parses the free-format output files. Typically
> more documentation is available for the free-format output files, more
> information is contained in them (although additional keywords may be
> required), and most people only retain these as they are smaller. I take
> your point, but this will require two parsers for every computational
> chemistry program. While this is not a bad idea, I would prefer to
> concentrate on getting more programs in cclib and writing more algorithms.
> You are welcome to write two parsers, but I would really like you to focus
> on the free-format output file as being the more useful (though more
> difficult).
>
> It could be that I have a bias towards the free-format output files as the
> formatted output files typically do not contain information on the progress
> of geometry optimisations or the convergence of the SCF. I find this
> information very useful for large systems to assess whether the system is
> likely to converge with the chosen convergence method (this is used by
> GaussSum).
>
> Regards,
>  Noel
>

On Mon, Sep 25, 2006 at 01:55:19PM +0100, Noel O'Boyle wrote:
> I've see you've just imported the Molpro parser and associated data. That's
> great.

I'm guilty

> I've added it to testGeoOpt. I'm about to change one thing: "au_to_cm-1"
> should be "hartree_to_cm-1", I think, to be consistent with the other
> conversions.

that's ok 

> Could you update the wiki with information on the hessian, so that we can
> add it to the other parsers. We will also (at some point) need to write a
> unit test (similar to testgbasis.py) for the hessian, so that all of the
> parsers agree.

For the hessian only the lower triangle part is stored so one would need
an LowerTriangleArrayToMatrix function i.e.:  [a11, a12, a22, a13,..., ann] to
corresponding Matrix, so one can effectively work with. I will probably
put it in units.py (it is already written).

For the Wiki I have to put for the hessian as array of rank 1, as it is
sometimes easier to work with.

Mehdi

> > Could you update the wiki with information on the hessian, so that we
> can
> > add it to the other parsers. We will also (at some point) need to write
> a
> > unit test (similar to testgbasis.py) for the hessian, so that all of the
> > parsers agree.
>
> For the hessian only the lower triangle part is stored so one would need
> an LowerTriangleArrayToMatrix function i.e.:  [a11, a12, a22, a13,...,
> ann] to
> corresponding Matrix, so one can effectively work with. I will probably
> put it in units.py (it is already written).

It's up to you. I usually just fill in a full matrix as I read the data, e.g.
data[i,j] = data[j,i] = myfloat

For the Wiki I have to put for the hessian as array of rank 1, as it is
> sometimes easier to work with.

Wouldn't it be better as a rank 2 array. If you need to flatten it for
convenience, you can just use myarray.flat().

Mehdi
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys -- and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>

Sorry - I should have pointed you to the development pages on the wiki. The
"parsed data" you edited was information on the release. I've made the
necessary changes. The correct page is linked to from the "Development" link
in the side-bar on the wiki.

There's a couple of other things I was wondering about: (I know this is
being pedantic, but we want to get all of this straight from the start,
before we have 6 parsers doing slightly different things)
(1) Is mass-weighted better than the alternative (mass-unweighted or
something)? I know we can compute one from the other (at least, I hope so!),
but which one should we extract?
(2) What about geometry optimisations? Are we only going to extract the
hessian from a frequency calculation?
(3) What are the dimensions of the hessian? Can you discuss this some more
on
http://cclib.sourceforge.net/wiki/index.php?title=Hessian&action=edit

Regards,
  Noel

On 25/09/06, Noel O'Boyle <bao...@gm...> wrote:
>
>
> > Could you update the wiki with information on the hessian, so that we
> > can
> > > add it to the other parsers. We will also (at some point) need to
> > write a
> > > unit test (similar to testgbasis.py) for the hessian, so that all of
> > the
> > > parsers agree.
> >
> > For the hessian only the lower triangle part is stored so one would need
> > an LowerTriangleArrayToMatrix function i.e.:  [a11, a12, a22, a13,...,
> > ann] to
> > corresponding Matrix, so one can effectively work with. I will probably
> > put it in units.py (it is already written).
>
>
> It's up to you. I usually just fill in a full matrix as I read the data,
> e.g. data[i,j] = data[j,i] = myfloat
>
> For the Wiki I have to put for the hessian as array of rank 1, as it is
> > sometimes easier to work with.
>
>
> Wouldn't it be better as a rank 2 array. If you need to flatten it for
> convenience, you can just use myarray.flat().
>
> Mehdi
> >
> >
> > -------------------------------------------------------------------------
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share
> > your
> > opinions on IT & business topics through brief surveys -- and earn cash
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> >
> > _______________________________________________
> > cclib-devel mailing list
> > ccl...@li...
> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
> >
>
>

> There's a couple of other things I was wondering about: (I know this is
> being pedantic, but we want to get all of this straight from the start,
> before we have 6 parsers doing slightly different things)
> (1) Is mass-weighted better than the alternative (mass-unweighted or
> something)? I know we can compute one from the other (at least, I hope so!),
> but which one should we extract?

Mass-unweighted is generally prefered, (although in this case I parsed
the  weighted one, I will change this) the reason is that then one can
specify different masses and get frequencies and normal modes of for
example the deuteraded species.

> (2) What about geometry optimisations? Are we only going to extract the
> hessian from a frequency calculation?

Extracting the hessian from a geometry optimisation is in general
pointless (and not printed out in most cases) it is anyway an updated
one which is very dependent on the your procedure, what is more
important in my opinion, is to extract the gradient, also if possible
from a geometry optimisation, this is a lot of information about your
PES that should not be thrown away and could/is useful, at least in my
opinion.

> (3) What are the dimensions of the hessian? Can you discuss this some more

I have done a few updates 

Cheers,

Mehdi