cclib-devel — Developer's mailing list

Re: [cclib-devel] A mail stuck on hold

[Martin R. <mar...@gm...>]

Hi Adam,

Ok, that's good and bad news then! Glad you figured it out!

And sure, I relenquish any copyright. It's only the nitrogen atom :)

Best,
Martin


On Sun, Sep 14, 2014 at 12:24 PM, Adam Tenderholt <ate...@gm...>
wrote:

> Hi Martin,
>
> Sorry that you’re having problems. It seems that you’ve found a bug, and
> we’d probably like to include it with our regression tests. Are you willing
> to place your output file in the public domain (i.e. release your
> copyright)?
>
> Also, the bug stems from the fact that we expect the beta set to be
> immediately after the alpha set. In your file, the LZ Value analysis is
> printed in between. Simply removing that seems to fix the problem.
>
> Adam
>
>
>
> On Sun, Sep 14, 2014 at 8:13 AM, Martin Rahm <mar...@gm...>
> wrote:
>
>> Hi Adam,
>>
>> I'm writing you since I suspect you're one of the moderators of cclib. I
>> have a peculiar problem, and posted the below message four days ago. It
>> hasn't reached the list however.
>>
>> Thanks,
>> Martin
>>
>>  Hi,
>>
>> I am trying a script that is working fine with both gaussian and orca
>> output. I wish to print out the alpha and beta orbital energies of the
>> nitrogen atom (here in the quartet ground state). The script prints out
>> beta instead of alpha, and then returns "IndexError: list index out of
>> range".
>>
>> I hope this is something stupefyingly simple, and would really appreciate
>> the help! The output file is attached if anyone would like to give it a
>> try.
>>
>> Many thanks,
>> Martin
>>
>> My script:
>>  ---------------
>> from numpy import *
>> from cclib.parser import ccopen
>> import cclib
>> import sys
>>
>> inputfile = sys.argv
>> data = ccopen(inputfile)
>> data = data.parse()
>>
>> print "alpha:"
>> print data.moenergies[0]
>> print "beta:"
>> print data.moenergies[1]
>> print "done"
>>  ------------
>>
>> My output:
>> --------------
>>  [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute
>> atomcoords[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute atomnos[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute gbasis[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute nbasis:
>> 20
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute charge: 0
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute mult: 4
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute homos[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute natom: 1
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute
>> scftargets[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute
>> scfvalues[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute
>> scfenergies[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute
>> moenergies[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute mosyms[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute nmo: 20
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute
>> mocoeffs[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute
>> atombasis[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute aonames[]
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute
>> atomcharges: {}
>> [GAMESS ['./analyze.py', 'N-d-UHF.out'] INFO] Creating attribute
>> coreelectrons[]
>> alpha:
>> [-423.94520709  -19.75274401    3.43679786    3.43679786    3.43679786
>>    15.00163627   15.19755824   15.19755824   15.19755824   59.89225728
>>    59.89225728   59.89225728   61.45146962   68.88017758   68.88017758
>>    68.88017758   68.88017758   68.88017758  162.94721266  976.48325392]
>> beta:
>> Traceback (most recent call last):
>>   File "./analyze.py", line 21, in <module>
>>     print data.moenergies[1]
>> IndexError: list index out of range
>>  -----------
>>
>>
>>  <N-UHF.out>
>
>
>

[cclib-devel] QChem parser added

[Karol L. <kar...@gm...>]

Greetings cclibers,

Today I merged the QChem parser into master. Thank you, Eric, for the great
code! I also updated the docs at cclib.github.io to reflect this (in the
development sections).

I think now would be a good time start thinking about releasing the next
version. We now have three new parsers since 1.2 (NWChem and Psi4 in
addition to QChem), and they all seem to be in pretty good shape. I added a
milestone with a due date of Sep 30, but that will probably be to soon.

Cheers,
Karol

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol L. <kar...@gm...>]

I created an issue for this in cclib for future work:
https://github.com/cclib/cclib/issues/120


On Tue, Aug 19, 2014 at 5:06 PM, Karol M. Langner <kar...@gm...>
wrote:

> All good counterpoints, the usual show stoppers in this context. And I like
> the idea of gathering the tools for reading binary files in a separate code
> base, perhaps using/requiring data from the logfile, because cclib is now
> somewhat mature and has a well-defined scope. Still, we could keep it
> within the cclib project on github, and call it cclib-binary or whatever.
>
> @Noel @Adam -- any thoughts on this topic?
>
> On Aug 19 2014, Martin Blood-Forsythe wrote:
> > 1. Yes it is true that most codes delete their binary files unless you
> tell
> > them not to.  Which is why I was mentioning the need to be careful about
> > checking for the existence of a file. ORCA is actually a bit unusual in
> the
> > regard that it always leaves the *.gbw binary file around.  I don't know
> > the *entire* contents of the file, but I know how to parse the MO
> > coefficients/energies, occupation numbers, job name, and perhaps orbital
> > labels (1pz 2s etc, the offsets to these are a little tricky since it is
> a
> > mixed binary/ascii format). As far as I know, the *.hess file is a text
> > file, but the point about job specific binaries, like the *.cis file, is
> > certainly correct.
> >
> > 2 . With regard to Q-Chem's custom binary files:
> > There are two different categories of binary files.  The ones that get
> > saved if you do 'qchem infile outfile savename' and the more complete set
> > that are stored if you do the '-save' option.
> >  For the most part I think that the standard ones saved without the
> '-save'
> > command are pretty safe to trust. These include the 53.0 [MO
> coefficients +
> > eigenvalues] and 54.0 [density matrix].  There are a few other binary
> files
> > that I trust: 320.0 [overlap matrix], 601.0 [NBO coefficients].
> > The general point though that binary files are a pain and can require
> some
> > developer level knowledge is well taken. One reason that I thought adding
> > binary parsing support might be neat is exactly because non-developers
> have
> > no idea what is in these files.
> >
> > 2b. The point about developers misusing the binary files in Q-chem is
> > certainly a good one, and is a strong reason to not support parsing any
> of
> > the non-vanilla binary files.  I'd argue that the 53.0 and 54.0 files are
> > quite safe to parse since they have a very well defined structure that
> > hasn't changed across versions 3 - 4.2 as far as I can tell.  Their
> content
> > mapping is very easy and only requires knowing the total number of
> orbitals.
> >
> > 3. I agree that any package that attempts to unpack a whole bunch of
> other
> > binary files should probably be separate from cclib.  Mainly I was
> > wondering if the MO coefficients & eigenvalues would be of interest for
> > visualization purposes.  The tricky part there is that a lot of codes use
> > non-standard basis set definitions that differ from the EMSL
> specifications
> > of the same name.
> >
> > 4. My scripts that unpack *.gbw and Q-Chem binaries are currently wrapped
> > up in with some other code.  I strip them out and add them to
> > https://github.com/mbloodforsythe/qc-binary-parsers
> >
> >
> > ---------------------------------------------------
> > Martin Blood-Forsythe
> > Graduate Student in Physics
> > Harvard University
> > Aspuru-Guzik Lab
> >
> >
> > On Tue, Aug 19, 2014 at 12:57 PM, Eric Berquist <er...@pi...> wrote:
> >
> > > This would certainly be interesting, but I have a few comments that are
> > > probably Q-Chem specific:
> > >
> > > 1. As Martin already mentioned, unless you modify the `qchem` driver
> > > script or you pass the flag `-save` when running, the folder with all
> of
> > > these binary files gets deleted. This means they usually aren't
> present,
> > > except when you want them, in contrast to the regular plain-text output
> > > file which is always produced. This is in contrast to something like
> the
> > > HESS file, which is only present when performing an analytic Hessian
> > > calculation. ORCA is even worse about this, since it deletes
> scratch/temp
> > > files early and often.
> > >
> > > 2. Since almost all of these files are named with a number, anyone who
> is
> > > not a Q-Chem developer probably doesn't know what's in these files. For
> > > example, 53 corresponds to the MO coefficients. Even knowing the
> > > number/content mapping doesn't easily tell you the dimensionality of
> the
> > > file; you have to know what all of the reads and writes to that file
> look
> > > like.
> > >
> > > 2b. Once you know the number/content mapping and the dimensionality,
> there
> > > is still no guarantee about what's in the file, due to nasty bits in
> the
> > > source code with people using files for things other than what they are
> > > specified as containing.
> > >
> > > If one wanted to make a Q-Chem-specific parser for a bunch of binary
> > > files, it would need to be maintained by someone who is a developer and
> > > knows where all of the reads/writes to that file are, but even then
> it's
> > > quite risky. You just can't perform a visual spot-check on them to see
> if
> > > there are problems. It's definitely functionality for expert users.
> > >
> > > It seems as though this is something that should either be in a
> separate
> > > package that depends on cclib, or at least is separate from the
> parsers/
> > > directory.
> > >
> > > Martin, do you have scripts for parsing any of these that are publicly
> > > viewable? I'd be interested to see how you do it. I'd really love to
> know
> > > the structure of ORCA's *.gbw files.
> > >
> > > Eric
> > >
> > >
> > > On Tue, Aug 19, 2014 at 8:21 AM, Karol M. Langner <
> kar...@gm...
> > > > wrote:
> > >
> > >> Here is the issue I created for this:
> > >> https://github.com/cclib/cclib/issues/120
> > >>
> > >> On Aug 18 2014, Karol M. Langner wrote:
> > >> > Hi Martin,
> > >> >
> > >> > That's true, technically it is not hard. I was thinking more about
> > >> integration
> > >> > with the rest of cclib. Binary files will not be "parsed" in the
> sense
> > >> logfiles
> > >> > are, that is the task is not one of reading a text file line by
> line and
> > >> > extracting useful information. Rather, it is one of reading data
> that
> > >> is stored
> > >> > in a predefined format. This is obvious, but it implies some design
> > >> choices
> > >> > that have not been discussed here. Above all, the line-by-line
> paradigm
> > >> > used in the class LogFile (inherited by all parsers) is not useful
> and
> > >> > we cannot just pass one extra file.
> > >> >
> > >> > Anyway, this can definitely be done, I'm just saying it needs a
> little
> > >> > bit of forethought.
> > >> >
> > >> > Cheers,
> > >> > Karol
> > >> >
> > >> > On Aug 18 2014, Martin Blood-Forsythe wrote:
> > >> > > I'll have to look at the way that the multiple file parse is
> > >> implemented
> > >> > > for Molpro, but at least for ORCA and Q-Chem adding a binary file
> > >> parser
> > >> > > wouldn't require much different than the way a text based multiple
> > >> file
> > >> > > parsing works. I've had a lot of success parsing these files using
> > >> > > numpy.fromfile and numpy.frombuffer.  Typically the only required
> > >> knowledge
> > >> > > is the number of orbitals (to determine the array shape to dump
> into)
> > >> and
> > >> > > the file name.
> > >> > >
> > >> > > -Martin
> > >>
> > >> --
> > >> written by Karol M. Langner
> > >> Tue Aug 19 08:20:53 EDT 2014
> > >>
> > >
> > >
>
> --
> written by Karol M. Langner
> Tue Aug 19 17:00:06 EDT 2014
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

All good counterpoints, the usual show stoppers in this context. And I like
the idea of gathering the tools for reading binary files in a separate code
base, perhaps using/requiring data from the logfile, because cclib is now
somewhat mature and has a well-defined scope. Still, we could keep it
within the cclib project on github, and call it cclib-binary or whatever.

@Noel @Adam -- any thoughts on this topic?

On Aug 19 2014, Martin Blood-Forsythe wrote:
> 1. Yes it is true that most codes delete their binary files unless you tell
> them not to.  Which is why I was mentioning the need to be careful about
> checking for the existence of a file. ORCA is actually a bit unusual in the
> regard that it always leaves the *.gbw binary file around.  I don't know
> the *entire* contents of the file, but I know how to parse the MO
> coefficients/energies, occupation numbers, job name, and perhaps orbital
> labels (1pz 2s etc, the offsets to these are a little tricky since it is a
> mixed binary/ascii format). As far as I know, the *.hess file is a text
> file, but the point about job specific binaries, like the *.cis file, is
> certainly correct.
> 
> 2 . With regard to Q-Chem's custom binary files:
> There are two different categories of binary files.  The ones that get
> saved if you do 'qchem infile outfile savename' and the more complete set
> that are stored if you do the '-save' option.
>  For the most part I think that the standard ones saved without the '-save'
> command are pretty safe to trust. These include the 53.0 [MO coefficients +
> eigenvalues] and 54.0 [density matrix].  There are a few other binary files
> that I trust: 320.0 [overlap matrix], 601.0 [NBO coefficients].
> The general point though that binary files are a pain and can require some
> developer level knowledge is well taken. One reason that I thought adding
> binary parsing support might be neat is exactly because non-developers have
> no idea what is in these files.
> 
> 2b. The point about developers misusing the binary files in Q-chem is
> certainly a good one, and is a strong reason to not support parsing any of
> the non-vanilla binary files.  I'd argue that the 53.0 and 54.0 files are
> quite safe to parse since they have a very well defined structure that
> hasn't changed across versions 3 - 4.2 as far as I can tell.  Their content
> mapping is very easy and only requires knowing the total number of orbitals.
> 
> 3. I agree that any package that attempts to unpack a whole bunch of other
> binary files should probably be separate from cclib.  Mainly I was
> wondering if the MO coefficients & eigenvalues would be of interest for
> visualization purposes.  The tricky part there is that a lot of codes use
> non-standard basis set definitions that differ from the EMSL specifications
> of the same name.
> 
> 4. My scripts that unpack *.gbw and Q-Chem binaries are currently wrapped
> up in with some other code.  I strip them out and add them to
> https://github.com/mbloodforsythe/qc-binary-parsers
> 
> 
> ---------------------------------------------------
> Martin Blood-Forsythe
> Graduate Student in Physics
> Harvard University
> Aspuru-Guzik Lab
> 
> 
> On Tue, Aug 19, 2014 at 12:57 PM, Eric Berquist <er...@pi...> wrote:
> 
> > This would certainly be interesting, but I have a few comments that are
> > probably Q-Chem specific:
> >
> > 1. As Martin already mentioned, unless you modify the `qchem` driver
> > script or you pass the flag `-save` when running, the folder with all of
> > these binary files gets deleted. This means they usually aren't present,
> > except when you want them, in contrast to the regular plain-text output
> > file which is always produced. This is in contrast to something like the
> > HESS file, which is only present when performing an analytic Hessian
> > calculation. ORCA is even worse about this, since it deletes scratch/temp
> > files early and often.
> >
> > 2. Since almost all of these files are named with a number, anyone who is
> > not a Q-Chem developer probably doesn't know what's in these files. For
> > example, 53 corresponds to the MO coefficients. Even knowing the
> > number/content mapping doesn't easily tell you the dimensionality of the
> > file; you have to know what all of the reads and writes to that file look
> > like.
> >
> > 2b. Once you know the number/content mapping and the dimensionality, there
> > is still no guarantee about what's in the file, due to nasty bits in the
> > source code with people using files for things other than what they are
> > specified as containing.
> >
> > If one wanted to make a Q-Chem-specific parser for a bunch of binary
> > files, it would need to be maintained by someone who is a developer and
> > knows where all of the reads/writes to that file are, but even then it's
> > quite risky. You just can't perform a visual spot-check on them to see if
> > there are problems. It's definitely functionality for expert users.
> >
> > It seems as though this is something that should either be in a separate
> > package that depends on cclib, or at least is separate from the parsers/
> > directory.
> >
> > Martin, do you have scripts for parsing any of these that are publicly
> > viewable? I'd be interested to see how you do it. I'd really love to know
> > the structure of ORCA's *.gbw files.
> >
> > Eric
> >
> >
> > On Tue, Aug 19, 2014 at 8:21 AM, Karol M. Langner <kar...@gm...
> > > wrote:
> >
> >> Here is the issue I created for this:
> >> https://github.com/cclib/cclib/issues/120
> >>
> >> On Aug 18 2014, Karol M. Langner wrote:
> >> > Hi Martin,
> >> >
> >> > That's true, technically it is not hard. I was thinking more about
> >> integration
> >> > with the rest of cclib. Binary files will not be "parsed" in the sense
> >> logfiles
> >> > are, that is the task is not one of reading a text file line by line and
> >> > extracting useful information. Rather, it is one of reading data that
> >> is stored
> >> > in a predefined format. This is obvious, but it implies some design
> >> choices
> >> > that have not been discussed here. Above all, the line-by-line paradigm
> >> > used in the class LogFile (inherited by all parsers) is not useful and
> >> > we cannot just pass one extra file.
> >> >
> >> > Anyway, this can definitely be done, I'm just saying it needs a little
> >> > bit of forethought.
> >> >
> >> > Cheers,
> >> > Karol
> >> >
> >> > On Aug 18 2014, Martin Blood-Forsythe wrote:
> >> > > I'll have to look at the way that the multiple file parse is
> >> implemented
> >> > > for Molpro, but at least for ORCA and Q-Chem adding a binary file
> >> parser
> >> > > wouldn't require much different than the way a text based multiple
> >> file
> >> > > parsing works. I've had a lot of success parsing these files using
> >> > > numpy.fromfile and numpy.frombuffer.  Typically the only required
> >> knowledge
> >> > > is the number of orbitals (to determine the array shape to dump into)
> >> and
> >> > > the file name.
> >> > >
> >> > > -Martin
> >>
> >> --
> >> written by Karol M. Langner
> >> Tue Aug 19 08:20:53 EDT 2014
> >>
> >
> >

-- 
written by Karol M. Langner
Tue Aug 19 17:00:06 EDT 2014

Re: [cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

1. Yes it is true that most codes delete their binary files unless you tell
them not to.  Which is why I was mentioning the need to be careful about
checking for the existence of a file. ORCA is actually a bit unusual in the
regard that it always leaves the *.gbw binary file around.  I don't know
the *entire* contents of the file, but I know how to parse the MO
coefficients/energies, occupation numbers, job name, and perhaps orbital
labels (1pz 2s etc, the offsets to these are a little tricky since it is a
mixed binary/ascii format). As far as I know, the *.hess file is a text
file, but the point about job specific binaries, like the *.cis file, is
certainly correct.

2 . With regard to Q-Chem's custom binary files:
There are two different categories of binary files.  The ones that get
saved if you do 'qchem infile outfile savename' and the more complete set
that are stored if you do the '-save' option.
 For the most part I think that the standard ones saved without the '-save'
command are pretty safe to trust. These include the 53.0 [MO coefficients +
eigenvalues] and 54.0 [density matrix].  There are a few other binary files
that I trust: 320.0 [overlap matrix], 601.0 [NBO coefficients].
The general point though that binary files are a pain and can require some
developer level knowledge is well taken. One reason that I thought adding
binary parsing support might be neat is exactly because non-developers have
no idea what is in these files.

2b. The point about developers misusing the binary files in Q-chem is
certainly a good one, and is a strong reason to not support parsing any of
the non-vanilla binary files.  I'd argue that the 53.0 and 54.0 files are
quite safe to parse since they have a very well defined structure that
hasn't changed across versions 3 - 4.2 as far as I can tell.  Their content
mapping is very easy and only requires knowing the total number of orbitals.

3. I agree that any package that attempts to unpack a whole bunch of other
binary files should probably be separate from cclib.  Mainly I was
wondering if the MO coefficients & eigenvalues would be of interest for
visualization purposes.  The tricky part there is that a lot of codes use
non-standard basis set definitions that differ from the EMSL specifications
of the same name.

4. My scripts that unpack *.gbw and Q-Chem binaries are currently wrapped
up in with some other code.  I strip them out and add them to
https://github.com/mbloodforsythe/qc-binary-parsers


---------------------------------------------------
Martin Blood-Forsythe
Graduate Student in Physics
Harvard University
Aspuru-Guzik Lab


On Tue, Aug 19, 2014 at 12:57 PM, Eric Berquist <er...@pi...> wrote:

> This would certainly be interesting, but I have a few comments that are
> probably Q-Chem specific:
>
> 1. As Martin already mentioned, unless you modify the `qchem` driver
> script or you pass the flag `-save` when running, the folder with all of
> these binary files gets deleted. This means they usually aren't present,
> except when you want them, in contrast to the regular plain-text output
> file which is always produced. This is in contrast to something like the
> HESS file, which is only present when performing an analytic Hessian
> calculation. ORCA is even worse about this, since it deletes scratch/temp
> files early and often.
>
> 2. Since almost all of these files are named with a number, anyone who is
> not a Q-Chem developer probably doesn't know what's in these files. For
> example, 53 corresponds to the MO coefficients. Even knowing the
> number/content mapping doesn't easily tell you the dimensionality of the
> file; you have to know what all of the reads and writes to that file look
> like.
>
> 2b. Once you know the number/content mapping and the dimensionality, there
> is still no guarantee about what's in the file, due to nasty bits in the
> source code with people using files for things other than what they are
> specified as containing.
>
> If one wanted to make a Q-Chem-specific parser for a bunch of binary
> files, it would need to be maintained by someone who is a developer and
> knows where all of the reads/writes to that file are, but even then it's
> quite risky. You just can't perform a visual spot-check on them to see if
> there are problems. It's definitely functionality for expert users.
>
> It seems as though this is something that should either be in a separate
> package that depends on cclib, or at least is separate from the parsers/
> directory.
>
> Martin, do you have scripts for parsing any of these that are publicly
> viewable? I'd be interested to see how you do it. I'd really love to know
> the structure of ORCA's *.gbw files.
>
> Eric
>
>
> On Tue, Aug 19, 2014 at 8:21 AM, Karol M. Langner <kar...@gm...
> > wrote:
>
>> Here is the issue I created for this:
>> https://github.com/cclib/cclib/issues/120
>>
>> On Aug 18 2014, Karol M. Langner wrote:
>> > Hi Martin,
>> >
>> > That's true, technically it is not hard. I was thinking more about
>> integration
>> > with the rest of cclib. Binary files will not be "parsed" in the sense
>> logfiles
>> > are, that is the task is not one of reading a text file line by line and
>> > extracting useful information. Rather, it is one of reading data that
>> is stored
>> > in a predefined format. This is obvious, but it implies some design
>> choices
>> > that have not been discussed here. Above all, the line-by-line paradigm
>> > used in the class LogFile (inherited by all parsers) is not useful and
>> > we cannot just pass one extra file.
>> >
>> > Anyway, this can definitely be done, I'm just saying it needs a little
>> > bit of forethought.
>> >
>> > Cheers,
>> > Karol
>> >
>> > On Aug 18 2014, Martin Blood-Forsythe wrote:
>> > > I'll have to look at the way that the multiple file parse is
>> implemented
>> > > for Molpro, but at least for ORCA and Q-Chem adding a binary file
>> parser
>> > > wouldn't require much different than the way a text based multiple
>> file
>> > > parsing works. I've had a lot of success parsing these files using
>> > > numpy.fromfile and numpy.frombuffer.  Typically the only required
>> knowledge
>> > > is the number of orbitals (to determine the array shape to dump into)
>> and
>> > > the file name.
>> > >
>> > > -Martin
>>
>> --
>> written by Karol M. Langner
>> Tue Aug 19 08:20:53 EDT 2014
>>
>
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Eric B. <er...@pi...>]

This would certainly be interesting, but I have a few comments that are
probably Q-Chem specific:

1. As Martin already mentioned, unless you modify the `qchem` driver script
or you pass the flag `-save` when running, the folder with all of these
binary files gets deleted. This means they usually aren't present, except
when you want them, in contrast to the regular plain-text output file which
is always produced. This is in contrast to something like the HESS file,
which is only present when performing an analytic Hessian calculation. ORCA
is even worse about this, since it deletes scratch/temp files early and
often.

2. Since almost all of these files are named with a number, anyone who is
not a Q-Chem developer probably doesn't know what's in these files. For
example, 53 corresponds to the MO coefficients. Even knowing the
number/content mapping doesn't easily tell you the dimensionality of the
file; you have to know what all of the reads and writes to that file look
like.

2b. Once you know the number/content mapping and the dimensionality, there
is still no guarantee about what's in the file, due to nasty bits in the
source code with people using files for things other than what they are
specified as containing.

If one wanted to make a Q-Chem-specific parser for a bunch of binary files,
it would need to be maintained by someone who is a developer and knows
where all of the reads/writes to that file are, but even then it's quite
risky. You just can't perform a visual spot-check on them to see if there
are problems. It's definitely functionality for expert users.

It seems as though this is something that should either be in a separate
package that depends on cclib, or at least is separate from the parsers/
directory.

Martin, do you have scripts for parsing any of these that are publicly
viewable? I'd be interested to see how you do it. I'd really love to know
the structure of ORCA's *.gbw files.

Eric


On Tue, Aug 19, 2014 at 8:21 AM, Karol M. Langner <kar...@gm...>
wrote:

> Here is the issue I created for this:
> https://github.com/cclib/cclib/issues/120
>
> On Aug 18 2014, Karol M. Langner wrote:
> > Hi Martin,
> >
> > That's true, technically it is not hard. I was thinking more about
> integration
> > with the rest of cclib. Binary files will not be "parsed" in the sense
> logfiles
> > are, that is the task is not one of reading a text file line by line and
> > extracting useful information. Rather, it is one of reading data that is
> stored
> > in a predefined format. This is obvious, but it implies some design
> choices
> > that have not been discussed here. Above all, the line-by-line paradigm
> > used in the class LogFile (inherited by all parsers) is not useful and
> > we cannot just pass one extra file.
> >
> > Anyway, this can definitely be done, I'm just saying it needs a little
> > bit of forethought.
> >
> > Cheers,
> > Karol
> >
> > On Aug 18 2014, Martin Blood-Forsythe wrote:
> > > I'll have to look at the way that the multiple file parse is
> implemented
> > > for Molpro, but at least for ORCA and Q-Chem adding a binary file
> parser
> > > wouldn't require much different than the way a text based multiple file
> > > parsing works. I've had a lot of success parsing these files using
> > > numpy.fromfile and numpy.frombuffer.  Typically the only required
> knowledge
> > > is the number of orbitals (to determine the array shape to dump into)
> and
> > > the file name.
> > >
> > > -Martin
>
> --
> written by Karol M. Langner
> Tue Aug 19 08:20:53 EDT 2014
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

Here is the issue I created for this: https://github.com/cclib/cclib/issues/120

On Aug 18 2014, Karol M. Langner wrote:
> Hi Martin,
> 
> That's true, technically it is not hard. I was thinking more about integration
> with the rest of cclib. Binary files will not be "parsed" in the sense logfiles
> are, that is the task is not one of reading a text file line by line and
> extracting useful information. Rather, it is one of reading data that is stored
> in a predefined format. This is obvious, but it implies some design choices
> that have not been discussed here. Above all, the line-by-line paradigm
> used in the class LogFile (inherited by all parsers) is not useful and
> we cannot just pass one extra file.
> 
> Anyway, this can definitely be done, I'm just saying it needs a little
> bit of forethought.
> 
> Cheers,
> Karol
> 
> On Aug 18 2014, Martin Blood-Forsythe wrote:
> > I'll have to look at the way that the multiple file parse is implemented
> > for Molpro, but at least for ORCA and Q-Chem adding a binary file parser
> > wouldn't require much different than the way a text based multiple file
> > parsing works. I've had a lot of success parsing these files using
> > numpy.fromfile and numpy.frombuffer.  Typically the only required knowledge
> > is the number of orbitals (to determine the array shape to dump into) and
> > the file name.
> > 
> > -Martin

-- 
written by Karol M. Langner
Tue Aug 19 08:20:53 EDT 2014

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

Hi Martin,

That's true, technically it is not hard. I was thinking more about integration
with the rest of cclib. Binary files will not be "parsed" in the sense logfiles
are, that is the task is not one of reading a text file line by line and
extracting useful information. Rather, it is one of reading data that is stored
in a predefined format. This is obvious, but it implies some design choices
that have not been discussed here. Above all, the line-by-line paradigm
used in the class LogFile (inherited by all parsers) is not useful and
we cannot just pass one extra file.

Anyway, this can definitely be done, I'm just saying it needs a little
bit of forethought.

Cheers,
Karol

On Aug 18 2014, Martin Blood-Forsythe wrote:
> I'll have to look at the way that the multiple file parse is implemented
> for Molpro, but at least for ORCA and Q-Chem adding a binary file parser
> wouldn't require much different than the way a text based multiple file
> parsing works. I've had a lot of success parsing these files using
> numpy.fromfile and numpy.frombuffer.  Typically the only required knowledge
> is the number of orbitals (to determine the array shape to dump into) and
> the file name.
> 
> -Martin
> 
> 
> 
> 
> On Mon, Aug 18, 2014 at 1:52 PM, Karol Langner <kar...@gm...>
> wrote:
> 
> > Well, we do support parsing multiple files for one job, although this has
> > been limited so far to Molpro if I recall correctly (it was probably also
> > in the works for Turbomole, but this is not an officially supported parser
> > ATM). We have not been parsing binary files at all, though, so that might
> > involve a major change in the way cclib works. It is worth doing, however,
> > or the reasons you mentioned as well as others.
> >
> > Karol
> >
> >
> > On Mon, Aug 18, 2014 at 10:44 AM, Martin Blood-Forsythe <
> > mbl...@ph...> wrote:
> >
> >> Sounds great.  I'll take a look at your qchem branch and see if there are
> >> areas that I think I can add to.  Would there be interest in my adding
> >> parsers for some quantities directly from the binary save files that qchem
> >> leaves around (with an if file exists condition of course)? The primary
> >> advantage of this is getting everything in float precision, but it can also
> >> capture output that the user did not request be printed to the .out file.
> >>
> >> -Martin
> >>
> >>
> >> ---------------------------------------------------
> >> Martin Blood-Forsythe
> >> Graduate Student in Physics
> >> Harvard University
> >> Aspuru-Guzik Lab
> >>
> >>
> >> On Mon, Aug 18, 2014 at 10:14 AM, Eric Berquist <er...@pi...> wrote:
> >>
> >>> Hi Karol and Martin,
> >>>
> >>> My Q-Chem parser is currently working on many "difficult" files in
> >>> addition to some tests that I've made. It hasn't been merged in yet since
> >>> the Travis CI build fails for a silly reason, which I will get to by the
> >>> end of the week. It has to do with there being no regression folder/data in
> >>> the cclib_data repository; I'm going to find a trouble file somewhere and
> >>> add that.
> >>>
> >>> If anyone would like to test out the parser and submit pull requests, my
> >>> GitHub username is `berquist`, and I've created a `qchem` branch.
> >>>
> >>> Eric
> >>>
> >>> On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
> >>> wrote:
> >>>
> >>>> Hi Martin,
> >>>>
> >>>> Nice to hear from you. We actually upport ORCA in the newest release of
> >>>> cclib,
> >>>> which would be 1.2 -- have you looked at the repostiory at
> >>>> https://github.com/cclib/cclib ?
> >>>> Of course, there is still some work to do for ORCA.
> >>>>
> >>>> As far as Q-Chem is concerned, I am fairly certain someone else has
> >>>> recently
> >>>> expressed interest in contributing to that (on this ML), so you would
> >>>> not be alone.
> >>>> Contributing unit test output is already tremendous, since it gives a
> >>>> starting
> >>>> point to get the parser working.
> >>>>
> >>>> In any case, you are more than welcome to fork from our git repository
> >>>> and put in
> >>>> pull requests. In fact, we would be tremendously happy to merge new
> >>>> features.
> >>>>
> >>>> Developmentwise, cclib has been quiet during this summer.
> >>>>
> >>>> You might also want to look at the new docs: http://cclib.github.io/
> >>>>
> >>>> The sourceforge website and wiki is now obsolete, and will probably be
> >>>> soon offline.
> >>>>
> >>>> Best,
> >>>> Karol
> >>>>
> >>>> On Aug 12 2014, Martin Blood-Forsythe wrote:
> >>>> > Hi cclib developers,
> >>>> > I'm interested in contributing towards the efforts to get Q-Chem and
> >>>> ORCA
> >>>> > parsers supported in cclib.  I'd be happy to contribute unit test
> >>>> output
> >>>> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> >>>> > experience with writing python parsers for both the text-based and
> >>>> binary
> >>>> > output files from both ORCA and Q-Chem, so I'm happy to help in any
> >>>> way I
> >>>> > can.
> >>>> > Best wishes,
> >>>> > -Martin
> >>>> >
> >>>> > ---------------------------------------------------
> >>>> > Martin Blood-Forsythe
> >>>> > Graduate Student in Physics
> >>>> > Harvard University
> >>>>
> >>>> >
> >>>> ------------------------------------------------------------------------------
> >>>>
> >>>> > _______________________________________________
> >>>> > cclib-devel mailing list
> >>>> > ccl...@li...
> >>>> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
> >>>>
> >>>>
> >>>> --
> >>>> written by Karol M. Langner
> >>>> Thu Aug 14 22:06:25 EDT 2014
> >>>>
> >>>>
> >>>> ------------------------------------------------------------------------------
> >>>> _______________________________________________
> >>>> cclib-devel mailing list
> >>>> ccl...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/cclib-devel
> >>>>
> >>>
> >>
> >

-- 
written by Karol M. Langner
Mon Aug 18 20:35:25 EDT 2014

Re: [cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

I'll have to look at the way that the multiple file parse is implemented
for Molpro, but at least for ORCA and Q-Chem adding a binary file parser
wouldn't require much different than the way a text based multiple file
parsing works. I've had a lot of success parsing these files using
numpy.fromfile and numpy.frombuffer.  Typically the only required knowledge
is the number of orbitals (to determine the array shape to dump into) and
the file name.

-Martin




On Mon, Aug 18, 2014 at 1:52 PM, Karol Langner <kar...@gm...>
wrote:

> Well, we do support parsing multiple files for one job, although this has
> been limited so far to Molpro if I recall correctly (it was probably also
> in the works for Turbomole, but this is not an officially supported parser
> ATM). We have not been parsing binary files at all, though, so that might
> involve a major change in the way cclib works. It is worth doing, however,
> or the reasons you mentioned as well as others.
>
> Karol
>
>
> On Mon, Aug 18, 2014 at 10:44 AM, Martin Blood-Forsythe <
> mbl...@ph...> wrote:
>
>> Sounds great.  I'll take a look at your qchem branch and see if there are
>> areas that I think I can add to.  Would there be interest in my adding
>> parsers for some quantities directly from the binary save files that qchem
>> leaves around (with an if file exists condition of course)? The primary
>> advantage of this is getting everything in float precision, but it can also
>> capture output that the user did not request be printed to the .out file.
>>
>> -Martin
>>
>>
>> ---------------------------------------------------
>> Martin Blood-Forsythe
>> Graduate Student in Physics
>> Harvard University
>> Aspuru-Guzik Lab
>>
>>
>> On Mon, Aug 18, 2014 at 10:14 AM, Eric Berquist <er...@pi...> wrote:
>>
>>> Hi Karol and Martin,
>>>
>>> My Q-Chem parser is currently working on many "difficult" files in
>>> addition to some tests that I've made. It hasn't been merged in yet since
>>> the Travis CI build fails for a silly reason, which I will get to by the
>>> end of the week. It has to do with there being no regression folder/data in
>>> the cclib_data repository; I'm going to find a trouble file somewhere and
>>> add that.
>>>
>>> If anyone would like to test out the parser and submit pull requests, my
>>> GitHub username is `berquist`, and I've created a `qchem` branch.
>>>
>>> Eric
>>>
>>> On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
>>> wrote:
>>>
>>>> Hi Martin,
>>>>
>>>> Nice to hear from you. We actually upport ORCA in the newest release of
>>>> cclib,
>>>> which would be 1.2 -- have you looked at the repostiory at
>>>> https://github.com/cclib/cclib ?
>>>> Of course, there is still some work to do for ORCA.
>>>>
>>>> As far as Q-Chem is concerned, I am fairly certain someone else has
>>>> recently
>>>> expressed interest in contributing to that (on this ML), so you would
>>>> not be alone.
>>>> Contributing unit test output is already tremendous, since it gives a
>>>> starting
>>>> point to get the parser working.
>>>>
>>>> In any case, you are more than welcome to fork from our git repository
>>>> and put in
>>>> pull requests. In fact, we would be tremendously happy to merge new
>>>> features.
>>>>
>>>> Developmentwise, cclib has been quiet during this summer.
>>>>
>>>> You might also want to look at the new docs: http://cclib.github.io/
>>>>
>>>> The sourceforge website and wiki is now obsolete, and will probably be
>>>> soon offline.
>>>>
>>>> Best,
>>>> Karol
>>>>
>>>> On Aug 12 2014, Martin Blood-Forsythe wrote:
>>>> > Hi cclib developers,
>>>> > I'm interested in contributing towards the efforts to get Q-Chem and
>>>> ORCA
>>>> > parsers supported in cclib.  I'd be happy to contribute unit test
>>>> output
>>>> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
>>>> > experience with writing python parsers for both the text-based and
>>>> binary
>>>> > output files from both ORCA and Q-Chem, so I'm happy to help in any
>>>> way I
>>>> > can.
>>>> > Best wishes,
>>>> > -Martin
>>>> >
>>>> > ---------------------------------------------------
>>>> > Martin Blood-Forsythe
>>>> > Graduate Student in Physics
>>>> > Harvard University
>>>>
>>>> >
>>>> ------------------------------------------------------------------------------
>>>>
>>>> > _______________________________________________
>>>> > cclib-devel mailing list
>>>> > ccl...@li...
>>>> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>>>
>>>>
>>>> --
>>>> written by Karol M. Langner
>>>> Thu Aug 14 22:06:25 EDT 2014
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> _______________________________________________
>>>> cclib-devel mailing list
>>>> ccl...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>>>
>>>
>>
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol L. <kar...@gm...>]

Well, we do support parsing multiple files for one job, although this has
been limited so far to Molpro if I recall correctly (it was probably also
in the works for Turbomole, but this is not an officially supported parser
ATM). We have not been parsing binary files at all, though, so that might
involve a major change in the way cclib works. It is worth doing, however,
or the reasons you mentioned as well as others.

Karol


On Mon, Aug 18, 2014 at 10:44 AM, Martin Blood-Forsythe <
mbl...@ph...> wrote:

> Sounds great.  I'll take a look at your qchem branch and see if there are
> areas that I think I can add to.  Would there be interest in my adding
> parsers for some quantities directly from the binary save files that qchem
> leaves around (with an if file exists condition of course)? The primary
> advantage of this is getting everything in float precision, but it can also
> capture output that the user did not request be printed to the .out file.
>
> -Martin
>
>
> ---------------------------------------------------
> Martin Blood-Forsythe
> Graduate Student in Physics
> Harvard University
> Aspuru-Guzik Lab
>
>
> On Mon, Aug 18, 2014 at 10:14 AM, Eric Berquist <er...@pi...> wrote:
>
>> Hi Karol and Martin,
>>
>> My Q-Chem parser is currently working on many "difficult" files in
>> addition to some tests that I've made. It hasn't been merged in yet since
>> the Travis CI build fails for a silly reason, which I will get to by the
>> end of the week. It has to do with there being no regression folder/data in
>> the cclib_data repository; I'm going to find a trouble file somewhere and
>> add that.
>>
>> If anyone would like to test out the parser and submit pull requests, my
>> GitHub username is `berquist`, and I've created a `qchem` branch.
>>
>> Eric
>>
>> On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
>> wrote:
>>
>>> Hi Martin,
>>>
>>> Nice to hear from you. We actually upport ORCA in the newest release of
>>> cclib,
>>> which would be 1.2 -- have you looked at the repostiory at
>>> https://github.com/cclib/cclib ?
>>> Of course, there is still some work to do for ORCA.
>>>
>>> As far as Q-Chem is concerned, I am fairly certain someone else has
>>> recently
>>> expressed interest in contributing to that (on this ML), so you would
>>> not be alone.
>>> Contributing unit test output is already tremendous, since it gives a
>>> starting
>>> point to get the parser working.
>>>
>>> In any case, you are more than welcome to fork from our git repository
>>> and put in
>>> pull requests. In fact, we would be tremendously happy to merge new
>>> features.
>>>
>>> Developmentwise, cclib has been quiet during this summer.
>>>
>>> You might also want to look at the new docs: http://cclib.github.io/
>>>
>>> The sourceforge website and wiki is now obsolete, and will probably be
>>> soon offline.
>>>
>>> Best,
>>> Karol
>>>
>>> On Aug 12 2014, Martin Blood-Forsythe wrote:
>>> > Hi cclib developers,
>>> > I'm interested in contributing towards the efforts to get Q-Chem and
>>> ORCA
>>> > parsers supported in cclib.  I'd be happy to contribute unit test
>>> output
>>> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
>>> > experience with writing python parsers for both the text-based and
>>> binary
>>> > output files from both ORCA and Q-Chem, so I'm happy to help in any
>>> way I
>>> > can.
>>> > Best wishes,
>>> > -Martin
>>> >
>>> > ---------------------------------------------------
>>> > Martin Blood-Forsythe
>>> > Graduate Student in Physics
>>> > Harvard University
>>>
>>> >
>>> ------------------------------------------------------------------------------
>>>
>>> > _______________________________________________
>>> > cclib-devel mailing list
>>> > ccl...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>>
>>>
>>> --
>>> written by Karol M. Langner
>>> Thu Aug 14 22:06:25 EDT 2014
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> cclib-devel mailing list
>>> ccl...@li...
>>> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>>
>>
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

Sounds great.  I'll take a look at your qchem branch and see if there are
areas that I think I can add to.  Would there be interest in my adding
parsers for some quantities directly from the binary save files that qchem
leaves around (with an if file exists condition of course)? The primary
advantage of this is getting everything in float precision, but it can also
capture output that the user did not request be printed to the .out file.

-Martin


---------------------------------------------------
Martin Blood-Forsythe
Graduate Student in Physics
Harvard University
Aspuru-Guzik Lab


On Mon, Aug 18, 2014 at 10:14 AM, Eric Berquist <er...@pi...> wrote:

> Hi Karol and Martin,
>
> My Q-Chem parser is currently working on many "difficult" files in
> addition to some tests that I've made. It hasn't been merged in yet since
> the Travis CI build fails for a silly reason, which I will get to by the
> end of the week. It has to do with there being no regression folder/data in
> the cclib_data repository; I'm going to find a trouble file somewhere and
> add that.
>
> If anyone would like to test out the parser and submit pull requests, my
> GitHub username is `berquist`, and I've created a `qchem` branch.
>
> Eric
>
> On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
> wrote:
>
>> Hi Martin,
>>
>> Nice to hear from you. We actually upport ORCA in the newest release of
>> cclib,
>> which would be 1.2 -- have you looked at the repostiory at
>> https://github.com/cclib/cclib ?
>> Of course, there is still some work to do for ORCA.
>>
>> As far as Q-Chem is concerned, I am fairly certain someone else has
>> recently
>> expressed interest in contributing to that (on this ML), so you would not
>> be alone.
>> Contributing unit test output is already tremendous, since it gives a
>> starting
>> point to get the parser working.
>>
>> In any case, you are more than welcome to fork from our git repository
>> and put in
>> pull requests. In fact, we would be tremendously happy to merge new
>> features.
>>
>> Developmentwise, cclib has been quiet during this summer.
>>
>> You might also want to look at the new docs: http://cclib.github.io/
>>
>> The sourceforge website and wiki is now obsolete, and will probably be
>> soon offline.
>>
>> Best,
>> Karol
>>
>> On Aug 12 2014, Martin Blood-Forsythe wrote:
>> > Hi cclib developers,
>> > I'm interested in contributing towards the efforts to get Q-Chem and
>> ORCA
>> > parsers supported in cclib.  I'd be happy to contribute unit test output
>> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
>> > experience with writing python parsers for both the text-based and
>> binary
>> > output files from both ORCA and Q-Chem, so I'm happy to help in any way
>> I
>> > can.
>> > Best wishes,
>> > -Martin
>> >
>> > ---------------------------------------------------
>> > Martin Blood-Forsythe
>> > Graduate Student in Physics
>> > Harvard University
>>
>> >
>> ------------------------------------------------------------------------------
>>
>> > _______________________________________________
>> > cclib-devel mailing list
>> > ccl...@li...
>> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>
>>
>> --
>> written by Karol M. Langner
>> Thu Aug 14 22:06:25 EDT 2014
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> cclib-devel mailing list
>> ccl...@li...
>> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Eric B. <er...@pi...>]

Hi Karol and Martin,

My Q-Chem parser is currently working on many "difficult" files in addition
to some tests that I've made. It hasn't been merged in yet since the Travis
CI build fails for a silly reason, which I will get to by the end of the
week. It has to do with there being no regression folder/data in the
cclib_data repository; I'm going to find a trouble file somewhere and add
that.

If anyone would like to test out the parser and submit pull requests, my
GitHub username is `berquist`, and I've created a `qchem` branch.

Eric

On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
wrote:

> Hi Martin,
>
> Nice to hear from you. We actually upport ORCA in the newest release of
> cclib,
> which would be 1.2 -- have you looked at the repostiory at
> https://github.com/cclib/cclib ?
> Of course, there is still some work to do for ORCA.
>
> As far as Q-Chem is concerned, I am fairly certain someone else has
> recently
> expressed interest in contributing to that (on this ML), so you would not
> be alone.
> Contributing unit test output is already tremendous, since it gives a
> starting
> point to get the parser working.
>
> In any case, you are more than welcome to fork from our git repository and
> put in
> pull requests. In fact, we would be tremendously happy to merge new
> features.
>
> Developmentwise, cclib has been quiet during this summer.
>
> You might also want to look at the new docs: http://cclib.github.io/
>
> The sourceforge website and wiki is now obsolete, and will probably be
> soon offline.
>
> Best,
> Karol
>
> On Aug 12 2014, Martin Blood-Forsythe wrote:
> > Hi cclib developers,
> > I'm interested in contributing towards the efforts to get Q-Chem and ORCA
> > parsers supported in cclib.  I'd be happy to contribute unit test output
> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> > experience with writing python parsers for both the text-based and binary
> > output files from both ORCA and Q-Chem, so I'm happy to help in any way I
> > can.
> > Best wishes,
> > -Martin
> >
> > ---------------------------------------------------
> > Martin Blood-Forsythe
> > Graduate Student in Physics
> > Harvard University
>
> >
> ------------------------------------------------------------------------------
>
> > _______________________________________________
> > cclib-devel mailing list
> > ccl...@li...
> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>
>
> --
> written by Karol M. Langner
> Thu Aug 14 22:06:25 EDT 2014
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

Hi Martin,

So I checked and the Q-Chem changes came from Eric Berquist. And it seems that
I have some things to do in that area, with a pending pull request for the
initial Q-Chem parser. So... I will try to get that in order ASAP!

- Karol

On Aug 14 2014, Martin Blood-Forsythe wrote:
> Hi Karol,
> I hadn't seen the github repository.  I did notice that the ORCA support
> seemed to be more extensive than the sourceforge wiki indicated.  I'll make
> a fork from github and look into the Q-Chem parser.  I can also provide
> NWChem outputs if that would be of interest, but I have less experience
> with NWChem so I have less ability to say that the outputs are 100%
> prototypical.
> Best,
> -Martin
> 
> 
> ---------------------------------------------------
> Martin Blood-Forsythe
> Graduate Student in Physics
> Harvard University
> Aspuru-Guzik Lab
> 
> 
> On Thu, Aug 14, 2014 at 10:27 PM, Karol M. Langner <kar...@gm...>
> wrote:
> 
> > Hi Martin,
> >
> > Nice to hear from you. We actually upport ORCA in the newest release of
> > cclib,
> > which would be 1.2 -- have you looked at the repostiory at
> > https://github.com/cclib/cclib ?
> > Of course, there is still some work to do for ORCA.
> >
> > As far as Q-Chem is concerned, I am fairly certain someone else has
> > recently
> > expressed interest in contributing to that (on this ML), so you would not
> > be alone.
> > Contributing unit test output is already tremendous, since it gives a
> > starting
> > point to get the parser working.
> >
> > In any case, you are more than welcome to fork from our git repository and
> > put in
> > pull requests. In fact, we would be tremendously happy to merge new
> > features.
> >
> > Developmentwise, cclib has been quiet during this summer.
> >
> > You might also want to look at the new docs: http://cclib.github.io/
> >
> > The sourceforge website and wiki is now obsolete, and will probably be
> > soon offline.
> >
> > Best,
> > Karol
> >
> > On Aug 12 2014, Martin Blood-Forsythe wrote:
> > > Hi cclib developers,
> > > I'm interested in contributing towards the efforts to get Q-Chem and ORCA
> > > parsers supported in cclib.  I'd be happy to contribute unit test output
> > > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> > > experience with writing python parsers for both the text-based and binary
> > > output files from both ORCA and Q-Chem, so I'm happy to help in any way I
> > > can.
> > > Best wishes,
> > > -Martin
> > >
> > > ---------------------------------------------------
> > > Martin Blood-Forsythe
> > > Graduate Student in Physics
> > > Harvard University
> >
> > >
> > ------------------------------------------------------------------------------
> >
> > > _______________________________________________
> > > cclib-devel mailing list
> > > ccl...@li...
> > > https://lists.sourceforge.net/lists/listinfo/cclib-devel
> >
> >
> > --
> > written by Karol M. Langner
> > Thu Aug 14 22:06:25 EDT 2014
> >

-- 
written by Karol M. Langner
Fri Aug 15 08:29:16 EDT 2014

Re: [cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

Hi Karol,
I hadn't seen the github repository.  I did notice that the ORCA support
seemed to be more extensive than the sourceforge wiki indicated.  I'll make
a fork from github and look into the Q-Chem parser.  I can also provide
NWChem outputs if that would be of interest, but I have less experience
with NWChem so I have less ability to say that the outputs are 100%
prototypical.
Best,
-Martin


---------------------------------------------------
Martin Blood-Forsythe
Graduate Student in Physics
Harvard University
Aspuru-Guzik Lab


On Thu, Aug 14, 2014 at 10:27 PM, Karol M. Langner <kar...@gm...>
wrote:

> Hi Martin,
>
> Nice to hear from you. We actually upport ORCA in the newest release of
> cclib,
> which would be 1.2 -- have you looked at the repostiory at
> https://github.com/cclib/cclib ?
> Of course, there is still some work to do for ORCA.
>
> As far as Q-Chem is concerned, I am fairly certain someone else has
> recently
> expressed interest in contributing to that (on this ML), so you would not
> be alone.
> Contributing unit test output is already tremendous, since it gives a
> starting
> point to get the parser working.
>
> In any case, you are more than welcome to fork from our git repository and
> put in
> pull requests. In fact, we would be tremendously happy to merge new
> features.
>
> Developmentwise, cclib has been quiet during this summer.
>
> You might also want to look at the new docs: http://cclib.github.io/
>
> The sourceforge website and wiki is now obsolete, and will probably be
> soon offline.
>
> Best,
> Karol
>
> On Aug 12 2014, Martin Blood-Forsythe wrote:
> > Hi cclib developers,
> > I'm interested in contributing towards the efforts to get Q-Chem and ORCA
> > parsers supported in cclib.  I'd be happy to contribute unit test output
> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> > experience with writing python parsers for both the text-based and binary
> > output files from both ORCA and Q-Chem, so I'm happy to help in any way I
> > can.
> > Best wishes,
> > -Martin
> >
> > ---------------------------------------------------
> > Martin Blood-Forsythe
> > Graduate Student in Physics
> > Harvard University
>
> >
> ------------------------------------------------------------------------------
>
> > _______________________________________________
> > cclib-devel mailing list
> > ccl...@li...
> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>
>
> --
> written by Karol M. Langner
> Thu Aug 14 22:06:25 EDT 2014
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

Hi Martin,

Nice to hear from you. We actually upport ORCA in the newest release of cclib,
which would be 1.2 -- have you looked at the repostiory at https://github.com/cclib/cclib ?
Of course, there is still some work to do for ORCA.

As far as Q-Chem is concerned, I am fairly certain someone else has recently
expressed interest in contributing to that (on this ML), so you would not be alone.
Contributing unit test output is already tremendous, since it gives a starting
point to get the parser working.

In any case, you are more than welcome to fork from our git repository and put in
pull requests. In fact, we would be tremendously happy to merge new features.

Developmentwise, cclib has been quiet during this summer.

You might also want to look at the new docs: http://cclib.github.io/

The sourceforge website and wiki is now obsolete, and will probably be soon offline.

Best,
Karol

On Aug 12 2014, Martin Blood-Forsythe wrote:
> Hi cclib developers,
> I'm interested in contributing towards the efforts to get Q-Chem and ORCA
> parsers supported in cclib.  I'd be happy to contribute unit test output
> files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> experience with writing python parsers for both the text-based and binary
> output files from both ORCA and Q-Chem, so I'm happy to help in any way I
> can.
> Best wishes,
> -Martin
> 
> ---------------------------------------------------
> Martin Blood-Forsythe
> Graduate Student in Physics
> Harvard University

> ------------------------------------------------------------------------------

> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel


-- 
written by Karol M. Langner
Thu Aug 14 22:06:25 EDT 2014

[cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

Hi cclib developers,
I'm interested in contributing towards the efforts to get Q-Chem and ORCA
parsers supported in cclib.  I'd be happy to contribute unit test output
files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
experience with writing python parsers for both the text-based and binary
output files from both ORCA and Q-Chem, so I'm happy to help in any way I
can.
Best wishes,
-Martin

---------------------------------------------------
Martin Blood-Forsythe
Graduate Student in Physics
Harvard University

Re: [cclib-devel] Q-Chem Parser

[Eric B. <er...@pi...>]

Hi Karol,

I'll take a look at the unit tests and see what sorts of jobs are there.
Let me know if there is anything specific/special that should be included.

Eric


On Fri, Jun 20, 2014 at 2:49 PM, Karol M. Langner <kar...@gm...>
wrote:

> Hi Eric,
>
> No one is working on a Q-Chem parser at this time, but we would be
> very interested if you would be willing to contribute one.
>
> I think the easiest way to start off is to replicate some basic
> unit tests we have in Q-Chem, and start hacking away at the
> attributes you can parse from those outputs.
>
> Cheers,
> Karol
>
> On Jun 19 2014, Eric Berquist wrote:
> > Hello,
> >
> > I'm interested in contributing a Q-Chem parser to cclib, but I want to
> make
> > sure first that no one else is working on one. Is this something that is
> of
> > interest?
> >
> > Eric
>
> >
> ------------------------------------------------------------------------------
> > HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
> > Find What Matters Most in Your Big Data with HPCC Systems
> > Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
> > Leverages Graph Analysis for Fast Processing & Easy Data Exploration
> > http://p.sf.net/sfu/hpccsystems
>
> > _______________________________________________
> > cclib-devel mailing list
> > ccl...@li...
> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>
>
> --
> written by Karol M. Langner
> Fri Jun 20 14:48:10 EDT 2014
>

Re: [cclib-devel] Q-Chem Parser

[Karol M. L. <kar...@gm...>]

Hi Eric,

No one is working on a Q-Chem parser at this time, but we would be
very interested if you would be willing to contribute one.

I think the easiest way to start off is to replicate some basic
unit tests we have in Q-Chem, and start hacking away at the
attributes you can parse from those outputs.

Cheers,
Karol

On Jun 19 2014, Eric Berquist wrote:
> Hello,
> 
> I'm interested in contributing a Q-Chem parser to cclib, but I want to make
> sure first that no one else is working on one. Is this something that is of
> interest?
> 
> Eric

> ------------------------------------------------------------------------------
> HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
> Find What Matters Most in Your Big Data with HPCC Systems
> Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
> Leverages Graph Analysis for Fast Processing & Easy Data Exploration
> http://p.sf.net/sfu/hpccsystems

> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel


-- 
written by Karol M. Langner
Fri Jun 20 14:48:10 EDT 2014

[cclib-devel] Q-Chem Parser

[Eric B. <er...@pi...>]

Hello,

I'm interested in contributing a Q-Chem parser to cclib, but I want to make
sure first that no one else is working on one. Is this something that is of
interest?

Eric

Re: [cclib-devel] New docs theme

[Karol L. <kar...@gm...>]

I resolved the ghpages issue, so the new theme is also online at
cclib.github.io
Still, let me know what you think.

- Karol

On Sat, Apr 19, 2014 at 12:14 PM, Karol Langner <kar...@gm...>wrote:

> Hi guys,
>
> When we talked about the future of cclib's documentation some time ago,
> Read The Docs came up. I don't think it's the best place to place cclib's
> documentation, simply because we have so little documentation in the code,
> and so much static text. The things we do want to generate automatically
> (attribute tables) we need to do programmatically anyway.
>
> However, I do like the theme they have over at Read the Docs. It's much
> more easy to use than the default Sphinx theme. So for v1.2 that's coming
> up I've given it a try -- added another repo for cclib with the theme. The
> only thing I chnaged for now if the CSS max-width to 100%, so that our wide
> tables are not squezzed on large screens. We can tweak a couple other such
> details, but I don't think any major changes are needed.
>
> The github pages are not accepting the changes for some reason (I have
> contacted their support), but here are the docs with the new look at a
> temporary location:
> http://mmqc.org/~kml/tmp/cclib.github.io/
>
> Just let me know whether you like it. If not, I can revert to the default
> theme.
>
> Karol
>

[cclib-devel] New docs theme

[Karol L. <kar...@gm...>]

Hi guys,

When we talked about the future of cclib's documentation some time ago,
Read The Docs came up. I don't think it's the best place to place cclib's
documentation, simply because we have so little documentation in the code,
and so much static text. The things we do want to generate automatically
(attribute tables) we need to do programmatically anyway.

However, I do like the theme they have over at Read the Docs. It's much
more easy to use than the default Sphinx theme. So for v1.2 that's coming
up I've given it a try -- added another repo for cclib with the theme. The
only thing I chnaged for now if the CSS max-width to 100%, so that our wide
tables are not squezzed on large screens. We can tweak a couple other such
details, but I don't think any major changes are needed.

The github pages are not accepting the changes for some reason (I have
contacted their support), but here are the docs with the new look at a
temporary location:
http://mmqc.org/~kml/tmp/cclib.github.io/

Just let me know whether you like it. If not, I can revert to the default
theme.

Karol

Re: [cclib-devel] Moving to github (Was: github fork)

[Karol M. L. <kar...@gm...>]

Hi guys,

On Dec 30 2013, Noel O'Boyle wrote:
> Regarding workflow, the tidiest way would be to do all your work on a
> branch of your own fork of the repo, then click on github to send a
> pull request, and then either merge it yourself on github, or email
> the list asking for a code review. What do you think? We should all
> get an email when the pull request arrives, and if you want you can
> wait a day or two to see whether someone else leaves a comment before
> merging.

This sounds good. I agree, it would be best practice to have someone else
do the review and merge than the author. I'll put in some requests in
the upcoming days, so feel free to pull them in in your free time :)

- Karol

-- 
written by Karol M. Langner
Tue Jan  7 06:00:31 EST 2014

[cclib-devel] about the license for cclib-data

[Karol M. L. <kar...@gm...>]

Hi guys,

I slapped a GPL license onto the new cclib-data repo, and Noel suggested that
some Public Domain license would be more appropriate. He is certainly right,
since this is data, not code, and I was thinking about CC0 or PDDL.

However, I'm not sure if it would be OK for us to claim such a blanket license
for all of the files in cclib-data. I think there are three separate issues:

1) The logfiles often contain explicit copyright messages printed by the
programs (Gaussian, ADF, etc.). Even though by common law these restrictions
do not apply to generated numeric data (which is what we are extracting),
how can we apply a PD license to such messages? They are a part of the file, no?

2) Some text is copied verbatim by the code into the logfile, which therefore
technically might be covered by copyright (if I understand these issues correctly).
Therefore, we cannot claim that the entire files are in the PD.

3) Some programs (Gaussian and others) require attribution to both the program
and publications for some specific theoretical methods. I don't think we can
release anyone who uses these files from such an obligation.

I would hesitate to modify the files in any way, which I think means effectively
that we cannot put any license on them with their current content, and we should
therefore remove the current license (it does not apply). Maybe in lieu of a license
we should add some sort of comment that includes the points listed above.

Please comment,
Karol

-- 
written by Karol M. Langner
Sat Jan  4 13:31:55 EST 2014

Re: [cclib-devel] github fork

[Clyde F. <cly...@gm...>]

 Hi,

Thanks for the update, great to see cclib on github!

Happy New Year

Clyde


On 30 December 2013 23:19, Karol M. Langner <kar...@gm...> wrote:

> Hi Clyde,
>
> You might be interested to know cclib is now on github:
> https://github.com/cclib/cclib
>
> Cheers,
> Karol
>
> On Nov 08 2013, Clyde Fare wrote:
> > Hi Noel,
> >
> > I've changed the readme to specifically state it's an unofficial fork and
> > for those who wish to contribute to follow your link.
> >
> > I'm trying to construct, submit to the server, collect from the server,
> > extract data from the subsequent files and analyse them entirely from
> > within an IPython notebook.Consequently I'm using several different
> python
> > quantum chemistry wrappers. Part of the reason I've put cclib into a git
> is
> > because I'd like to have all the codes I'm modifying in one code
> management
> > system and partly it's because I assumed some of the modifications that
> > would be useful to me would not necessarily be useful to the majority of
> > the package users.
> >
> > In terms of cclib i've modified the units not because I think they are
> > wrong but simply because I want the three parsers I use to produce
> > consistent results. For my Gaussian calculations I use cclib to extract
> > data from the main body of the log file, I use ASE's gaussian reader (
> >
> https://trac.fysik.dtu.dk/projects/ase/browser/trunk/ase/io/gaussian_reader.py
> )
> > to extract data from the machine readable content at the end of the
> > logfile, and I use molmod's fchk reader (
> > https://github.com/molmod/molmod/blob/master/molmod/io/fchk.py) to
> extract
> > data from the .fchk point files.
> >
> > At the moment ASE's gaussian_reader and cclib use a different set of
> units
> > (and gaussian itself uses a third set of units).
> >
> > Of course I would be thrilled if I could convince cclib and ase to
> > standardize their units. In terms of any changes I make, I'm happy to
> push
> > them back to the official version if it's thought suitable. (Of course if
> > you were on github that would be easier :D)
> >
> > Cheers
> >
> > Clyde
> >
> >
> > On 8 November 2013 09:47, Noel O'Boyle <bao...@gm...> wrote:
> >
> > > (ccing to the list)
> > >
> > > Hi Clyde,
> > >
> > > Thanks for letting me know. You're free to do this of course - might
> > > be good to make it clear that this is an "unofficial" fork though.
> > > Just to save people contacting us asking about ASE.
> > >
> > > Naturally for us we would prefer if you contribute any fixes directly
> > > to cclib. In this case, if any of our units are incorrect we would
> > > like to fix our source.
> > >
> > > This might be the push we need to consider moving to Github. Git on
> > > Windows though is a joke. The interface is not a lot better than the
> > > old CVS one (you probably don't remember this) which also used to call
> > > out to the command line. If only Github ran mercurial...:-/
> > >
> > > - Noel
> > >
> > >
> > >
> > > On 7 November 2013 15:05, Clyde Fare <cly...@gm...> wrote:
> > > > Hi Noel
> > > >
> > > > I've got a git repository with cclib in it because I am faffing about
> > > trying
> > > > to put together a bunch of python of tools associated with quantum
> > > chemistry
> > > > to use with the ipython notebook. In my version I've changed the
> units
> > > that
> > > > cclib uses in it's conversions because I want the various parsers I
> use
> > > to
> > > > all use the same units.
> > > >
> > > > As I wanted an easy way of loading the package into Anaconda I've
> put the
> > > > result on github here:
> > > > https://github.com/Clyde-fare/cclib
> > > >
> > > > Github  have a very fork-me friendly mentality but it occurred to me
> that
> > > > you might not be so happy that I'd done this so just wanted to check
> in
> > > with
> > > > you - I could write a version that means the units used are chosen
> based
> > > on
> > > > some flag (and your initial units are the default) which would mean I
> > > could
> > > > keep my version synced with yours if that's something you would
> prefer.
> > > >
> > > > Cheers
> > > >
> > > > Clyde
> > > >
> > >
>
> >
> ------------------------------------------------------------------------------
> > November Webinars for C, C++, Fortran Developers
> > Accelerate application performance with scalable programming models.
> Explore
> > techniques for threading, error checking, porting, and tuning. Get the
> most
> > from the latest Intel processors and coprocessors. See abstracts and
> register
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
>
> > _______________________________________________
> > cclib-devel mailing list
> > ccl...@li...
> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>
>
> --
> written by Karol M. Langner
> Mon Dec 30 18:18:55 EST 2013
>

Re: [cclib-devel] [Debichem-devel] Bug#733737: Add python3-cclib package

[Karol M. L. <kar...@gm...>]

Hi Daniel,

We also recently moved cclib to git and github: https://github.com/cclib/cclib,
so the upstream repository used will need an update eventually.

More importantly, earlier this year -- after releasing 1.1 -- we switched to Python3,
and the master branch is now intended only for Python3. There is a python2 branch,
although I'm not sure if all new developments will get ported there.

- Karol

On Dec 31 2013, Daniel Leidert wrote:
> Source: cclib
> Version: 1.1-1
> Severity: wishlist
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Gausssum 3 is only for Python 3. Therefor we need a python3-cclib package.
> 
> Regards, Daniel
> 
> 
> - -- System Information:
> Debian Release: jessie/sid
>   APT prefers unstable
>   APT policy: (850, 'unstable'), (700, 'testing'), (560, 'stable'), (500, 'oldstable'), (110, 'experimental')
> Architecture: amd64 (x86_64)
> Foreign Architectures: i386
> 
> Kernel: Linux 3.12-1-amd64 (SMP w/2 CPU cores)
> Locale: LANG=de_DE.utf8, LC_CTYPE=de_DE.utf8 (charmap=UTF-8)
> Shell: /bin/sh linked to /bin/bash
> 
> - -- no debconf information
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.15 (GNU/Linux)
> 
> iEYEARECAAYFAlLCzE4ACgkQm0bx+wiPa4y2xwCePKrirqzd13xFs/Bi/lHc5SC4
> UCgAoJkdnJBhOH78xTF3TNECijszUHue
> =Z6u8
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> Debichem-devel mailing list
> Deb...@li...
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/debichem-devel

-- 
written by Karol M. Langner
Thu Jan  2 06:49:57 EST 2014