cclib-devel

[cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

Hi cclib developers,
I'm interested in contributing towards the efforts to get Q-Chem and ORCA
parsers supported in cclib.  I'd be happy to contribute unit test output
files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
experience with writing python parsers for both the text-based and binary
output files from both ORCA and Q-Chem, so I'm happy to help in any way I
can.
Best wishes,
-Martin

---------------------------------------------------
Martin Blood-Forsythe
Graduate Student in Physics
Harvard University

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

Hi Martin,

Nice to hear from you. We actually upport ORCA in the newest release of cclib,
which would be 1.2 -- have you looked at the repostiory at https://github.com/cclib/cclib ?
Of course, there is still some work to do for ORCA.

As far as Q-Chem is concerned, I am fairly certain someone else has recently
expressed interest in contributing to that (on this ML), so you would not be alone.
Contributing unit test output is already tremendous, since it gives a starting
point to get the parser working.

In any case, you are more than welcome to fork from our git repository and put in
pull requests. In fact, we would be tremendously happy to merge new features.

Developmentwise, cclib has been quiet during this summer.

You might also want to look at the new docs: http://cclib.github.io/

The sourceforge website and wiki is now obsolete, and will probably be soon offline.

Best,
Karol

On Aug 12 2014, Martin Blood-Forsythe wrote:
> Hi cclib developers,
> I'm interested in contributing towards the efforts to get Q-Chem and ORCA
> parsers supported in cclib.  I'd be happy to contribute unit test output
> files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> experience with writing python parsers for both the text-based and binary
> output files from both ORCA and Q-Chem, so I'm happy to help in any way I
> can.
> Best wishes,
> -Martin
> 
> ---------------------------------------------------
> Martin Blood-Forsythe
> Graduate Student in Physics
> Harvard University

> ------------------------------------------------------------------------------

> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel


-- 
written by Karol M. Langner
Thu Aug 14 22:06:25 EDT 2014

Re: [cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

Hi Karol,
I hadn't seen the github repository.  I did notice that the ORCA support
seemed to be more extensive than the sourceforge wiki indicated.  I'll make
a fork from github and look into the Q-Chem parser.  I can also provide
NWChem outputs if that would be of interest, but I have less experience
with NWChem so I have less ability to say that the outputs are 100%
prototypical.
Best,
-Martin


---------------------------------------------------
Martin Blood-Forsythe
Graduate Student in Physics
Harvard University
Aspuru-Guzik Lab


On Thu, Aug 14, 2014 at 10:27 PM, Karol M. Langner <kar...@gm...>
wrote:

> Hi Martin,
>
> Nice to hear from you. We actually upport ORCA in the newest release of
> cclib,
> which would be 1.2 -- have you looked at the repostiory at
> https://github.com/cclib/cclib ?
> Of course, there is still some work to do for ORCA.
>
> As far as Q-Chem is concerned, I am fairly certain someone else has
> recently
> expressed interest in contributing to that (on this ML), so you would not
> be alone.
> Contributing unit test output is already tremendous, since it gives a
> starting
> point to get the parser working.
>
> In any case, you are more than welcome to fork from our git repository and
> put in
> pull requests. In fact, we would be tremendously happy to merge new
> features.
>
> Developmentwise, cclib has been quiet during this summer.
>
> You might also want to look at the new docs: http://cclib.github.io/
>
> The sourceforge website and wiki is now obsolete, and will probably be
> soon offline.
>
> Best,
> Karol
>
> On Aug 12 2014, Martin Blood-Forsythe wrote:
> > Hi cclib developers,
> > I'm interested in contributing towards the efforts to get Q-Chem and ORCA
> > parsers supported in cclib.  I'd be happy to contribute unit test output
> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> > experience with writing python parsers for both the text-based and binary
> > output files from both ORCA and Q-Chem, so I'm happy to help in any way I
> > can.
> > Best wishes,
> > -Martin
> >
> > ---------------------------------------------------
> > Martin Blood-Forsythe
> > Graduate Student in Physics
> > Harvard University
>
> >
> ------------------------------------------------------------------------------
>
> > _______________________________________________
> > cclib-devel mailing list
> > ccl...@li...
> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>
>
> --
> written by Karol M. Langner
> Thu Aug 14 22:06:25 EDT 2014
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

Hi Martin,

So I checked and the Q-Chem changes came from Eric Berquist. And it seems that
I have some things to do in that area, with a pending pull request for the
initial Q-Chem parser. So... I will try to get that in order ASAP!

- Karol

On Aug 14 2014, Martin Blood-Forsythe wrote:
> Hi Karol,
> I hadn't seen the github repository.  I did notice that the ORCA support
> seemed to be more extensive than the sourceforge wiki indicated.  I'll make
> a fork from github and look into the Q-Chem parser.  I can also provide
> NWChem outputs if that would be of interest, but I have less experience
> with NWChem so I have less ability to say that the outputs are 100%
> prototypical.
> Best,
> -Martin
> 
> 
> ---------------------------------------------------
> Martin Blood-Forsythe
> Graduate Student in Physics
> Harvard University
> Aspuru-Guzik Lab
> 
> 
> On Thu, Aug 14, 2014 at 10:27 PM, Karol M. Langner <kar...@gm...>
> wrote:
> 
> > Hi Martin,
> >
> > Nice to hear from you. We actually upport ORCA in the newest release of
> > cclib,
> > which would be 1.2 -- have you looked at the repostiory at
> > https://github.com/cclib/cclib ?
> > Of course, there is still some work to do for ORCA.
> >
> > As far as Q-Chem is concerned, I am fairly certain someone else has
> > recently
> > expressed interest in contributing to that (on this ML), so you would not
> > be alone.
> > Contributing unit test output is already tremendous, since it gives a
> > starting
> > point to get the parser working.
> >
> > In any case, you are more than welcome to fork from our git repository and
> > put in
> > pull requests. In fact, we would be tremendously happy to merge new
> > features.
> >
> > Developmentwise, cclib has been quiet during this summer.
> >
> > You might also want to look at the new docs: http://cclib.github.io/
> >
> > The sourceforge website and wiki is now obsolete, and will probably be
> > soon offline.
> >
> > Best,
> > Karol
> >
> > On Aug 12 2014, Martin Blood-Forsythe wrote:
> > > Hi cclib developers,
> > > I'm interested in contributing towards the efforts to get Q-Chem and ORCA
> > > parsers supported in cclib.  I'd be happy to contribute unit test output
> > > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> > > experience with writing python parsers for both the text-based and binary
> > > output files from both ORCA and Q-Chem, so I'm happy to help in any way I
> > > can.
> > > Best wishes,
> > > -Martin
> > >
> > > ---------------------------------------------------
> > > Martin Blood-Forsythe
> > > Graduate Student in Physics
> > > Harvard University
> >
> > >
> > ------------------------------------------------------------------------------
> >
> > > _______________________________________________
> > > cclib-devel mailing list
> > > ccl...@li...
> > > https://lists.sourceforge.net/lists/listinfo/cclib-devel
> >
> >
> > --
> > written by Karol M. Langner
> > Thu Aug 14 22:06:25 EDT 2014
> >

-- 
written by Karol M. Langner
Fri Aug 15 08:29:16 EDT 2014

Re: [cclib-devel] Q-Chem and ORCA parsers

[Eric B. <er...@pi...>]

Hi Karol and Martin,

My Q-Chem parser is currently working on many "difficult" files in addition
to some tests that I've made. It hasn't been merged in yet since the Travis
CI build fails for a silly reason, which I will get to by the end of the
week. It has to do with there being no regression folder/data in the
cclib_data repository; I'm going to find a trouble file somewhere and add
that.

If anyone would like to test out the parser and submit pull requests, my
GitHub username is `berquist`, and I've created a `qchem` branch.

Eric

On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
wrote:

> Hi Martin,
>
> Nice to hear from you. We actually upport ORCA in the newest release of
> cclib,
> which would be 1.2 -- have you looked at the repostiory at
> https://github.com/cclib/cclib ?
> Of course, there is still some work to do for ORCA.
>
> As far as Q-Chem is concerned, I am fairly certain someone else has
> recently
> expressed interest in contributing to that (on this ML), so you would not
> be alone.
> Contributing unit test output is already tremendous, since it gives a
> starting
> point to get the parser working.
>
> In any case, you are more than welcome to fork from our git repository and
> put in
> pull requests. In fact, we would be tremendously happy to merge new
> features.
>
> Developmentwise, cclib has been quiet during this summer.
>
> You might also want to look at the new docs: http://cclib.github.io/
>
> The sourceforge website and wiki is now obsolete, and will probably be
> soon offline.
>
> Best,
> Karol
>
> On Aug 12 2014, Martin Blood-Forsythe wrote:
> > Hi cclib developers,
> > I'm interested in contributing towards the efforts to get Q-Chem and ORCA
> > parsers supported in cclib.  I'd be happy to contribute unit test output
> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> > experience with writing python parsers for both the text-based and binary
> > output files from both ORCA and Q-Chem, so I'm happy to help in any way I
> > can.
> > Best wishes,
> > -Martin
> >
> > ---------------------------------------------------
> > Martin Blood-Forsythe
> > Graduate Student in Physics
> > Harvard University
>
> >
> ------------------------------------------------------------------------------
>
> > _______________________________________________
> > cclib-devel mailing list
> > ccl...@li...
> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>
>
> --
> written by Karol M. Langner
> Thu Aug 14 22:06:25 EDT 2014
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

Sounds great.  I'll take a look at your qchem branch and see if there are
areas that I think I can add to.  Would there be interest in my adding
parsers for some quantities directly from the binary save files that qchem
leaves around (with an if file exists condition of course)? The primary
advantage of this is getting everything in float precision, but it can also
capture output that the user did not request be printed to the .out file.

-Martin


---------------------------------------------------
Martin Blood-Forsythe
Graduate Student in Physics
Harvard University
Aspuru-Guzik Lab


On Mon, Aug 18, 2014 at 10:14 AM, Eric Berquist <er...@pi...> wrote:

> Hi Karol and Martin,
>
> My Q-Chem parser is currently working on many "difficult" files in
> addition to some tests that I've made. It hasn't been merged in yet since
> the Travis CI build fails for a silly reason, which I will get to by the
> end of the week. It has to do with there being no regression folder/data in
> the cclib_data repository; I'm going to find a trouble file somewhere and
> add that.
>
> If anyone would like to test out the parser and submit pull requests, my
> GitHub username is `berquist`, and I've created a `qchem` branch.
>
> Eric
>
> On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
> wrote:
>
>> Hi Martin,
>>
>> Nice to hear from you. We actually upport ORCA in the newest release of
>> cclib,
>> which would be 1.2 -- have you looked at the repostiory at
>> https://github.com/cclib/cclib ?
>> Of course, there is still some work to do for ORCA.
>>
>> As far as Q-Chem is concerned, I am fairly certain someone else has
>> recently
>> expressed interest in contributing to that (on this ML), so you would not
>> be alone.
>> Contributing unit test output is already tremendous, since it gives a
>> starting
>> point to get the parser working.
>>
>> In any case, you are more than welcome to fork from our git repository
>> and put in
>> pull requests. In fact, we would be tremendously happy to merge new
>> features.
>>
>> Developmentwise, cclib has been quiet during this summer.
>>
>> You might also want to look at the new docs: http://cclib.github.io/
>>
>> The sourceforge website and wiki is now obsolete, and will probably be
>> soon offline.
>>
>> Best,
>> Karol
>>
>> On Aug 12 2014, Martin Blood-Forsythe wrote:
>> > Hi cclib developers,
>> > I'm interested in contributing towards the efforts to get Q-Chem and
>> ORCA
>> > parsers supported in cclib.  I'd be happy to contribute unit test output
>> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
>> > experience with writing python parsers for both the text-based and
>> binary
>> > output files from both ORCA and Q-Chem, so I'm happy to help in any way
>> I
>> > can.
>> > Best wishes,
>> > -Martin
>> >
>> > ---------------------------------------------------
>> > Martin Blood-Forsythe
>> > Graduate Student in Physics
>> > Harvard University
>>
>> >
>> ------------------------------------------------------------------------------
>>
>> > _______________________________________________
>> > cclib-devel mailing list
>> > ccl...@li...
>> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>
>>
>> --
>> written by Karol M. Langner
>> Thu Aug 14 22:06:25 EDT 2014
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> cclib-devel mailing list
>> ccl...@li...
>> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol L. <kar...@gm...>]

Well, we do support parsing multiple files for one job, although this has
been limited so far to Molpro if I recall correctly (it was probably also
in the works for Turbomole, but this is not an officially supported parser
ATM). We have not been parsing binary files at all, though, so that might
involve a major change in the way cclib works. It is worth doing, however,
or the reasons you mentioned as well as others.

Karol


On Mon, Aug 18, 2014 at 10:44 AM, Martin Blood-Forsythe <
mbl...@ph...> wrote:

> Sounds great.  I'll take a look at your qchem branch and see if there are
> areas that I think I can add to.  Would there be interest in my adding
> parsers for some quantities directly from the binary save files that qchem
> leaves around (with an if file exists condition of course)? The primary
> advantage of this is getting everything in float precision, but it can also
> capture output that the user did not request be printed to the .out file.
>
> -Martin
>
>
> ---------------------------------------------------
> Martin Blood-Forsythe
> Graduate Student in Physics
> Harvard University
> Aspuru-Guzik Lab
>
>
> On Mon, Aug 18, 2014 at 10:14 AM, Eric Berquist <er...@pi...> wrote:
>
>> Hi Karol and Martin,
>>
>> My Q-Chem parser is currently working on many "difficult" files in
>> addition to some tests that I've made. It hasn't been merged in yet since
>> the Travis CI build fails for a silly reason, which I will get to by the
>> end of the week. It has to do with there being no regression folder/data in
>> the cclib_data repository; I'm going to find a trouble file somewhere and
>> add that.
>>
>> If anyone would like to test out the parser and submit pull requests, my
>> GitHub username is `berquist`, and I've created a `qchem` branch.
>>
>> Eric
>>
>> On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
>> wrote:
>>
>>> Hi Martin,
>>>
>>> Nice to hear from you. We actually upport ORCA in the newest release of
>>> cclib,
>>> which would be 1.2 -- have you looked at the repostiory at
>>> https://github.com/cclib/cclib ?
>>> Of course, there is still some work to do for ORCA.
>>>
>>> As far as Q-Chem is concerned, I am fairly certain someone else has
>>> recently
>>> expressed interest in contributing to that (on this ML), so you would
>>> not be alone.
>>> Contributing unit test output is already tremendous, since it gives a
>>> starting
>>> point to get the parser working.
>>>
>>> In any case, you are more than welcome to fork from our git repository
>>> and put in
>>> pull requests. In fact, we would be tremendously happy to merge new
>>> features.
>>>
>>> Developmentwise, cclib has been quiet during this summer.
>>>
>>> You might also want to look at the new docs: http://cclib.github.io/
>>>
>>> The sourceforge website and wiki is now obsolete, and will probably be
>>> soon offline.
>>>
>>> Best,
>>> Karol
>>>
>>> On Aug 12 2014, Martin Blood-Forsythe wrote:
>>> > Hi cclib developers,
>>> > I'm interested in contributing towards the efforts to get Q-Chem and
>>> ORCA
>>> > parsers supported in cclib.  I'd be happy to contribute unit test
>>> output
>>> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
>>> > experience with writing python parsers for both the text-based and
>>> binary
>>> > output files from both ORCA and Q-Chem, so I'm happy to help in any
>>> way I
>>> > can.
>>> > Best wishes,
>>> > -Martin
>>> >
>>> > ---------------------------------------------------
>>> > Martin Blood-Forsythe
>>> > Graduate Student in Physics
>>> > Harvard University
>>>
>>> >
>>> ------------------------------------------------------------------------------
>>>
>>> > _______________________________________________
>>> > cclib-devel mailing list
>>> > ccl...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>>
>>>
>>> --
>>> written by Karol M. Langner
>>> Thu Aug 14 22:06:25 EDT 2014
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> cclib-devel mailing list
>>> ccl...@li...
>>> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>>
>>
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

I'll have to look at the way that the multiple file parse is implemented
for Molpro, but at least for ORCA and Q-Chem adding a binary file parser
wouldn't require much different than the way a text based multiple file
parsing works. I've had a lot of success parsing these files using
numpy.fromfile and numpy.frombuffer.  Typically the only required knowledge
is the number of orbitals (to determine the array shape to dump into) and
the file name.

-Martin




On Mon, Aug 18, 2014 at 1:52 PM, Karol Langner <kar...@gm...>
wrote:

> Well, we do support parsing multiple files for one job, although this has
> been limited so far to Molpro if I recall correctly (it was probably also
> in the works for Turbomole, but this is not an officially supported parser
> ATM). We have not been parsing binary files at all, though, so that might
> involve a major change in the way cclib works. It is worth doing, however,
> or the reasons you mentioned as well as others.
>
> Karol
>
>
> On Mon, Aug 18, 2014 at 10:44 AM, Martin Blood-Forsythe <
> mbl...@ph...> wrote:
>
>> Sounds great.  I'll take a look at your qchem branch and see if there are
>> areas that I think I can add to.  Would there be interest in my adding
>> parsers for some quantities directly from the binary save files that qchem
>> leaves around (with an if file exists condition of course)? The primary
>> advantage of this is getting everything in float precision, but it can also
>> capture output that the user did not request be printed to the .out file.
>>
>> -Martin
>>
>>
>> ---------------------------------------------------
>> Martin Blood-Forsythe
>> Graduate Student in Physics
>> Harvard University
>> Aspuru-Guzik Lab
>>
>>
>> On Mon, Aug 18, 2014 at 10:14 AM, Eric Berquist <er...@pi...> wrote:
>>
>>> Hi Karol and Martin,
>>>
>>> My Q-Chem parser is currently working on many "difficult" files in
>>> addition to some tests that I've made. It hasn't been merged in yet since
>>> the Travis CI build fails for a silly reason, which I will get to by the
>>> end of the week. It has to do with there being no regression folder/data in
>>> the cclib_data repository; I'm going to find a trouble file somewhere and
>>> add that.
>>>
>>> If anyone would like to test out the parser and submit pull requests, my
>>> GitHub username is `berquist`, and I've created a `qchem` branch.
>>>
>>> Eric
>>>
>>> On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
>>> wrote:
>>>
>>>> Hi Martin,
>>>>
>>>> Nice to hear from you. We actually upport ORCA in the newest release of
>>>> cclib,
>>>> which would be 1.2 -- have you looked at the repostiory at
>>>> https://github.com/cclib/cclib ?
>>>> Of course, there is still some work to do for ORCA.
>>>>
>>>> As far as Q-Chem is concerned, I am fairly certain someone else has
>>>> recently
>>>> expressed interest in contributing to that (on this ML), so you would
>>>> not be alone.
>>>> Contributing unit test output is already tremendous, since it gives a
>>>> starting
>>>> point to get the parser working.
>>>>
>>>> In any case, you are more than welcome to fork from our git repository
>>>> and put in
>>>> pull requests. In fact, we would be tremendously happy to merge new
>>>> features.
>>>>
>>>> Developmentwise, cclib has been quiet during this summer.
>>>>
>>>> You might also want to look at the new docs: http://cclib.github.io/
>>>>
>>>> The sourceforge website and wiki is now obsolete, and will probably be
>>>> soon offline.
>>>>
>>>> Best,
>>>> Karol
>>>>
>>>> On Aug 12 2014, Martin Blood-Forsythe wrote:
>>>> > Hi cclib developers,
>>>> > I'm interested in contributing towards the efforts to get Q-Chem and
>>>> ORCA
>>>> > parsers supported in cclib.  I'd be happy to contribute unit test
>>>> output
>>>> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
>>>> > experience with writing python parsers for both the text-based and
>>>> binary
>>>> > output files from both ORCA and Q-Chem, so I'm happy to help in any
>>>> way I
>>>> > can.
>>>> > Best wishes,
>>>> > -Martin
>>>> >
>>>> > ---------------------------------------------------
>>>> > Martin Blood-Forsythe
>>>> > Graduate Student in Physics
>>>> > Harvard University
>>>>
>>>> >
>>>> ------------------------------------------------------------------------------
>>>>
>>>> > _______________________________________________
>>>> > cclib-devel mailing list
>>>> > ccl...@li...
>>>> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>>>
>>>>
>>>> --
>>>> written by Karol M. Langner
>>>> Thu Aug 14 22:06:25 EDT 2014
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> _______________________________________________
>>>> cclib-devel mailing list
>>>> ccl...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>>>>
>>>
>>
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

Hi Martin,

That's true, technically it is not hard. I was thinking more about integration
with the rest of cclib. Binary files will not be "parsed" in the sense logfiles
are, that is the task is not one of reading a text file line by line and
extracting useful information. Rather, it is one of reading data that is stored
in a predefined format. This is obvious, but it implies some design choices
that have not been discussed here. Above all, the line-by-line paradigm
used in the class LogFile (inherited by all parsers) is not useful and
we cannot just pass one extra file.

Anyway, this can definitely be done, I'm just saying it needs a little
bit of forethought.

Cheers,
Karol

On Aug 18 2014, Martin Blood-Forsythe wrote:
> I'll have to look at the way that the multiple file parse is implemented
> for Molpro, but at least for ORCA and Q-Chem adding a binary file parser
> wouldn't require much different than the way a text based multiple file
> parsing works. I've had a lot of success parsing these files using
> numpy.fromfile and numpy.frombuffer.  Typically the only required knowledge
> is the number of orbitals (to determine the array shape to dump into) and
> the file name.
> 
> -Martin
> 
> 
> 
> 
> On Mon, Aug 18, 2014 at 1:52 PM, Karol Langner <kar...@gm...>
> wrote:
> 
> > Well, we do support parsing multiple files for one job, although this has
> > been limited so far to Molpro if I recall correctly (it was probably also
> > in the works for Turbomole, but this is not an officially supported parser
> > ATM). We have not been parsing binary files at all, though, so that might
> > involve a major change in the way cclib works. It is worth doing, however,
> > or the reasons you mentioned as well as others.
> >
> > Karol
> >
> >
> > On Mon, Aug 18, 2014 at 10:44 AM, Martin Blood-Forsythe <
> > mbl...@ph...> wrote:
> >
> >> Sounds great.  I'll take a look at your qchem branch and see if there are
> >> areas that I think I can add to.  Would there be interest in my adding
> >> parsers for some quantities directly from the binary save files that qchem
> >> leaves around (with an if file exists condition of course)? The primary
> >> advantage of this is getting everything in float precision, but it can also
> >> capture output that the user did not request be printed to the .out file.
> >>
> >> -Martin
> >>
> >>
> >> ---------------------------------------------------
> >> Martin Blood-Forsythe
> >> Graduate Student in Physics
> >> Harvard University
> >> Aspuru-Guzik Lab
> >>
> >>
> >> On Mon, Aug 18, 2014 at 10:14 AM, Eric Berquist <er...@pi...> wrote:
> >>
> >>> Hi Karol and Martin,
> >>>
> >>> My Q-Chem parser is currently working on many "difficult" files in
> >>> addition to some tests that I've made. It hasn't been merged in yet since
> >>> the Travis CI build fails for a silly reason, which I will get to by the
> >>> end of the week. It has to do with there being no regression folder/data in
> >>> the cclib_data repository; I'm going to find a trouble file somewhere and
> >>> add that.
> >>>
> >>> If anyone would like to test out the parser and submit pull requests, my
> >>> GitHub username is `berquist`, and I've created a `qchem` branch.
> >>>
> >>> Eric
> >>>
> >>> On Thursday, August 14, 2014, Karol M. Langner <kar...@gm...>
> >>> wrote:
> >>>
> >>>> Hi Martin,
> >>>>
> >>>> Nice to hear from you. We actually upport ORCA in the newest release of
> >>>> cclib,
> >>>> which would be 1.2 -- have you looked at the repostiory at
> >>>> https://github.com/cclib/cclib ?
> >>>> Of course, there is still some work to do for ORCA.
> >>>>
> >>>> As far as Q-Chem is concerned, I am fairly certain someone else has
> >>>> recently
> >>>> expressed interest in contributing to that (on this ML), so you would
> >>>> not be alone.
> >>>> Contributing unit test output is already tremendous, since it gives a
> >>>> starting
> >>>> point to get the parser working.
> >>>>
> >>>> In any case, you are more than welcome to fork from our git repository
> >>>> and put in
> >>>> pull requests. In fact, we would be tremendously happy to merge new
> >>>> features.
> >>>>
> >>>> Developmentwise, cclib has been quiet during this summer.
> >>>>
> >>>> You might also want to look at the new docs: http://cclib.github.io/
> >>>>
> >>>> The sourceforge website and wiki is now obsolete, and will probably be
> >>>> soon offline.
> >>>>
> >>>> Best,
> >>>> Karol
> >>>>
> >>>> On Aug 12 2014, Martin Blood-Forsythe wrote:
> >>>> > Hi cclib developers,
> >>>> > I'm interested in contributing towards the efforts to get Q-Chem and
> >>>> ORCA
> >>>> > parsers supported in cclib.  I'd be happy to contribute unit test
> >>>> output
> >>>> > files for ORCA 2.9.1, 3.0.1, 3.0.2, and Q-Chem 4.1.2.  I have some
> >>>> > experience with writing python parsers for both the text-based and
> >>>> binary
> >>>> > output files from both ORCA and Q-Chem, so I'm happy to help in any
> >>>> way I
> >>>> > can.
> >>>> > Best wishes,
> >>>> > -Martin
> >>>> >
> >>>> > ---------------------------------------------------
> >>>> > Martin Blood-Forsythe
> >>>> > Graduate Student in Physics
> >>>> > Harvard University
> >>>>
> >>>> >
> >>>> ------------------------------------------------------------------------------
> >>>>
> >>>> > _______________________________________________
> >>>> > cclib-devel mailing list
> >>>> > ccl...@li...
> >>>> > https://lists.sourceforge.net/lists/listinfo/cclib-devel
> >>>>
> >>>>
> >>>> --
> >>>> written by Karol M. Langner
> >>>> Thu Aug 14 22:06:25 EDT 2014
> >>>>
> >>>>
> >>>> ------------------------------------------------------------------------------
> >>>> _______________________________________________
> >>>> cclib-devel mailing list
> >>>> ccl...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/cclib-devel
> >>>>
> >>>
> >>
> >

-- 
written by Karol M. Langner
Mon Aug 18 20:35:25 EDT 2014

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

Here is the issue I created for this: https://github.com/cclib/cclib/issues/120

On Aug 18 2014, Karol M. Langner wrote:
> Hi Martin,
> 
> That's true, technically it is not hard. I was thinking more about integration
> with the rest of cclib. Binary files will not be "parsed" in the sense logfiles
> are, that is the task is not one of reading a text file line by line and
> extracting useful information. Rather, it is one of reading data that is stored
> in a predefined format. This is obvious, but it implies some design choices
> that have not been discussed here. Above all, the line-by-line paradigm
> used in the class LogFile (inherited by all parsers) is not useful and
> we cannot just pass one extra file.
> 
> Anyway, this can definitely be done, I'm just saying it needs a little
> bit of forethought.
> 
> Cheers,
> Karol
> 
> On Aug 18 2014, Martin Blood-Forsythe wrote:
> > I'll have to look at the way that the multiple file parse is implemented
> > for Molpro, but at least for ORCA and Q-Chem adding a binary file parser
> > wouldn't require much different than the way a text based multiple file
> > parsing works. I've had a lot of success parsing these files using
> > numpy.fromfile and numpy.frombuffer.  Typically the only required knowledge
> > is the number of orbitals (to determine the array shape to dump into) and
> > the file name.
> > 
> > -Martin

-- 
written by Karol M. Langner
Tue Aug 19 08:20:53 EDT 2014

Re: [cclib-devel] Q-Chem and ORCA parsers

[Eric B. <er...@pi...>]

This would certainly be interesting, but I have a few comments that are
probably Q-Chem specific:

1. As Martin already mentioned, unless you modify the `qchem` driver script
or you pass the flag `-save` when running, the folder with all of these
binary files gets deleted. This means they usually aren't present, except
when you want them, in contrast to the regular plain-text output file which
is always produced. This is in contrast to something like the HESS file,
which is only present when performing an analytic Hessian calculation. ORCA
is even worse about this, since it deletes scratch/temp files early and
often.

2. Since almost all of these files are named with a number, anyone who is
not a Q-Chem developer probably doesn't know what's in these files. For
example, 53 corresponds to the MO coefficients. Even knowing the
number/content mapping doesn't easily tell you the dimensionality of the
file; you have to know what all of the reads and writes to that file look
like.

2b. Once you know the number/content mapping and the dimensionality, there
is still no guarantee about what's in the file, due to nasty bits in the
source code with people using files for things other than what they are
specified as containing.

If one wanted to make a Q-Chem-specific parser for a bunch of binary files,
it would need to be maintained by someone who is a developer and knows
where all of the reads/writes to that file are, but even then it's quite
risky. You just can't perform a visual spot-check on them to see if there
are problems. It's definitely functionality for expert users.

It seems as though this is something that should either be in a separate
package that depends on cclib, or at least is separate from the parsers/
directory.

Martin, do you have scripts for parsing any of these that are publicly
viewable? I'd be interested to see how you do it. I'd really love to know
the structure of ORCA's *.gbw files.

Eric


On Tue, Aug 19, 2014 at 8:21 AM, Karol M. Langner <kar...@gm...>
wrote:

> Here is the issue I created for this:
> https://github.com/cclib/cclib/issues/120
>
> On Aug 18 2014, Karol M. Langner wrote:
> > Hi Martin,
> >
> > That's true, technically it is not hard. I was thinking more about
> integration
> > with the rest of cclib. Binary files will not be "parsed" in the sense
> logfiles
> > are, that is the task is not one of reading a text file line by line and
> > extracting useful information. Rather, it is one of reading data that is
> stored
> > in a predefined format. This is obvious, but it implies some design
> choices
> > that have not been discussed here. Above all, the line-by-line paradigm
> > used in the class LogFile (inherited by all parsers) is not useful and
> > we cannot just pass one extra file.
> >
> > Anyway, this can definitely be done, I'm just saying it needs a little
> > bit of forethought.
> >
> > Cheers,
> > Karol
> >
> > On Aug 18 2014, Martin Blood-Forsythe wrote:
> > > I'll have to look at the way that the multiple file parse is
> implemented
> > > for Molpro, but at least for ORCA and Q-Chem adding a binary file
> parser
> > > wouldn't require much different than the way a text based multiple file
> > > parsing works. I've had a lot of success parsing these files using
> > > numpy.fromfile and numpy.frombuffer.  Typically the only required
> knowledge
> > > is the number of orbitals (to determine the array shape to dump into)
> and
> > > the file name.
> > >
> > > -Martin
>
> --
> written by Karol M. Langner
> Tue Aug 19 08:20:53 EDT 2014
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Martin Blood-F. <mbl...@ph...>]

1. Yes it is true that most codes delete their binary files unless you tell
them not to.  Which is why I was mentioning the need to be careful about
checking for the existence of a file. ORCA is actually a bit unusual in the
regard that it always leaves the *.gbw binary file around.  I don't know
the *entire* contents of the file, but I know how to parse the MO
coefficients/energies, occupation numbers, job name, and perhaps orbital
labels (1pz 2s etc, the offsets to these are a little tricky since it is a
mixed binary/ascii format). As far as I know, the *.hess file is a text
file, but the point about job specific binaries, like the *.cis file, is
certainly correct.

2 . With regard to Q-Chem's custom binary files:
There are two different categories of binary files.  The ones that get
saved if you do 'qchem infile outfile savename' and the more complete set
that are stored if you do the '-save' option.
 For the most part I think that the standard ones saved without the '-save'
command are pretty safe to trust. These include the 53.0 [MO coefficients +
eigenvalues] and 54.0 [density matrix].  There are a few other binary files
that I trust: 320.0 [overlap matrix], 601.0 [NBO coefficients].
The general point though that binary files are a pain and can require some
developer level knowledge is well taken. One reason that I thought adding
binary parsing support might be neat is exactly because non-developers have
no idea what is in these files.

2b. The point about developers misusing the binary files in Q-chem is
certainly a good one, and is a strong reason to not support parsing any of
the non-vanilla binary files.  I'd argue that the 53.0 and 54.0 files are
quite safe to parse since they have a very well defined structure that
hasn't changed across versions 3 - 4.2 as far as I can tell.  Their content
mapping is very easy and only requires knowing the total number of orbitals.

3. I agree that any package that attempts to unpack a whole bunch of other
binary files should probably be separate from cclib.  Mainly I was
wondering if the MO coefficients & eigenvalues would be of interest for
visualization purposes.  The tricky part there is that a lot of codes use
non-standard basis set definitions that differ from the EMSL specifications
of the same name.

4. My scripts that unpack *.gbw and Q-Chem binaries are currently wrapped
up in with some other code.  I strip them out and add them to
https://github.com/mbloodforsythe/qc-binary-parsers


---------------------------------------------------
Martin Blood-Forsythe
Graduate Student in Physics
Harvard University
Aspuru-Guzik Lab


On Tue, Aug 19, 2014 at 12:57 PM, Eric Berquist <er...@pi...> wrote:

> This would certainly be interesting, but I have a few comments that are
> probably Q-Chem specific:
>
> 1. As Martin already mentioned, unless you modify the `qchem` driver
> script or you pass the flag `-save` when running, the folder with all of
> these binary files gets deleted. This means they usually aren't present,
> except when you want them, in contrast to the regular plain-text output
> file which is always produced. This is in contrast to something like the
> HESS file, which is only present when performing an analytic Hessian
> calculation. ORCA is even worse about this, since it deletes scratch/temp
> files early and often.
>
> 2. Since almost all of these files are named with a number, anyone who is
> not a Q-Chem developer probably doesn't know what's in these files. For
> example, 53 corresponds to the MO coefficients. Even knowing the
> number/content mapping doesn't easily tell you the dimensionality of the
> file; you have to know what all of the reads and writes to that file look
> like.
>
> 2b. Once you know the number/content mapping and the dimensionality, there
> is still no guarantee about what's in the file, due to nasty bits in the
> source code with people using files for things other than what they are
> specified as containing.
>
> If one wanted to make a Q-Chem-specific parser for a bunch of binary
> files, it would need to be maintained by someone who is a developer and
> knows where all of the reads/writes to that file are, but even then it's
> quite risky. You just can't perform a visual spot-check on them to see if
> there are problems. It's definitely functionality for expert users.
>
> It seems as though this is something that should either be in a separate
> package that depends on cclib, or at least is separate from the parsers/
> directory.
>
> Martin, do you have scripts for parsing any of these that are publicly
> viewable? I'd be interested to see how you do it. I'd really love to know
> the structure of ORCA's *.gbw files.
>
> Eric
>
>
> On Tue, Aug 19, 2014 at 8:21 AM, Karol M. Langner <kar...@gm...
> > wrote:
>
>> Here is the issue I created for this:
>> https://github.com/cclib/cclib/issues/120
>>
>> On Aug 18 2014, Karol M. Langner wrote:
>> > Hi Martin,
>> >
>> > That's true, technically it is not hard. I was thinking more about
>> integration
>> > with the rest of cclib. Binary files will not be "parsed" in the sense
>> logfiles
>> > are, that is the task is not one of reading a text file line by line and
>> > extracting useful information. Rather, it is one of reading data that
>> is stored
>> > in a predefined format. This is obvious, but it implies some design
>> choices
>> > that have not been discussed here. Above all, the line-by-line paradigm
>> > used in the class LogFile (inherited by all parsers) is not useful and
>> > we cannot just pass one extra file.
>> >
>> > Anyway, this can definitely be done, I'm just saying it needs a little
>> > bit of forethought.
>> >
>> > Cheers,
>> > Karol
>> >
>> > On Aug 18 2014, Martin Blood-Forsythe wrote:
>> > > I'll have to look at the way that the multiple file parse is
>> implemented
>> > > for Molpro, but at least for ORCA and Q-Chem adding a binary file
>> parser
>> > > wouldn't require much different than the way a text based multiple
>> file
>> > > parsing works. I've had a lot of success parsing these files using
>> > > numpy.fromfile and numpy.frombuffer.  Typically the only required
>> knowledge
>> > > is the number of orbitals (to determine the array shape to dump into)
>> and
>> > > the file name.
>> > >
>> > > -Martin
>>
>> --
>> written by Karol M. Langner
>> Tue Aug 19 08:20:53 EDT 2014
>>
>
>

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol M. L. <kar...@gm...>]

All good counterpoints, the usual show stoppers in this context. And I like
the idea of gathering the tools for reading binary files in a separate code
base, perhaps using/requiring data from the logfile, because cclib is now
somewhat mature and has a well-defined scope. Still, we could keep it
within the cclib project on github, and call it cclib-binary or whatever.

@Noel @Adam -- any thoughts on this topic?

On Aug 19 2014, Martin Blood-Forsythe wrote:
> 1. Yes it is true that most codes delete their binary files unless you tell
> them not to.  Which is why I was mentioning the need to be careful about
> checking for the existence of a file. ORCA is actually a bit unusual in the
> regard that it always leaves the *.gbw binary file around.  I don't know
> the *entire* contents of the file, but I know how to parse the MO
> coefficients/energies, occupation numbers, job name, and perhaps orbital
> labels (1pz 2s etc, the offsets to these are a little tricky since it is a
> mixed binary/ascii format). As far as I know, the *.hess file is a text
> file, but the point about job specific binaries, like the *.cis file, is
> certainly correct.
> 
> 2 . With regard to Q-Chem's custom binary files:
> There are two different categories of binary files.  The ones that get
> saved if you do 'qchem infile outfile savename' and the more complete set
> that are stored if you do the '-save' option.
>  For the most part I think that the standard ones saved without the '-save'
> command are pretty safe to trust. These include the 53.0 [MO coefficients +
> eigenvalues] and 54.0 [density matrix].  There are a few other binary files
> that I trust: 320.0 [overlap matrix], 601.0 [NBO coefficients].
> The general point though that binary files are a pain and can require some
> developer level knowledge is well taken. One reason that I thought adding
> binary parsing support might be neat is exactly because non-developers have
> no idea what is in these files.
> 
> 2b. The point about developers misusing the binary files in Q-chem is
> certainly a good one, and is a strong reason to not support parsing any of
> the non-vanilla binary files.  I'd argue that the 53.0 and 54.0 files are
> quite safe to parse since they have a very well defined structure that
> hasn't changed across versions 3 - 4.2 as far as I can tell.  Their content
> mapping is very easy and only requires knowing the total number of orbitals.
> 
> 3. I agree that any package that attempts to unpack a whole bunch of other
> binary files should probably be separate from cclib.  Mainly I was
> wondering if the MO coefficients & eigenvalues would be of interest for
> visualization purposes.  The tricky part there is that a lot of codes use
> non-standard basis set definitions that differ from the EMSL specifications
> of the same name.
> 
> 4. My scripts that unpack *.gbw and Q-Chem binaries are currently wrapped
> up in with some other code.  I strip them out and add them to
> https://github.com/mbloodforsythe/qc-binary-parsers
> 
> 
> ---------------------------------------------------
> Martin Blood-Forsythe
> Graduate Student in Physics
> Harvard University
> Aspuru-Guzik Lab
> 
> 
> On Tue, Aug 19, 2014 at 12:57 PM, Eric Berquist <er...@pi...> wrote:
> 
> > This would certainly be interesting, but I have a few comments that are
> > probably Q-Chem specific:
> >
> > 1. As Martin already mentioned, unless you modify the `qchem` driver
> > script or you pass the flag `-save` when running, the folder with all of
> > these binary files gets deleted. This means they usually aren't present,
> > except when you want them, in contrast to the regular plain-text output
> > file which is always produced. This is in contrast to something like the
> > HESS file, which is only present when performing an analytic Hessian
> > calculation. ORCA is even worse about this, since it deletes scratch/temp
> > files early and often.
> >
> > 2. Since almost all of these files are named with a number, anyone who is
> > not a Q-Chem developer probably doesn't know what's in these files. For
> > example, 53 corresponds to the MO coefficients. Even knowing the
> > number/content mapping doesn't easily tell you the dimensionality of the
> > file; you have to know what all of the reads and writes to that file look
> > like.
> >
> > 2b. Once you know the number/content mapping and the dimensionality, there
> > is still no guarantee about what's in the file, due to nasty bits in the
> > source code with people using files for things other than what they are
> > specified as containing.
> >
> > If one wanted to make a Q-Chem-specific parser for a bunch of binary
> > files, it would need to be maintained by someone who is a developer and
> > knows where all of the reads/writes to that file are, but even then it's
> > quite risky. You just can't perform a visual spot-check on them to see if
> > there are problems. It's definitely functionality for expert users.
> >
> > It seems as though this is something that should either be in a separate
> > package that depends on cclib, or at least is separate from the parsers/
> > directory.
> >
> > Martin, do you have scripts for parsing any of these that are publicly
> > viewable? I'd be interested to see how you do it. I'd really love to know
> > the structure of ORCA's *.gbw files.
> >
> > Eric
> >
> >
> > On Tue, Aug 19, 2014 at 8:21 AM, Karol M. Langner <kar...@gm...
> > > wrote:
> >
> >> Here is the issue I created for this:
> >> https://github.com/cclib/cclib/issues/120
> >>
> >> On Aug 18 2014, Karol M. Langner wrote:
> >> > Hi Martin,
> >> >
> >> > That's true, technically it is not hard. I was thinking more about
> >> integration
> >> > with the rest of cclib. Binary files will not be "parsed" in the sense
> >> logfiles
> >> > are, that is the task is not one of reading a text file line by line and
> >> > extracting useful information. Rather, it is one of reading data that
> >> is stored
> >> > in a predefined format. This is obvious, but it implies some design
> >> choices
> >> > that have not been discussed here. Above all, the line-by-line paradigm
> >> > used in the class LogFile (inherited by all parsers) is not useful and
> >> > we cannot just pass one extra file.
> >> >
> >> > Anyway, this can definitely be done, I'm just saying it needs a little
> >> > bit of forethought.
> >> >
> >> > Cheers,
> >> > Karol
> >> >
> >> > On Aug 18 2014, Martin Blood-Forsythe wrote:
> >> > > I'll have to look at the way that the multiple file parse is
> >> implemented
> >> > > for Molpro, but at least for ORCA and Q-Chem adding a binary file
> >> parser
> >> > > wouldn't require much different than the way a text based multiple
> >> file
> >> > > parsing works. I've had a lot of success parsing these files using
> >> > > numpy.fromfile and numpy.frombuffer.  Typically the only required
> >> knowledge
> >> > > is the number of orbitals (to determine the array shape to dump into)
> >> and
> >> > > the file name.
> >> > >
> >> > > -Martin
> >>
> >> --
> >> written by Karol M. Langner
> >> Tue Aug 19 08:20:53 EDT 2014
> >>
> >
> >

-- 
written by Karol M. Langner
Tue Aug 19 17:00:06 EDT 2014

Re: [cclib-devel] Q-Chem and ORCA parsers

[Karol L. <kar...@gm...>]

I created an issue for this in cclib for future work:
https://github.com/cclib/cclib/issues/120


On Tue, Aug 19, 2014 at 5:06 PM, Karol M. Langner <kar...@gm...>
wrote:

> All good counterpoints, the usual show stoppers in this context. And I like
> the idea of gathering the tools for reading binary files in a separate code
> base, perhaps using/requiring data from the logfile, because cclib is now
> somewhat mature and has a well-defined scope. Still, we could keep it
> within the cclib project on github, and call it cclib-binary or whatever.
>
> @Noel @Adam -- any thoughts on this topic?
>
> On Aug 19 2014, Martin Blood-Forsythe wrote:
> > 1. Yes it is true that most codes delete their binary files unless you
> tell
> > them not to.  Which is why I was mentioning the need to be careful about
> > checking for the existence of a file. ORCA is actually a bit unusual in
> the
> > regard that it always leaves the *.gbw binary file around.  I don't know
> > the *entire* contents of the file, but I know how to parse the MO
> > coefficients/energies, occupation numbers, job name, and perhaps orbital
> > labels (1pz 2s etc, the offsets to these are a little tricky since it is
> a
> > mixed binary/ascii format). As far as I know, the *.hess file is a text
> > file, but the point about job specific binaries, like the *.cis file, is
> > certainly correct.
> >
> > 2 . With regard to Q-Chem's custom binary files:
> > There are two different categories of binary files.  The ones that get
> > saved if you do 'qchem infile outfile savename' and the more complete set
> > that are stored if you do the '-save' option.
> >  For the most part I think that the standard ones saved without the
> '-save'
> > command are pretty safe to trust. These include the 53.0 [MO
> coefficients +
> > eigenvalues] and 54.0 [density matrix].  There are a few other binary
> files
> > that I trust: 320.0 [overlap matrix], 601.0 [NBO coefficients].
> > The general point though that binary files are a pain and can require
> some
> > developer level knowledge is well taken. One reason that I thought adding
> > binary parsing support might be neat is exactly because non-developers
> have
> > no idea what is in these files.
> >
> > 2b. The point about developers misusing the binary files in Q-chem is
> > certainly a good one, and is a strong reason to not support parsing any
> of
> > the non-vanilla binary files.  I'd argue that the 53.0 and 54.0 files are
> > quite safe to parse since they have a very well defined structure that
> > hasn't changed across versions 3 - 4.2 as far as I can tell.  Their
> content
> > mapping is very easy and only requires knowing the total number of
> orbitals.
> >
> > 3. I agree that any package that attempts to unpack a whole bunch of
> other
> > binary files should probably be separate from cclib.  Mainly I was
> > wondering if the MO coefficients & eigenvalues would be of interest for
> > visualization purposes.  The tricky part there is that a lot of codes use
> > non-standard basis set definitions that differ from the EMSL
> specifications
> > of the same name.
> >
> > 4. My scripts that unpack *.gbw and Q-Chem binaries are currently wrapped
> > up in with some other code.  I strip them out and add them to
> > https://github.com/mbloodforsythe/qc-binary-parsers
> >
> >
> > ---------------------------------------------------
> > Martin Blood-Forsythe
> > Graduate Student in Physics
> > Harvard University
> > Aspuru-Guzik Lab
> >
> >
> > On Tue, Aug 19, 2014 at 12:57 PM, Eric Berquist <er...@pi...> wrote:
> >
> > > This would certainly be interesting, but I have a few comments that are
> > > probably Q-Chem specific:
> > >
> > > 1. As Martin already mentioned, unless you modify the `qchem` driver
> > > script or you pass the flag `-save` when running, the folder with all
> of
> > > these binary files gets deleted. This means they usually aren't
> present,
> > > except when you want them, in contrast to the regular plain-text output
> > > file which is always produced. This is in contrast to something like
> the
> > > HESS file, which is only present when performing an analytic Hessian
> > > calculation. ORCA is even worse about this, since it deletes
> scratch/temp
> > > files early and often.
> > >
> > > 2. Since almost all of these files are named with a number, anyone who
> is
> > > not a Q-Chem developer probably doesn't know what's in these files. For
> > > example, 53 corresponds to the MO coefficients. Even knowing the
> > > number/content mapping doesn't easily tell you the dimensionality of
> the
> > > file; you have to know what all of the reads and writes to that file
> look
> > > like.
> > >
> > > 2b. Once you know the number/content mapping and the dimensionality,
> there
> > > is still no guarantee about what's in the file, due to nasty bits in
> the
> > > source code with people using files for things other than what they are
> > > specified as containing.
> > >
> > > If one wanted to make a Q-Chem-specific parser for a bunch of binary
> > > files, it would need to be maintained by someone who is a developer and
> > > knows where all of the reads/writes to that file are, but even then
> it's
> > > quite risky. You just can't perform a visual spot-check on them to see
> if
> > > there are problems. It's definitely functionality for expert users.
> > >
> > > It seems as though this is something that should either be in a
> separate
> > > package that depends on cclib, or at least is separate from the
> parsers/
> > > directory.
> > >
> > > Martin, do you have scripts for parsing any of these that are publicly
> > > viewable? I'd be interested to see how you do it. I'd really love to
> know
> > > the structure of ORCA's *.gbw files.
> > >
> > > Eric
> > >
> > >
> > > On Tue, Aug 19, 2014 at 8:21 AM, Karol M. Langner <
> kar...@gm...
> > > > wrote:
> > >
> > >> Here is the issue I created for this:
> > >> https://github.com/cclib/cclib/issues/120
> > >>
> > >> On Aug 18 2014, Karol M. Langner wrote:
> > >> > Hi Martin,
> > >> >
> > >> > That's true, technically it is not hard. I was thinking more about
> > >> integration
> > >> > with the rest of cclib. Binary files will not be "parsed" in the
> sense
> > >> logfiles
> > >> > are, that is the task is not one of reading a text file line by
> line and
> > >> > extracting useful information. Rather, it is one of reading data
> that
> > >> is stored
> > >> > in a predefined format. This is obvious, but it implies some design
> > >> choices
> > >> > that have not been discussed here. Above all, the line-by-line
> paradigm
> > >> > used in the class LogFile (inherited by all parsers) is not useful
> and
> > >> > we cannot just pass one extra file.
> > >> >
> > >> > Anyway, this can definitely be done, I'm just saying it needs a
> little
> > >> > bit of forethought.
> > >> >
> > >> > Cheers,
> > >> > Karol
> > >> >
> > >> > On Aug 18 2014, Martin Blood-Forsythe wrote:
> > >> > > I'll have to look at the way that the multiple file parse is
> > >> implemented
> > >> > > for Molpro, but at least for ORCA and Q-Chem adding a binary file
> > >> parser
> > >> > > wouldn't require much different than the way a text based multiple
> > >> file
> > >> > > parsing works. I've had a lot of success parsing these files using
> > >> > > numpy.fromfile and numpy.frombuffer.  Typically the only required
> > >> knowledge
> > >> > > is the number of orbitals (to determine the array shape to dump
> into)
> > >> and
> > >> > > the file name.
> > >> > >
> > >> > > -Martin
> > >>
> > >> --
> > >> written by Karol M. Langner
> > >> Tue Aug 19 08:20:53 EDT 2014
> > >>
> > >
> > >
>
> --
> written by Karol M. Langner
> Tue Aug 19 17:00:06 EDT 2014
>