cclib-devel

[cclib-devel] Jaguar data files

[Karol L. <kar...@kn...>]

Hi all,

 It's been fairly quiet for some time now. Let me start things up again with a 
simple issue - why do we have three different basicJaguar* directories in 
trunk/data/? It seems logical to use data files for unittests only from the 
newest Jaguar version and to change the rest to regressions. That is the 
situation for Gaussian and ADF (although here basicADF2004 is not the newest 
version). Why should Jaguar be an exception?

Cheers,
Karol

-- 
written by Karol Langner
Thu Oct 11 23:59:44 EDT 2007

Re: [cclib-devel] Jaguar data files

[Noel O'B. <bao...@gm...>]

On 12/10/2007, Karol Langner <kar...@kn...> wrote:
> Hi all,
>
>  It's been fairly quiet for some time now.
I'm going to get out a release (beta?) of cclib 0.8 this weekend.
Would appreciate any relevant updates to the Changelog...

> Let me start things up again with a
> simple issue - why do we have three different basicJaguar* directories in
> trunk/data/? It seems logical to use data files for unittests only from the
> newest Jaguar version and to change the rest to regressions. That is the
> situation for Gaussian and ADF (although here basicADF2004 is not the newest
> version). Why should Jaguar be an exception?
If a newer version of Gaussian comes out or if we have access to a
newer version of ADF, Jaguar won't be the exception. I'm not sure what
you mean by changing the unittests to regressions. If we support
JaguarX, then we should have unittests to ensure this.

The historical reason we have unittests for different versions of
Jaguar is that my colleagues in Cambridge were using Jag4.2 (or so),
Adam's colleagues were using 6.?, and finally we got the latest Jaguar
6.5 (which is now superseded by Jag 7.0 - maybe we should ask for
this).

Regards,
   Noel (aka 'test nut')

> Cheers,
> Karol
>
> --
> written by Karol Langner
> Thu Oct 11 23:59:44 EDT 2007
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc.
> Still grepping through log files to find problems?  Stop.
> Now Search log events and configuration files using AJAX and a browser.
> Download your FREE copy of Splunk now >> http://get.splunk.com/
> _______________________________________________
> cclib-devel mailing list
> ccl...@li...
> https://lists.sourceforge.net/lists/listinfo/cclib-devel
>

Re: [cclib-devel] Jaguar data files

[Karol L. <kar...@kn...>]

On Friday 12 October 2007 03:07, you wrote:
> On 12/10/2007, Karol Langner <kar...@kn...> wrote:
> > Hi all,
> >
> >  It's been fairly quiet for some time now.
>
> I'm going to get out a release (beta?) of cclib 0.8 this weekend.
> Would appreciate any relevant updates to the Changelog...
I added as much as I could find after looking through the logs yesterday.

> > Let me start things up again with a
> > simple issue - why do we have three different basicJaguar* directories in
> > trunk/data/? It seems logical to use data files for unittests only from
> > the newest Jaguar version and to change the rest to regressions. That is
> > the situation for Gaussian and ADF (although here basicADF2004 is not the
> > newest version). Why should Jaguar be an exception?
>
> If a newer version of Gaussian comes out or if we have access to a
> newer version of ADF, Jaguar won't be the exception. I'm not sure what
> you mean by changing the unittests to regressions. If we support
> JaguarX, then we should have unittests to ensure this.
There are probably more people using Gaussian98 than there are using 
Jaguar4.2, so I don't see why we should only support older version of Jaguar 
and not Gaussian.

I was referring to data files rather than unittests, of course. What I mean 
be 'changing the unittests to regressions' is not distributing the older data 
files with cclib anymore and adding them to the family of regression tests.

I think it is sufficient and more efficient to have a good set of test log 
files only for the newest available version of all the programs, and to 
support all older versions with appropriate regression tests (works fine for 
Gaussian, GAMESS, ADF).

> The historical reason we have unittests for different versions of
> Jaguar is that my colleagues in Cambridge were using Jag4.2 (or so),
> Adam's colleagues were using 6.?, and finally we got the latest Jaguar
> 6.5 (which is now superseded by Jag 7.0 - maybe we should ask for
> this).
This is redundant, since then we will have as many basicJaguar* directories as 
there are Jaguar versions being used. Already, the Jaguar4.2 and Jaguar6.0 
tests are "behind" Jaguar6.5 since we don't have easy access to them. 

Notice also that the Jaguar6.0 data files take up alot more space than the 
Jaguar6.5 ones. I would prefer to see a complete set of log files for 
Jaguar6.5 and everything else put in directories that can be downloaded as 
regressions (so we can still check if they are parsed and pass the tests we 
want).

My two cents,
Karol

-- 
written by Karol Langner
Fri Oct 12 20:12:49 EDT 2007

Re: [cclib-devel] Jaguar data files

[Noel O'B. <bao...@gm...>]

On 13/10/2007, Karol Langner <kar...@kn...> wrote:
> On Friday 12 October 2007 03:07, you wrote:
> > On 12/10/2007, Karol Langner <kar...@kn...> wrote:
> > > Hi all,
> > >
> > >  It's been fairly quiet for some time now.
> >
> > I'm going to get out a release (beta?) of cclib 0.8 this weekend.
> > Would appreciate any relevant updates to the Changelog...
> I added as much as I could find after looking through the logs yesterday.

That's great. I made some minor edits but it looks pretty comprehensive now.

> > > Let me start things up again with a
> > > simple issue - why do we have three different basicJaguar* directories in
> > > trunk/data/? It seems logical to use data files for unittests only from
> > > the newest Jaguar version and to change the rest to regressions. That is
> > > the situation for Gaussian and ADF (although here basicADF2004 is not the
> > > newest version). Why should Jaguar be an exception?
> >
> > If a newer version of Gaussian comes out or if we have access to a
> > newer version of ADF, Jaguar won't be the exception. I'm not sure what
> > you mean by changing the unittests to regressions. If we support
> > JaguarX, then we should have unittests to ensure this.
> There are probably more people using Gaussian98 than there are using
> Jaguar4.2, so I don't see why we should only support older version of Jaguar
> and not Gaussian.

There's no reason. We just started supporting whatever programs we had
access to.

> I was referring to data files rather than unittests, of course. What I mean
> be 'changing the unittests to regressions' is not distributing the older data
> files with cclib anymore and adding them to the family of regression tests.
>
> I think it is sufficient and more efficient to have a good set of test log
> files only for the newest available version of all the programs, and to
> support all older versions with appropriate regression tests (works fine for
> Gaussian, GAMESS, ADF).

Right, okay - now I understand. Yes - that's fine by me. Probably the
easiest way to do this is simply to have the testall.py script (or
whatever) only run the tests if the test file exists. At the same
time, I can change manifest.py (or MANIFEST.in if we can make this
work in future) so that it only includes the latest files for each.

> > The historical reason we have unittests for different versions of
> > Jaguar is that my colleagues in Cambridge were using Jag4.2 (or so),
> > Adam's colleagues were using 6.?, and finally we got the latest Jaguar
> > 6.5 (which is now superseded by Jag 7.0 - maybe we should ask for
> > this).
> This is redundant, since then we will have as many basicJaguar* directories as
> there are Jaguar versions being used. Already, the Jaguar4.2 and Jaguar6.0
> tests are "behind" Jaguar6.5 since we don't have easy access to them.
>
> Notice also that the Jaguar6.0 data files take up alot more space than the
> Jaguar6.5 ones. I would prefer to see a complete set of log files for
> Jaguar6.5 and everything else put in directories that can be downloaded as
> regressions (so we can still check if they are parsed and pass the tests we
> want).

Yes - I think we agree.

> Karol
>
> --
> written by Karol Langner
> Fri Oct 12 20:12:49 EDT 2007
>

Re: [cclib-devel] Jaguar data files

[Karol L. <kar...@kn...>]

On Saturday 13 October 2007 08:37, Noel O'Boyle wrote:
> On 13/10/2007, Karol Langner <kar...@kn...> wrote:
> > > > Let me start things up again with a
> > > > simple issue - why do we have three different basicJaguar*
> > > > directories in trunk/data/? It seems logical to use data files for
> > > > unittests only from the newest Jaguar version and to change the rest
> > > > to regressions. That is the situation for Gaussian and ADF (although
> > > > here basicADF2004 is not the newest version). Why should Jaguar be an
> > > > exception?
> > >
> > > If a newer version of Gaussian comes out or if we have access to a
> > > newer version of ADF, Jaguar won't be the exception. I'm not sure what
> > > you mean by changing the unittests to regressions. If we support
> > > JaguarX, then we should have unittests to ensure this.
> >
> > There are probably more people using Gaussian98 than there are using
> > Jaguar4.2, so I don't see why we should only support older version of
> > Jaguar and not Gaussian.
>
> There's no reason. We just started supporting whatever programs we had
> access to.

Of course, and access changes and versions become old. The data files packaged 
along with cclib should reflect the current state of our access to the newest 
program versions.

> > I was referring to data files rather than unittests, of course. What I
> > mean be 'changing the unittests to regressions' is not distributing the
> > older data files with cclib anymore and adding them to the family of
> > regression tests.
> >
> > I think it is sufficient and more efficient to have a good set of test
> > log files only for the newest available version of all the programs, and
> > to support all older versions with appropriate regression tests (works
> > fine for Gaussian, GAMESS, ADF).
>
> Right, okay - now I understand. Yes - that's fine by me. Probably the
> easiest way to do this is simply to have the testall.py script (or
> whatever) only run the tests if the test file exists. At the same
> time, I can change manifest.py (or MANIFEST.in if we can make this
> work in future) so that it only includes the latest files for each.

The appropriate tests can be imported and run by regression.py to make them 
truly regressions.

> > > The historical reason we have unittests for different versions of
> > > Jaguar is that my colleagues in Cambridge were using Jag4.2 (or so),
> > > Adam's colleagues were using 6.?, and finally we got the latest Jaguar
> > > 6.5 (which is now superseded by Jag 7.0 - maybe we should ask for
> > > this).
> >
> > This is redundant, since then we will have as many basicJaguar*
> > directories as there are Jaguar versions being used. Already, the
> > Jaguar4.2 and Jaguar6.0 tests are "behind" Jaguar6.5 since we don't have
> > easy access to them.
> >
> > Notice also that the Jaguar6.0 data files take up alot more space than
> > the Jaguar6.5 ones. I would prefer to see a complete set of log files for
> > Jaguar6.5 and everything else put in directories that can be downloaded
> > as regressions (so we can still check if they are parsed and pass the
> > tests we want).
>
> Yes - I think we agree.

Is that a green light for doing this? Probably after the release, anyway.

As you mentioned, Jaguar7.0 is out. Now would be a good time to update the 
installation, which by the way has changed (the machine it was on is dead... 
there is a new one now) so I need to write about that later.

On a related note, ADF2007.01 has been released. We have all the tests in 
ADF2004.01 and should probably think about upgrading this also. I can rerun 
them in 2006.01 or 2007.01 - so the question is whether we want to update 
this soon and, if so, to which version?

Karol

-- 
written by Karol Langner
Fri Oct 19 01:04:14 EDT 2007

Re: [cclib-devel] Jaguar data files

[Noel O'B. <bao...@gm...>]

On 19/10/2007, Karol Langner <kar...@kn...> wrote:
> On Saturday 13 October 2007 08:37, Noel O'Boyle wrote:
> > On 13/10/2007, Karol Langner <kar...@kn...> wrote:
> > > > > Let me start things up again with a
> > > > > simple issue - why do we have three different basicJaguar*
> > > > > directories in trunk/data/? It seems logical to use data files for
> > > > > unittests only from the newest Jaguar version and to change the rest
> > > > > to regressions. That is the situation for Gaussian and ADF (although
> > > > > here basicADF2004 is not the newest version). Why should Jaguar be an
> > > > > exception?
> > > >
> > > > If a newer version of Gaussian comes out or if we have access to a
> > > > newer version of ADF, Jaguar won't be the exception. I'm not sure what
> > > > you mean by changing the unittests to regressions. If we support
> > > > JaguarX, then we should have unittests to ensure this.
> > >
> > > There are probably more people using Gaussian98 than there are using
> > > Jaguar4.2, so I don't see why we should only support older version of
> > > Jaguar and not Gaussian.
> >
> > There's no reason. We just started supporting whatever programs we had
> > access to.
>
> Of course, and access changes and versions become old. The data files packaged
> along with cclib should reflect the current state of our access to the newest
> program versions.

That's fine by me.

> > > I was referring to data files rather than unittests, of course. What I
> > > mean be 'changing the unittests to regressions' is not distributing the
> > > older data files with cclib anymore and adding them to the family of
> > > regression tests.
> > >
> > > I think it is sufficient and more efficient to have a good set of test
> > > log files only for the newest available version of all the programs, and
> > > to support all older versions with appropriate regression tests (works
> > > fine for Gaussian, GAMESS, ADF).
> >
> > Right, okay - now I understand. Yes - that's fine by me. Probably the
> > easiest way to do this is simply to have the testall.py script (or
> > whatever) only run the tests if the test file exists. At the same
> > time, I can change manifest.py (or MANIFEST.in if we can make this
> > work in future) so that it only includes the latest files for each.
>
> The appropriate tests can be imported and run by regression.py to make them
> truly regressions.

Right.

> > > > The historical reason we have unittests for different versions of
> > > > Jaguar is that my colleagues in Cambridge were using Jag4.2 (or so),
> > > > Adam's colleagues were using 6.?, and finally we got the latest Jaguar
> > > > 6.5 (which is now superseded by Jag 7.0 - maybe we should ask for
> > > > this).
> > >
> > > This is redundant, since then we will have as many basicJaguar*
> > > directories as there are Jaguar versions being used. Already, the
> > > Jaguar4.2 and Jaguar6.0 tests are "behind" Jaguar6.5 since we don't have
> > > easy access to them.
> > >
> > > Notice also that the Jaguar6.0 data files take up alot more space than
> > > the Jaguar6.5 ones. I would prefer to see a complete set of log files for
> > > Jaguar6.5 and everything else put in directories that can be downloaded
> > > as regressions (so we can still check if they are parsed and pass the
> > > tests we want).
> >
> > Yes - I think we agree.
>
> Is that a green light for doing this? Probably after the release, anyway.

Yes - it is. :-) I have realised that I have become too
conservative...I will talk about this after the final release.

> As you mentioned, Jaguar7.0 is out. Now would be a good time to update the
> installation, which by the way has changed (the machine it was on is dead...
> there is a new one now) so I need to write about that later.
>
> On a related note, ADF2007.01 has been released. We have all the tests in
> ADF2004.01 and should probably think about upgrading this also. I can rerun
> them in 2006.01 or 2007.01 - so the question is whether we want to update
> this soon and, if so, to which version?

I would run them on both just to see if they fail the current tests,
and whether we need to change the parser to deal with them. If you
have access to 2007.01, then I see no reason not to include those
tests as the latest ones.

> Karol
>
> --
> written by Karol Langner
> Fri Oct 19 01:04:14 EDT 2007
>

Re: [cclib-devel] Jaguar data files

[Karol L. <kar...@kn...>]

On Saturday 20 October 2007 18:20, Noel O'Boyle wrote:
> On 19/10/2007, Karol Langner <kar...@kn...> wrote:
> > On Saturday 13 October 2007 08:37, Noel O'Boyle wrote:
> > > On 13/10/2007, Karol Langner <kar...@kn...> wrote:
> > > > > > Let me start things up again with a
> > > > > > simple issue - why do we have three different basicJaguar*
> > > > > > directories in trunk/data/? It seems logical to use data files
> > > > > > for unittests only from the newest Jaguar version and to change
> > > > > > the rest to regressions. That is the situation for Gaussian and
> > > > > > ADF (although here basicADF2004 is not the newest version). Why
> > > > > > should Jaguar be an exception?
> > > > >
> > > > > If a newer version of Gaussian comes out or if we have access to a
> > > > > newer version of ADF, Jaguar won't be the exception. I'm not sure
> > > > > what you mean by changing the unittests to regressions. If we
> > > > > support JaguarX, then we should have unittests to ensure this.
> > > >
> > > > There are probably more people using Gaussian98 than there are using
> > > > Jaguar4.2, so I don't see why we should only support older version of
> > > > Jaguar and not Gaussian.
> > >
> > > There's no reason. We just started supporting whatever programs we had
> > > access to.
> >
> > Of course, and access changes and versions become old. The data files
> > packaged along with cclib should reflect the current state of our access
> > to the newest program versions.
>
> That's fine by me.

Done. The cclib package is now almost 50% slimmer, so that might be something 
to still include in the 0.8 final.
Still to do:
1. Support reading multiple files in regressions.py, plus general cleanup.
2. Allow unittests to be run in regressions.py, for those used-to-be-unittests 
and maybe other files.
3. Update further to Jaguar7.0 when we get the liscence.

> > On a related note, ADF2007.01 has been released. We have all the tests in
> > ADF2004.01 and should probably think about upgrading this also. I can
> > rerun them in 2006.01 or 2007.01 - so the question is whether we want to
> > update this soon and, if so, to which version?
>
> I would run them on both just to see if they fail the current tests,
> and whether we need to change the parser to deal with them. If you
> have access to 2007.01, then I see no reason not to include those
> tests as the latest ones.

In the works.

-Karol

-- 
written by Karol Langner
Mon Oct 29 12:42:51 EDT 2007

Re: [cclib-devel] Jaguar data files

[Noel O'B. <bao...@gm...>]

> Done. The cclib package is now almost 50% slimmer, so that might be something
> to still include in the 0.8 final.
> Still to do:
> 1. Support reading multiple files in regressions.py, plus general cleanup.
> 2. Allow unittests to be run in regressions.py, for those used-to-be-unittests
> and maybe other files.
> 3. Update further to Jaguar7.0 when we get the liscence.

Regarding 2, wouldn't it be easier just to have testall.py run those
tests if basicJaguar4.2 is present? This would be a lot easier to
implement and I don't see any downside.

Regarding 1, we should probably make a subfolder of tests, and put
everything that testall.py imports into that.

Noel

Re: [cclib-devel] Jaguar data files

[Karol L. <kar...@kn...>]

On Wednesday 31 October 2007 07:35, Noel O'Boyle wrote:
> > Done. The cclib package is now almost 50% slimmer, so that might be
> > something to still include in the 0.8 final.
> > Still to do:
> > 1. Support reading multiple files in regressions.py, plus general
> > cleanup. 2. Allow unittests to be run in regressions.py, for those
> > used-to-be-unittests and maybe other files.
> > 3. Update further to Jaguar7.0 when we get the liscence.
>
> Regarding 2, wouldn't it be easier just to have testall.py run those
> tests if basicJaguar4.2 is present? This would be a lot easier to
> implement and I don't see any downside.

You mean "Jaguar4.2" now. That would work, the coverage is the same, and I 
think both solutions are easy to implement. The question is just, how 
important is keeping the most-recent unittests separate from the regressions. 
I see one drawback: if you parse test these files with testall.py and then go 
on to run regression.py, you will actually parse them twice!

> Regarding 1, we should probably make a subfolder of tests, and put
> everything that testall.py imports into that.

I agree; also, for clarity I think the current 'testall' should be renamed 
to 'testparser' since thats what it does. In my view, a 'testall' script 
would run testparser, testmethod, whatever other testsuits there may be and 
possibly regressions, and print the results in **very** condensed form.

Wonder what you think about that,
Karol

-- 
written by Karol Langner
Wed Oct 31 13:41:45 EDT 2007

Re: [cclib-devel] Jaguar data files

[Noel O'B. <bao...@gm...>]

On 31/10/2007, Karol Langner <kar...@kn...> wrote:
> On Wednesday 31 October 2007 07:35, Noel O'Boyle wrote:
> > > Done. The cclib package is now almost 50% slimmer, so that might be
> > > something to still include in the 0.8 final.
> > > Still to do:
> > > 1. Support reading multiple files in regressions.py, plus general
> > > cleanup. 2. Allow unittests to be run in regressions.py, for those
> > > used-to-be-unittests and maybe other files.
> > > 3. Update further to Jaguar7.0 when we get the liscence.
> >
> > Regarding 2, wouldn't it be easier just to have testall.py run those
> > tests if basicJaguar4.2 is present? This would be a lot easier to
> > implement and I don't see any downside.
>
> You mean "Jaguar4.2" now. That would work, the coverage is the same, and I
> think both solutions are easy to implement. The question is just, how
> important is keeping the most-recent unittests separate from the regressions.

I don't think it's a good idea to keep them too separate. Otherwise,
failures will only be discovered at a later point...

> I see one drawback: if you parse test these files with testall.py and then go
> on to run regression.py, you will actually parse them twice!

Regression.py parses everything twice already, because there are files
in the basic folders which are not tested by "testall.py".

> > Regarding 1, we should probably make a subfolder of tests, and put
> > everything that testall.py imports into that.
>
> I agree; also, for clarity I think the current 'testall' should be renamed
> to 'testparser' since thats what it does. In my view, a 'testall' script
> would run testparser, testmethod, whatever other testsuits there may be and
> possibly regressions, and print the results in **very** condensed form.

Sure.

Noel