Re: [cclib-devel] Jaguar data files

[Karol L. <kar...@kn...>]

On Saturday 13 October 2007 08:37, Noel O'Boyle wrote:
> On 13/10/2007, Karol Langner <kar...@kn...> wrote:
> > > > Let me start things up again with a
> > > > simple issue - why do we have three different basicJaguar*
> > > > directories in trunk/data/? It seems logical to use data files for
> > > > unittests only from the newest Jaguar version and to change the rest
> > > > to regressions. That is the situation for Gaussian and ADF (although
> > > > here basicADF2004 is not the newest version). Why should Jaguar be an
> > > > exception?
> > >
> > > If a newer version of Gaussian comes out or if we have access to a
> > > newer version of ADF, Jaguar won't be the exception. I'm not sure what
> > > you mean by changing the unittests to regressions. If we support
> > > JaguarX, then we should have unittests to ensure this.
> >
> > There are probably more people using Gaussian98 than there are using
> > Jaguar4.2, so I don't see why we should only support older version of
> > Jaguar and not Gaussian.
>
> There's no reason. We just started supporting whatever programs we had
> access to.

Of course, and access changes and versions become old. The data files packaged 
along with cclib should reflect the current state of our access to the newest 
program versions.

> > I was referring to data files rather than unittests, of course. What I
> > mean be 'changing the unittests to regressions' is not distributing the
> > older data files with cclib anymore and adding them to the family of
> > regression tests.
> >
> > I think it is sufficient and more efficient to have a good set of test
> > log files only for the newest available version of all the programs, and
> > to support all older versions with appropriate regression tests (works
> > fine for Gaussian, GAMESS, ADF).
>
> Right, okay - now I understand. Yes - that's fine by me. Probably the
> easiest way to do this is simply to have the testall.py script (or
> whatever) only run the tests if the test file exists. At the same
> time, I can change manifest.py (or MANIFEST.in if we can make this
> work in future) so that it only includes the latest files for each.

The appropriate tests can be imported and run by regression.py to make them 
truly regressions.

> > > The historical reason we have unittests for different versions of
> > > Jaguar is that my colleagues in Cambridge were using Jag4.2 (or so),
> > > Adam's colleagues were using 6.?, and finally we got the latest Jaguar
> > > 6.5 (which is now superseded by Jag 7.0 - maybe we should ask for
> > > this).
> >
> > This is redundant, since then we will have as many basicJaguar*
> > directories as there are Jaguar versions being used. Already, the
> > Jaguar4.2 and Jaguar6.0 tests are "behind" Jaguar6.5 since we don't have
> > easy access to them.
> >
> > Notice also that the Jaguar6.0 data files take up alot more space than
> > the Jaguar6.5 ones. I would prefer to see a complete set of log files for
> > Jaguar6.5 and everything else put in directories that can be downloaded
> > as regressions (so we can still check if they are parsed and pass the
> > tests we want).
>
> Yes - I think we agree.

Is that a green light for doing this? Probably after the release, anyway.

As you mentioned, Jaguar7.0 is out. Now would be a good time to update the 
installation, which by the way has changed (the machine it was on is dead... 
there is a new one now) so I need to write about that later.

On a related note, ADF2007.01 has been released. We have all the tests in 
ADF2004.01 and should probably think about upgrading this also. I can rerun 
them in 2006.01 or 2007.01 - so the question is whether we want to update 
this soon and, if so, to which version?

Karol

-- 
written by Karol Langner
Fri Oct 19 01:04:14 EDT 2007