getdata-devel — General discussion of GetData development.

Re: [getdata-devel] symlinks

[D. V. W. <ge...@ke...>]

On Tue, Oct 18, 2011 at 05:51:05AM -0400, Barth Netterfield wrote:
> I think that the proposed 0.8 behavior sounds perfect, if it is well documented.
> 
> It is consistent with how other types of files are handled.  You
> wouldn't expect an image you have opened in gwenview to change if
> someone changes a symlink.
> 
> The reason I was asking was that kst seemed to have changed behavior
> wrst symlink handling, and I was investigating why.
> 
> cbn

Sweet.
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

Re: [getdata-devel] symlinks

[Barth N. <net...@as...>]

I think that the proposed 0.8 behavior sounds perfect, if it is well documented.

It is consistent with how other types of files are handled.  You
wouldn't expect an image you have opened in gwenview to change if
someone changes a symlink.

The reason I was asking was that kst seemed to have changed behavior
wrst symlink handling, and I was investigating why.

cbn

On Mon, Oct 17, 2011 at 8:11 PM, D. V. Wiebe <ge...@ke...> wrote:
> On Mon, Oct 17, 2011 at 12:48:15PM -0400, Barth Netterfield wrote:
>> Don,
>>
>> How does getdata currently handle symlinks?  In particular, if you
>> open a dirfile through a symlink, and then, while it is open, change
>> the symlink to point somewhere else, what happens?  Does getdata still
>> point to the original directory, or to the new one?  Has something
>> changed in the past year in this?
>
> Barth,
>
> Things have changed between 0.7 and (the as-yet-unreleased) trunk, but I
> don't think anything changed between 0.6 and 0.7.  (Unless some sort of
> unintended change due to OSX support?)
>
>
> GetData-0.7
> ===========
>
> In 0.7, GetData caches file descriptors to open raw data files.  So, I
> think the following can happen:
>
> 1) open("/some/symlink", ...) -- getdata caches "/some/symlink" as the
>    name of the dirfile, which, say, points to "/some/dirfile".
> 2) getdata("data", ...) -- Getdata opens "/some/symlink/data" which is
>    actually "/some/dirfile/data" due to the directory symlink, and
>    caches the open directory.
> 3) A third party changes "/some/symlink" to point to "/other/dirfile".
> 4) getdata("moredata", ...) -- Getdata opens "/some/symlink/moredata"
>    which is actually "/other/dirfile/moredata"
>
> And magically you have GetData unwittingly reading from two different
> dirfiles.  This is further complicated by things like gd_nframes, which
> typically just stat(3)'s the associated raw file.  So gd_nframes will
> report the length of "/other/dirfile" while getdata will continue to
> read fields it opened before the symlink change from the old location.
>
> In 0.7 you can work around this by calling gd_flush on a field before
> you read it:
>
> gd_flush("field");
> gd_getdata("field", ...);
>
> The flush call closes the raw file (after syncing any pending writes).
> The subsequent getdata call will then re-open the raw file, wherever it
> happens to be.  There is something of an I/O hit here; probably not too
> bad for a read-only dirfile, bu not sure how much you're willing to
> accept in terms of latency.
>
>
> GetData-0.8
> ===========
>
> Proposed behaviour of 0.8 changes things.  In 0.8, GetData caches
> directory descriptors as well, so now the procedure becomes:
>
> 1) open("/some/symlink", ...) -- getdata obtains a descriptor for the
>    directory pointed to by "/some/symlink", i.e. "/some/dirfile"
> 2) getdata("data", ...) -- Getdata opens "./data" relative to the
>    directory descriptor it has, ie. "/some/dirfile/data".
> 3) A third party changes "/some/symlink" to point to "/other/dirfile".
> 4) getdata("moredata", ...) -- Getdata opens "./moredata" relative to
>    the directory descriptor it has, ie. "/some/dirfile/moredata".
>
> So, changing dirfile symlinks no longer breaks GetData, but I imagine it
> also doesn't do what you want it to do, ie. notice the symlink has
> changed and update the in-core dirfile metadata.  (This behaviour change
> is a side-effect of the bug I was actually trying to fix, namely
> dirfile-smashing resulting from moving stuff around on the filesystem
> while GetData is writing to a database.  This behaviour also fixes the
> thread-saftey problems associated with opening a dirfile with a relative
> pathname.)
>
> In this new scheme, calling flush() doesn't work around it either, since
> Getdata will doggedly return to that first directory descriptor it
> cached, regardless of the flush.
>
> If you'd like GetData to take care of symlink monitoring, I can see a
> few ways to do it:
>
> 1) Ditch directory caching, returning to the 0.7 GetData-confusing
> behaviour, which leaves us with the dirfile-smashing problem as well.
> (The thread-saftey stuff could be fixed by absolutising relative path
> names when they're encountered.)  I'm not too excited about this, as I'd
> like GetData to not corrupt dirfiles, if possible.  Not to mention that
> if the metadata aren't identical between the two dirfiles, things can get
> very broken, even in the read-only case.
>
> 2) Have GetData actively monitor symlinks and update things when
> symlinks change.  I don't think there's a whole lot of work to implement
> this, but the details could get tricky.  We'd have some sort of flag, say,
> GD_MONITOR_SYMLINKS to the open call to make this optional.  And then
> GetData would inspect all parts of the path from the root dirfile
> directory down to the file we're interested in every time an I/O
> operation is called (getdata, putdata, &c.)
>
> The hardest problem with this is that if "/some/symlink" changes from
> pointing to "/some/dirfile" to pointing to "/other/dirfile", GetData
> has to conclude that the dirfile metdata have changed, which boils down
> to GetData throwing in the towel on the old dirfile, effectively doing a
> close/open thing.  I don't think GetData itself would mind so much, but
> it might be problematic for the caller, given that no notification is
> passed back up to indicate such a reinitialisation has occurred.
>
> So now, while using GD_MONITOR_SYMLINKS, the caller can no longer assume
> it knows anything about the database from one I/O call to the next: so no
> caching field names, field lengths, scalar values, &c.  How well does
> kst deal with this situation?
>
> 3) Volatile mode.  Have a open flag, say, GD_VOLATILE, which tells
> GetData not to cache anything involving paths, because they're likely to
> change while GetData is working.  There's more of a performance hit,
> since GetData wouldn't keep files open between read calls, but you'd
> never run into the broken behaviour of 0.7.  This is actually more
> flexible than (2), with little more work (possibly less).  GetData
> would have to monitor the format file(s) for changes and resort to
> the towel-in-throwing behaviour if it noticed they'd changed, but
> it wouldn't be any worse than (2).  The caller's in the same boat
> as before, not being able to cache anything about the dirfile.
>
> At the moment, I think I prefer (3); it's the most general of the
> solutions, but I'll see if I can think of something cleverer.
> Ideas welcome.
>
> Cheers,
> -dvw
> --
> D. V. Wiebe
> ge...@ke...
> http://getdata.sourceforge.net/
>



-- 
C. Barth Netterfield
University of Toronto
416-845-0946

Re: [getdata-devel] symlinks

[D. V. W. <ge...@ke...>]

On Mon, Oct 17, 2011 at 12:48:15PM -0400, Barth Netterfield wrote:
> Don,
> 
> How does getdata currently handle symlinks?  In particular, if you
> open a dirfile through a symlink, and then, while it is open, change
> the symlink to point somewhere else, what happens?  Does getdata still
> point to the original directory, or to the new one?  Has something
> changed in the past year in this?

Barth,

Things have changed between 0.7 and (the as-yet-unreleased) trunk, but I
don't think anything changed between 0.6 and 0.7.  (Unless some sort of
unintended change due to OSX support?)


GetData-0.7
===========

In 0.7, GetData caches file descriptors to open raw data files.  So, I
think the following can happen:

1) open("/some/symlink", ...) -- getdata caches "/some/symlink" as the
    name of the dirfile, which, say, points to "/some/dirfile".
2) getdata("data", ...) -- Getdata opens "/some/symlink/data" which is
    actually "/some/dirfile/data" due to the directory symlink, and
    caches the open directory.
3) A third party changes "/some/symlink" to point to "/other/dirfile".
4) getdata("moredata", ...) -- Getdata opens "/some/symlink/moredata"
    which is actually "/other/dirfile/moredata"

And magically you have GetData unwittingly reading from two different
dirfiles.  This is further complicated by things like gd_nframes, which
typically just stat(3)'s the associated raw file.  So gd_nframes will
report the length of "/other/dirfile" while getdata will continue to
read fields it opened before the symlink change from the old location.

In 0.7 you can work around this by calling gd_flush on a field before
you read it:

gd_flush("field");
gd_getdata("field", ...);

The flush call closes the raw file (after syncing any pending writes).
The subsequent getdata call will then re-open the raw file, wherever it
happens to be.  There is something of an I/O hit here; probably not too
bad for a read-only dirfile, bu not sure how much you're willing to
accept in terms of latency.


GetData-0.8
===========

Proposed behaviour of 0.8 changes things.  In 0.8, GetData caches
directory descriptors as well, so now the procedure becomes:

1) open("/some/symlink", ...) -- getdata obtains a descriptor for the
    directory pointed to by "/some/symlink", i.e. "/some/dirfile"
2) getdata("data", ...) -- Getdata opens "./data" relative to the
    directory descriptor it has, ie. "/some/dirfile/data".
3) A third party changes "/some/symlink" to point to "/other/dirfile".
4) getdata("moredata", ...) -- Getdata opens "./moredata" relative to
    the directory descriptor it has, ie. "/some/dirfile/moredata".

So, changing dirfile symlinks no longer breaks GetData, but I imagine it
also doesn't do what you want it to do, ie. notice the symlink has
changed and update the in-core dirfile metadata.  (This behaviour change
is a side-effect of the bug I was actually trying to fix, namely
dirfile-smashing resulting from moving stuff around on the filesystem
while GetData is writing to a database.  This behaviour also fixes the
thread-saftey problems associated with opening a dirfile with a relative
pathname.)

In this new scheme, calling flush() doesn't work around it either, since
Getdata will doggedly return to that first directory descriptor it
cached, regardless of the flush.

If you'd like GetData to take care of symlink monitoring, I can see a
few ways to do it:

1) Ditch directory caching, returning to the 0.7 GetData-confusing
behaviour, which leaves us with the dirfile-smashing problem as well.
(The thread-saftey stuff could be fixed by absolutising relative path
names when they're encountered.)  I'm not too excited about this, as I'd
like GetData to not corrupt dirfiles, if possible.  Not to mention that
if the metadata aren't identical between the two dirfiles, things can get
very broken, even in the read-only case.

2) Have GetData actively monitor symlinks and update things when
symlinks change.  I don't think there's a whole lot of work to implement
this, but the details could get tricky.  We'd have some sort of flag, say,
GD_MONITOR_SYMLINKS to the open call to make this optional.  And then
GetData would inspect all parts of the path from the root dirfile
directory down to the file we're interested in every time an I/O
operation is called (getdata, putdata, &c.)

The hardest problem with this is that if "/some/symlink" changes from
pointing to "/some/dirfile" to pointing to "/other/dirfile", GetData
has to conclude that the dirfile metdata have changed, which boils down
to GetData throwing in the towel on the old dirfile, effectively doing a
close/open thing.  I don't think GetData itself would mind so much, but
it might be problematic for the caller, given that no notification is
passed back up to indicate such a reinitialisation has occurred.

So now, while using GD_MONITOR_SYMLINKS, the caller can no longer assume
it knows anything about the database from one I/O call to the next: so no
caching field names, field lengths, scalar values, &c.  How well does
kst deal with this situation?

3) Volatile mode.  Have a open flag, say, GD_VOLATILE, which tells
GetData not to cache anything involving paths, because they're likely to
change while GetData is working.  There's more of a performance hit,
since GetData wouldn't keep files open between read calls, but you'd
never run into the broken behaviour of 0.7.  This is actually more
flexible than (2), with little more work (possibly less).  GetData
would have to monitor the format file(s) for changes and resort to
the towel-in-throwing behaviour if it noticed they'd changed, but
it wouldn't be any worse than (2).  The caller's in the same boat
as before, not being able to cache anything about the dirfile.

At the moment, I think I prefer (3); it's the most general of the
solutions, but I'll see if I can think of something cleverer.
Ideas welcome.

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

[getdata-devel] symlinks

[Barth N. <net...@as...>]

Don,

How does getdata currently handle symlinks?  In particular, if you
open a dirfile through a symlink, and then, while it is open, change
the symlink to point somewhere else, what happens?  Does getdata still
point to the original directory, or to the new one?  Has something
changed in the past year in this?

-- 
C. Barth Netterfield
University of Toronto
416-845-0946

Re: [getdata-devel] libgetdata in Debian

[D. V. W. <ge...@ke...>]

On Sat, Sep 24, 2011 at 11:14:18AM -0500, Michael Milligan wrote:
> Dear getdata community,
> 
> I would like to announce that getdata has been accepted into the
> Debian distribution. The base library, development files, and bindings
> (all but IDL) are available as separate packages. Please see
> http://packages.debian.org/src:libgetdata for more details.
> 
> At this time the package is only available in Debian unstable. If
> there is demand it can also be added to squeeze-backports to make it
> more easily availble to the Debian stable release.
> 
> Thanks to Steve Benton for doing most of the initial packaging work.
> 
> Cheers,
> ...Milligan


That's great news.  One can hardly call oneself a legitimate
open-source software product if one's not in debian.

Is it possible to add a "make check" to the build rules?  I see
debian packages the library for big-endian machines.  With the
Fedora packages, Matthew Truch and I now and again found that
GetData would build fine and then not work on these systems due to
some little endian assumption.  The test-suite was very helpful in
figuring out what was going on in these cases.

Regardless, thanks guys for your work.  I'll see if I can add something
about this to the GetData website.

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

Re: [getdata-devel] libgetdata patch

[D. V. W. <ge...@ke...>]

On Sat, Sep 24, 2011 at 03:53:48PM -0500, Michael Milligan wrote:
> Hi,
> 
> I am attaching a small patch that was generated during the development
> of the libgetdata Debian package. It standardizes the headers of some
> manpage files that were triggering lintian warnings in the package
> build.
> 
> ...Milligan

Thanks.  It's nice to know someone's reading those things (even if it's
just lintian).  I've applied your patch to trunk.

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

[getdata-devel] libgetdata patch

[Michael M. <mmi...@as...>]

Hi,

I am attaching a small patch that was generated during the development
of the libgetdata Debian package. It standardizes the headers of some
manpage files that were triggering lintian warnings in the package
build.

...Milligan

-- 

[getdata-devel] libgetdata in Debian

[Michael M. <mmi...@as...>]

Dear getdata community,

I would like to announce that getdata has been accepted into the
Debian distribution. The base library, development files, and bindings
(all but IDL) are available as separate packages. Please see
http://packages.debian.org/src:libgetdata for more details.

At this time the package is only available in Debian unstable. If
there is demand it can also be added to squeeze-backports to make it
more easily availble to the Debian stable release.

Thanks to Steve Benton for doing most of the initial packaging work.

Cheers,
...Milligan

-- 

Re: [getdata-devel] Bug in configure, with --disable-cplusplus

[D. V. W. <ge...@ke...>]

On Thu, Jul 21, 2011 at 01:25:53PM -0400, Steve Benton wrote:
> Hi again,
> 
> I have discovered a bug in the configure script that causes it to fail
> when the C++ bindings are set not to build.
> 
> $ ./configure --disable-cplusplus
> .....lots of output.....
> *** Writing configure output
> 
> configure: error: conditional "am__fastdepCXX" was never defined.
> Usually this means the macro was only invoked conditionally.
> 
> 
> I've traced this to version 0.7.2 where extra calls to AC_PROG_CXX and
> AC_PROG_CXX_0 (in configure.ac) were removed, leaving just calls that
> are conditionally reached only when the C++ bindings are  being built.
> Some google searching
> (http://sourceware.org/ml/automake/2005-01/msg00037.html) indicates that
> maybe these should always be run.
> 
> When I comment out the if statment (see attached diff) then configure
> succeeds.
> 
> -Steve

Ahhhh... yeah, at one point there were two instances of AC_PROG_CXX in
the configure script, one inside the conditional and one outside.  I
guess I deleted the wrong one.

Thanks,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

[getdata-devel] Bug in configure, with --disable-cplusplus

[Steve B. <sb...@ph...>]

Hi again,

I have discovered a bug in the configure script that causes it to fail
when the C++ bindings are set not to build.

$ ./configure --disable-cplusplus
.....lots of output.....
*** Writing configure output

configure: error: conditional "am__fastdepCXX" was never defined.
Usually this means the macro was only invoked conditionally.


I've traced this to version 0.7.2 where extra calls to AC_PROG_CXX and
AC_PROG_CXX_0 (in configure.ac) were removed, leaving just calls that
are conditionally reached only when the C++ bindings are  being built.
Some google searching
(http://sourceware.org/ml/automake/2005-01/msg00037.html) indicates that
maybe these should always be run.

When I comment out the if statment (see attached diff) then configure
succeeds.

-Steve

Re: [getdata-devel] "Useless" getdata linking

[D. V. W. <ge...@ke...>]

On Wed, Jul 20, 2011 at 11:52:43AM -0400, Steve Benton wrote:
> Hi getdata-devel,
> 
> In polishing the getdata .deb package, I have come across some warnings
> about libraries that are linked in the python and Fortran bindings, but
> not used. This would be nice (but not absolutely necessary) to resolve,
> so that package dependencies could be reduced.
> 
> This appears to be a result of the configure script. However, I am not
> familiar enough with autotools to diagnose further. After a quick look
> online, it also appears that this commonly happens indirectly via used
> libraries that then link other unused libraries. Such a case is harder,
> and probably not worthwhile, to fix.
> 
> The debian build configures once with:
> configure --disable-python
> and then multiple times with:
> configure --disable-bindings --enable-python
> --with-python=/usr/bin/python$$v
> where v loops over all python2 versions. (Different configs are, of
> course, built separately too)
> 
> The specific warnings from dpkg-shlibdeps are:
> dpkg-shlibdeps: warning: dependency on libm.so.6 could be avoided if
> "debian/libfgetdata2/usr/lib/libfgetdata.so.2.0.1" were not uselessly
> linked against it (they use none of its symbols).
> dpkg-shlibdeps: warning: dependency on libgetdata.so.4 could be avoided
> if "debian/libf95getdata2/usr/lib/libf95getdata.so.2.0.0" were not
> uselessly linked against it (they use none of its symbols).
> dpkg-shlibdeps: warning: dependency on libgcc_s.so.1 could be avoided if
> "debian/libf95getdata2/usr/lib/libf95getdata.so.2.0.0" were not
> uselessly linked against it (they use none of its symbols).
> dpkg-shlibdeps: warning: dependency on libm.so.6 could be avoided if
> "debian/libf95getdata2/usr/lib/libf95getdata.so.2.0.0" were not
> uselessly linked against it (they use none of its symbols).
> dpkg-shlibdeps: warning: dependency on libdl.so.2 could be avoided if
> "debian/python-pygetdata/usr/lib/pyshared/python2.6/pygetdata.so" were
> not uselessly linked against it (they use none of its symbols).
> dpkg-shlibdeps: warning: dependency on libm.so.6 could be avoided if
> "debian/python-pygetdata/usr/lib/pyshared/python2.6/pygetdata.so" were
> not uselessly linked against it (they use none of its symbols).
> dpkg-shlibdeps: warning: dependency on libutil.so.1 could be avoided if
> "debian/python-pygetdata/usr/lib/pyshared/python2.6/pygetdata.so" were
> not uselessly linked against it (they use none of its symbols).
> dpkg-shlibdeps: warning: dependency on libpthread.so.0 could be avoided
> if "debian/python-pygetdata/usr/lib/pyshared/python2.6/pygetdata.so"
> were not uselessly linked against it (they use none of its symbols).
> 
> 
> Cheers,
> -Steve Benton

Thanks for the report, Steve; you're right: the build system currently
does do some unnecessary linking, mostly because I've been lazy with the
autotools.  I'll take a look; it should be relatively easy to fix,
although I don't know how much benefit it would have.

For instance, while it's true that libf95getdata doesn't use libm or
libgetdata directly, there is still the depenency path libf95getdata ->
libfgetdata -> libgetdata -> libm

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

Re: [getdata-devel] "Useless" getdata linking

[Steve B. <sb...@ph...>]

On 11-07-20 06:41 PM, D. V. Wiebe wrote:
> For instance, while it's true that libf95getdata doesn't use libm or
> libgetdata directly, there is still the depenency path libf95getdata ->
> libfgetdata -> libgetdata -> libm
If that's true for all the lazy linking cases, then it would completely
negate my already tiny desire for this to be fixed. (Except for the
nagging at the back of my brain insisting that it could be "better"---if
only dubiously so.)

[getdata-devel] "Useless" getdata linking

[Steve B. <sb...@ph...>]

Hi getdata-devel,

In polishing the getdata .deb package, I have come across some warnings
about libraries that are linked in the python and Fortran bindings, but
not used. This would be nice (but not absolutely necessary) to resolve,
so that package dependencies could be reduced.

This appears to be a result of the configure script. However, I am not
familiar enough with autotools to diagnose further. After a quick look
online, it also appears that this commonly happens indirectly via used
libraries that then link other unused libraries. Such a case is harder,
and probably not worthwhile, to fix.

The debian build configures once with:
configure --disable-python
and then multiple times with:
configure --disable-bindings --enable-python
--with-python=/usr/bin/python$$v
where v loops over all python2 versions. (Different configs are, of
course, built separately too)

The specific warnings from dpkg-shlibdeps are:
dpkg-shlibdeps: warning: dependency on libm.so.6 could be avoided if
"debian/libfgetdata2/usr/lib/libfgetdata.so.2.0.1" were not uselessly
linked against it (they use none of its symbols).
dpkg-shlibdeps: warning: dependency on libgetdata.so.4 could be avoided
if "debian/libf95getdata2/usr/lib/libf95getdata.so.2.0.0" were not
uselessly linked against it (they use none of its symbols).
dpkg-shlibdeps: warning: dependency on libgcc_s.so.1 could be avoided if
"debian/libf95getdata2/usr/lib/libf95getdata.so.2.0.0" were not
uselessly linked against it (they use none of its symbols).
dpkg-shlibdeps: warning: dependency on libm.so.6 could be avoided if
"debian/libf95getdata2/usr/lib/libf95getdata.so.2.0.0" were not
uselessly linked against it (they use none of its symbols).
dpkg-shlibdeps: warning: dependency on libdl.so.2 could be avoided if
"debian/python-pygetdata/usr/lib/pyshared/python2.6/pygetdata.so" were
not uselessly linked against it (they use none of its symbols).
dpkg-shlibdeps: warning: dependency on libm.so.6 could be avoided if
"debian/python-pygetdata/usr/lib/pyshared/python2.6/pygetdata.so" were
not uselessly linked against it (they use none of its symbols).
dpkg-shlibdeps: warning: dependency on libutil.so.1 could be avoided if
"debian/python-pygetdata/usr/lib/pyshared/python2.6/pygetdata.so" were
not uselessly linked against it (they use none of its symbols).
dpkg-shlibdeps: warning: dependency on libpthread.so.0 could be avoided
if "debian/python-pygetdata/usr/lib/pyshared/python2.6/pygetdata.so"
were not uselessly linked against it (they use none of its symbols).


Cheers,
-Steve Benton

[getdata-devel] GetData 0.7.3 released.

[D. V. W. <ge...@ke...>]

GetData 0.7.3 has been released.  It may be downloaded from your local
SourceForge mirror:

http://sourceforge.net/projects/getdata/files/getdata/0.7.3/

GetData 0.7.3 is a bug-fix release, correcting a few errors found after
the release of 0.7.2.

Full release notes are provided at the bottom of the release page linked
above.

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

[getdata-devel] GetData 0.7.2 released.

[D. V. W. <ge...@ke...>]

GetData 0.7.2 has been released.  It may be downloaded from your local
SourceForget mirror:

http://sourceforge.net/projects/getdata/files/getdata/0.7.2/

GetData 0.7.2 is a bug-fix release, adding the documented but missing
functionality allowing absolute paths in /INCLUDE directives, and a few
other minor bugs.

More importantly, this release marks the introduction of a source
release targeted at Microsoft Corporation's Visual C++ compiler.  This
package is largely due to the effort of Peter Kümmel, who ported the
source code to MSVC and wrote the CMake build system used to create
project files to build GetData.  (You can also use it with MinGW.)

Finally, we've also released a source package containing only the IDL
bindings, meant to be built against an pre-installed GetData library.
This should be useful to people who use a pre-packaged GetData, and
would like GetData functionality in IDL without having to recompile the
whole library from source.

Full release notes are provided at the bottom of the release page linked
above.

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

Re: [getdata-devel] autoconf / 0.7

[Peter K. <syn...@gm...>]

On 24.03.2011 20:11, Matthew D Truch wrote:
> On Thu, Mar 24, 2011 at 08:08:33PM +0100, Peter Kümmel wrote:
>> I've checked out the 0.7 branch, called 'autoconf' in
>> the source directory, and get this error:
>>
>> configure.ac:282: error: possibly undefined macro: AM_INIT_AUTOMAKE
>>         If this token and others are legitimate, please use m4_pattern_allow.
>>         See the Autoconf documentation.
>> configure.ac:802: error: possibly undefined macro: AM_CONDITIONAL
>>
>>
>> What's wrong with my system?
>>
>> $ autoconf --version
>> autoconf (GNU Autoconf) 2.67
>>
>> $automake --version
>> automake (GNU automake) 1.11.1
>>
>> Or is it not that simple "call autoconf"?
>
> No.  You should call "autoreconf -vifs".  I only know this because I was
> recently looking at the getdata webpage which contains this explanation
> under the "Subversion" heading of the "Get GetData" section:
> http://getdata.sourceforge.net/index.html#svn
>

Thanks! I've updated INSTALL.

Peter

Re: [getdata-devel] autoconf / 0.7

[Matthew D T. <ma...@tr...>]

On Thu, Mar 24, 2011 at 08:08:33PM +0100, Peter Kümmel wrote:
> I've checked out the 0.7 branch, called 'autoconf' in
> the source directory, and get this error:
> 
> configure.ac:282: error: possibly undefined macro: AM_INIT_AUTOMAKE
>        If this token and others are legitimate, please use m4_pattern_allow.
>        See the Autoconf documentation.
> configure.ac:802: error: possibly undefined macro: AM_CONDITIONAL
> 
> 
> What's wrong with my system?
> 
> $ autoconf --version
> autoconf (GNU Autoconf) 2.67
> 
> $automake --version
> automake (GNU automake) 1.11.1
> 
> Or is it not that simple "call autoconf"?

No.  You should call "autoreconf -vifs".  I only know this because I was
recently looking at the getdata webpage which contains this explanation
under the "Subversion" heading of the "Get GetData" section:
http://getdata.sourceforge.net/index.html#svn

-- 
"299792458 m/s.  It's not just a good idea, It's the law."
--------------------------
Matthew Truch
Department of Physics and Astronomy
University of Pennsylvania
ma...@tr...
http://matt.truch.net/

[getdata-devel] autoconf / 0.7

[Peter K. <syn...@gm...>]

I've checked out the 0.7 branch, called 'autoconf' in
the source directory, and get this error:

configure.ac:282: error: possibly undefined macro: AM_INIT_AUTOMAKE
       If this token and others are legitimate, please use m4_pattern_allow.
       See the Autoconf documentation.
configure.ac:802: error: possibly undefined macro: AM_CONDITIONAL


What's wrong with my system?

$ autoconf --version
autoconf (GNU Autoconf) 2.67

$automake --version
automake (GNU automake) 1.11.1

Or is it not that simple "call autoconf"?

Peter

Re: [getdata-devel] [Getdata-commits] SF.net SVN: getdata:[544] branches/getdata-0.7

[Peter K. <syn...@gm...>]

On 23.03.2011 12:11, Peter Kümmel wrote:
> On 22.03.2011 23:54, ket...@us... wrote:
>> Revision: 544
>>             http://getdata.svn.sourceforge.net/getdata/?rev=544&view=rev
>> Author:   ketiltrout
>> Date:     2011-03-22 22:54:06 +0000 (Tue, 22 Mar 2011)
>>
>> Log Message:
>> -----------
>> Revert some inadvertant changes from r537.
>>
>> Modified Paths:
>> --------------
>>       branches/getdata-0.7/cmake/CMakeLists.txt
>>       branches/getdata-0.7/src/getdata.h.in
>>
>
> Does not build with msvc.

Fixed it. All but some parse_include tests passes:

1>  99% tests passed, 3 tests failed out of 650
1>
1>  Total Test time (real) =  29.18 sec
1>
1>  The following tests FAILED:
1>  	456 - test_parse_include_absolute (Failed)
1>  	457 - test_parse_include_absrel (Failed)
1>  	459 - test_parse_include_relabs (Failed)


Peter



>
>
>> Modified: branches/getdata-0.7/cmake/CMakeLists.txt
>> ===================================================================
>> --- branches/getdata-0.7/cmake/CMakeLists.txt	2011-03-22 11:08:06 UTC (rev 543)
>> +++ branches/getdata-0.7/cmake/CMakeLists.txt	2011-03-22 22:54:06 UTC (rev 544)
>> @@ -29,6 +29,20 @@
>>    	-DPACKAGE_BUGREPORT=\"get...@li...\")
>>
>>
>> +## Substitutions needed to build getdata.h
>> +# build in ANSI C mode
>> +set(DEFINE_GD_NO_C99_API "#define GD_NO_C99_API")
>> +
>> +# kst2 doesn't need the legacy API
>> +set(DEFINE_GD_LEGACY_API "/* #undef GD_LEGACY_API */")
>> +
>> +# MSVCRT integer types
>> +set(DEFINE_gd_int16_t "#define gd_int16_t short int")
>> +set(DEFINE_gd_uint16_t "#define gd_int16_t unsigned short int")
>> +set(DEFINE_gd_int64_t "#define gd_int64_t __int64")
>> +
>> +
>> +
>>    if(MSVC)
>>    	add_definitions(-D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS)
>>    	add_definitions(
>> @@ -45,7 +59,6 @@
>>    		-DHAVE__COMMIT
>>    		-DHAVE__STRTOI64
>>    		-DHAVE__STRTOUI64
>> -		-DGD_NO_C99_API
>>    		)
>>    	set(CMAKE_DEBUG_POSTFIX d)
>>    	
>>
>> Modified: branches/getdata-0.7/src/getdata.h.in
>> ===================================================================
>> --- branches/getdata-0.7/src/getdata.h.in	2011-03-22 11:08:06 UTC (rev 543)
>> +++ branches/getdata-0.7/src/getdata.h.in	2011-03-22 22:54:06 UTC (rev 544)
>> @@ -56,7 +56,7 @@
>>    #include<sys/types.h>
>>
>>    /* If this symbol is defined here, the C99-API is not present in the library */
>> -#undef GD_NO_C99_API /**/
>> +@DEFINE_GD_NO_C99_API@
>>
>>    #if defined(GD_NO_C99_API)&&   ! defined(GD_C89_API)
>>    # define GD_C89_API
>> @@ -227,15 +227,9 @@
>>    /* GD_NO_ENTRY is not part of this count */
>>    #define GD_N_ENTYPES 14
>>
>> -#if defined(__MINGW32__) || defined(_MSC_VER)
>> -typedef short _gd_int16_t;
>> -typedef unsigned short _gd_uint16_t;
>> -typedef __int64 _gd_int64_t;
>> -#else
>> -#undef _gd_int16_t /**/
>> -#undef _gd_uint16_t /**/
>> -#undef _gd_int64_t /**/
>> -#endif
>> +@DEFINE_gd_int16_t@
>> +@DEFINE_gd_uint16_t@
>> +@DEFINE_gd_int64_t@
>>
>>    #if ! defined GD_C89_API
>>    #  include<inttypes.h>
>> @@ -247,9 +241,9 @@
>>    #else
>>    #  define GD_DCOMPLEXM(v) double v[2]
>>    #  define GD_DCOMPLEXP(v) double *v
>> -typedef _gd_int16_t gd_bit_t;
>> -typedef _gd_uint16_t gd_spf_t;
>> -typedef _gd_int64_t gd_shift_t;
>> +typedef gd_int16_t gd_bit_t;
>> +typedef gd_uint16_t gd_spf_t;
>> +typedef gd_int64_t gd_shift_t;
>>    #endif
>>
>>    /* Data types -- No valid type may set 0x40 */
>>
>>
>> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
>>
>> ------------------------------------------------------------------------------
>> Enable your software for Intel(R) Active Management Technology to meet the
>> growing manageability and security demands of your customers. Businesses
>> are taking advantage of Intel(R) vPro (TM) technology - will your software
>> be a part of the solution? Download the Intel(R) Manageability Checker
>> today! http://p.sf.net/sfu/intel-dev2devmar
>> _______________________________________________
>> getdata-commits mailing list
>> get...@li...
>> https://lists.sourceforge.net/lists/listinfo/getdata-commits
>>
>
> ------------------------------------------------------------------------------
> Enable your software for Intel(R) Active Management Technology to meet the
> growing manageability and security demands of your customers. Businesses
> are taking advantage of Intel(R) vPro (TM) technology - will your software
> be a part of the solution? Download the Intel(R) Manageability Checker
> today! http://p.sf.net/sfu/intel-dev2devmar
> _______________________________________________
> getdata-devel mailing list
> get...@li...
> https://lists.sourceforge.net/lists/listinfo/getdata-devel
>

Re: [getdata-devel] Flush time + usage philosophy

[Ross W. <ros...@gm...>]

Cheers that change worked.

Oh and thanks everyone for helping me get up to speed with the getdata stuff.

Ross

On Tue, Mar 22, 2011 at 6:46 PM, D. V. Wiebe <ge...@ke...> wrote:
> On Tue, Mar 22, 2011 at 04:26:22PM -0700, D. V. Wiebe wrote:
>> On Tue, Mar 22, 2011 at 06:20:38PM -0500, Ross Williamson wrote:
>> > OK I think I'm getting there.
>> >
>> > The problem seems to be the way that I've setup my directory
>> > structure.  Even though I create a top-level dirfile I don't actually
>> > have any fields in there. The format file just includes the
>> > subdirectories.  That toplevel dirfile however still requires a
>> > /REFERENCE so it takes it to be the first RAW entry inside my dirfile
>> > hierarchy - in this case it is array::frame::status (where array and
>> > frame are also dirfiles).  When call to free takes place it happy
>> > deletes the REFERENCE in the toplevel. (i.e. fragment[0] deletes
>> > fine). It then moves onto fragment[1] (array) which also has
>> > array::frame::status as it's REFERENCE. It tries to delete it (but
>> > it's already been deleted) and hence we get the "double free or
>> > corruption" error.
>> >
>> > So...
>> >
>> > I'm guessing I'm going to have to come up with a more sensible way of
>> > laying out my directories or maybe there is a better way of dealing
>> > with this.
>>
>> Nope.  You're directory structure is fine.  There's no reason you need a
>> raw field in the top level fragment.  GetData should be able to handle
>> things.  Let me see if I can reproduce this.
>>
>> -dvw
>
> Yep, GetData bug.  It'll be fixed in 0.7.2 which will be released in the
> next week or so.  But, if you're anxious, all you need to change in the
> 0.7.1 source is line 483 of add.c from
>
>  D->fragment[i].ref_name = (char *)new_ref;
>
> to
>
>  D->fragment[i].ref_name = strdup(new_ref);
>
> Thanks for the investigation Ross,
> -dvw
> --
> D. V. Wiebe
> ge...@ke...
> http://getdata.sourceforge.net/
>



-- 
Ross Williamson
University of Chicago
Department of Astronomy & Astrophysics
773-834-9785 (office)
312-504-3051 (Cell)

Re: [getdata-devel] [Getdata-commits] SF.net SVN: getdata:[544] branches/getdata-0.7

[Peter K. <syn...@gm...>]

On 22.03.2011 23:54, ket...@us... wrote:
> Revision: 544
>            http://getdata.svn.sourceforge.net/getdata/?rev=544&view=rev
> Author:   ketiltrout
> Date:     2011-03-22 22:54:06 +0000 (Tue, 22 Mar 2011)
>
> Log Message:
> -----------
> Revert some inadvertant changes from r537.
>
> Modified Paths:
> --------------
>      branches/getdata-0.7/cmake/CMakeLists.txt
>      branches/getdata-0.7/src/getdata.h.in
>

Does not build with msvc. No msvc in 0.7.2.
I'll wait for 0.8 and trunk.

Peter


> Modified: branches/getdata-0.7/cmake/CMakeLists.txt
> ===================================================================
> --- branches/getdata-0.7/cmake/CMakeLists.txt	2011-03-22 11:08:06 UTC (rev 543)
> +++ branches/getdata-0.7/cmake/CMakeLists.txt	2011-03-22 22:54:06 UTC (rev 544)
> @@ -29,6 +29,20 @@
>   	-DPACKAGE_BUGREPORT=\"get...@li...\")
>
>
> +## Substitutions needed to build getdata.h
> +# build in ANSI C mode
> +set(DEFINE_GD_NO_C99_API "#define GD_NO_C99_API")
> +
> +# kst2 doesn't need the legacy API
> +set(DEFINE_GD_LEGACY_API "/* #undef GD_LEGACY_API */")
> +
> +# MSVCRT integer types
> +set(DEFINE_gd_int16_t "#define gd_int16_t short int")
> +set(DEFINE_gd_uint16_t "#define gd_int16_t unsigned short int")
> +set(DEFINE_gd_int64_t "#define gd_int64_t __int64")
> +
> +
> +
>   if(MSVC)
>   	add_definitions(-D_CRT_SECURE_NO_WARNINGS -D_CRT_NONSTDC_NO_WARNINGS)
>   	add_definitions(
> @@ -45,7 +59,6 @@
>   		-DHAVE__COMMIT
>   		-DHAVE__STRTOI64
>   		-DHAVE__STRTOUI64
> -		-DGD_NO_C99_API
>   		)
>   	set(CMAKE_DEBUG_POSTFIX d)
>   	
>
> Modified: branches/getdata-0.7/src/getdata.h.in
> ===================================================================
> --- branches/getdata-0.7/src/getdata.h.in	2011-03-22 11:08:06 UTC (rev 543)
> +++ branches/getdata-0.7/src/getdata.h.in	2011-03-22 22:54:06 UTC (rev 544)
> @@ -56,7 +56,7 @@
>   #include<sys/types.h>
>
>   /* If this symbol is defined here, the C99-API is not present in the library */
> -#undef GD_NO_C99_API /**/
> +@DEFINE_GD_NO_C99_API@
>
>   #if defined(GD_NO_C99_API)&&  ! defined(GD_C89_API)
>   # define GD_C89_API
> @@ -227,15 +227,9 @@
>   /* GD_NO_ENTRY is not part of this count */
>   #define GD_N_ENTYPES 14
>
> -#if defined(__MINGW32__) || defined(_MSC_VER)
> -typedef short _gd_int16_t;
> -typedef unsigned short _gd_uint16_t;
> -typedef __int64 _gd_int64_t;
> -#else
> -#undef _gd_int16_t /**/
> -#undef _gd_uint16_t /**/
> -#undef _gd_int64_t /**/
> -#endif
> +@DEFINE_gd_int16_t@
> +@DEFINE_gd_uint16_t@
> +@DEFINE_gd_int64_t@
>
>   #if ! defined GD_C89_API
>   #  include<inttypes.h>
> @@ -247,9 +241,9 @@
>   #else
>   #  define GD_DCOMPLEXM(v) double v[2]
>   #  define GD_DCOMPLEXP(v) double *v
> -typedef _gd_int16_t gd_bit_t;
> -typedef _gd_uint16_t gd_spf_t;
> -typedef _gd_int64_t gd_shift_t;
> +typedef gd_int16_t gd_bit_t;
> +typedef gd_uint16_t gd_spf_t;
> +typedef gd_int64_t gd_shift_t;
>   #endif
>
>   /* Data types -- No valid type may set 0x40 */
>
>
> This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
>
> ------------------------------------------------------------------------------
> Enable your software for Intel(R) Active Management Technology to meet the
> growing manageability and security demands of your customers. Businesses
> are taking advantage of Intel(R) vPro (TM) technology - will your software
> be a part of the solution? Download the Intel(R) Manageability Checker
> today! http://p.sf.net/sfu/intel-dev2devmar
> _______________________________________________
> getdata-commits mailing list
> get...@li...
> https://lists.sourceforge.net/lists/listinfo/getdata-commits
>

Re: [getdata-devel] Flush time + usage philosophy

[D. V. W. <ge...@ke...>]

On Tue, Mar 22, 2011 at 04:26:22PM -0700, D. V. Wiebe wrote:
> On Tue, Mar 22, 2011 at 06:20:38PM -0500, Ross Williamson wrote:
> > OK I think I'm getting there.
> > 
> > The problem seems to be the way that I've setup my directory
> > structure.  Even though I create a top-level dirfile I don't actually
> > have any fields in there. The format file just includes the
> > subdirectories.  That toplevel dirfile however still requires a
> > /REFERENCE so it takes it to be the first RAW entry inside my dirfile
> > hierarchy - in this case it is array::frame::status (where array and
> > frame are also dirfiles).  When call to free takes place it happy
> > deletes the REFERENCE in the toplevel. (i.e. fragment[0] deletes
> > fine). It then moves onto fragment[1] (array) which also has
> > array::frame::status as it's REFERENCE. It tries to delete it (but
> > it's already been deleted) and hence we get the "double free or
> > corruption" error.
> > 
> > So...
> > 
> > I'm guessing I'm going to have to come up with a more sensible way of
> > laying out my directories or maybe there is a better way of dealing
> > with this.
> 
> Nope.  You're directory structure is fine.  There's no reason you need a
> raw field in the top level fragment.  GetData should be able to handle
> things.  Let me see if I can reproduce this.
> 
> -dvw

Yep, GetData bug.  It'll be fixed in 0.7.2 which will be released in the
next week or so.  But, if you're anxious, all you need to change in the
0.7.1 source is line 483 of add.c from

  D->fragment[i].ref_name = (char *)new_ref;

to

  D->fragment[i].ref_name = strdup(new_ref);

Thanks for the investigation Ross,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

Re: [getdata-devel] Flush time + usage philosophy

[D. V. W. <ge...@ke...>]

On Tue, Mar 22, 2011 at 06:20:38PM -0500, Ross Williamson wrote:
> OK I think I'm getting there.
> 
> The problem seems to be the way that I've setup my directory
> structure.  Even though I create a top-level dirfile I don't actually
> have any fields in there. The format file just includes the
> subdirectories.  That toplevel dirfile however still requires a
> /REFERENCE so it takes it to be the first RAW entry inside my dirfile
> hierarchy - in this case it is array::frame::status (where array and
> frame are also dirfiles).  When call to free takes place it happy
> deletes the REFERENCE in the toplevel. (i.e. fragment[0] deletes
> fine). It then moves onto fragment[1] (array) which also has
> array::frame::status as it's REFERENCE. It tries to delete it (but
> it's already been deleted) and hence we get the "double free or
> corruption" error.
> 
> So...
> 
> I'm guessing I'm going to have to come up with a more sensible way of
> laying out my directories or maybe there is a better way of dealing
> with this.

Nope.  You're directory structure is fine.  There's no reason you need a
raw field in the top level fragment.  GetData should be able to handle
things.  Let me see if I can reproduce this.

-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

Re: [getdata-devel] Flush time + usage philosophy

[Ross W. <ros...@gm...>]

OK I think I'm getting there.

The problem seems to be the way that I've setup my directory
structure.  Even though I create a top-level dirfile I don't actually
have any fields in there. The format file just includes the
subdirectories.  That toplevel dirfile however still requires a
/REFERENCE so it takes it to be the first RAW entry inside my dirfile
hierarchy - in this case it is array::frame::status (where array and
frame are also dirfiles).  When call to free takes place it happy
deletes the REFERENCE in the toplevel. (i.e. fragment[0] deletes
fine). It then moves onto fragment[1] (array) which also has
array::frame::status as it's REFERENCE. It tries to delete it (but
it's already been deleted) and hence we get the "double free or
corruption" error.

So...

I'm guessing I'm going to have to come up with a more sensible way of
laying out my directories or maybe there is a better way of dealing
with this.

Cheers

Ross

On Tue, Mar 22, 2011 at 5:17 PM, D. V. Wiebe <ge...@ke...> wrote:
> On Tue, Mar 22, 2011 at 04:34:30PM -0500, Ross Williamson wrote:
>> Ok so I took it upon myself to have a go at hacking close.c. I think
>> the issue might the counting of fragments.  I have a simple situation
>> where there are 4 fragments inside a top level dirfile.  I initially
>> looked at the value of D-> n_fragment which returned a value of 5.
>> This seemed odd as there are only 4.
>
> You have the top level fragment (the "format" file) plus four
> INCLUDEd subfragments?  That's five fragments total.
>
> You can map fragment indices to filenames using gd_fragmentname().  If
> the count is wrong, that function should fail for the highest number.
>
>> I also though we might need to deallocate the fragments in reverse
>> order - I changed the loop code to (making the mistake that I should
>> have done j>=0)
>>
>> for (j = D->n_fragment; j>0; j--)
>
> At this point we're just getting rid of everything.  It doesn't matter
> what order we do that in.
>
>> Whooo - This worked. Changing it to j>=0 crapped out at index
>> fragment[0].  Just to make sure I changed the code to:
>>
>> for(j=1; j<D->n_fragment; ++j) and that also worked - so it seems that
>> there are one two many fragments in n_fragment and fragment[0] does
>> not exist.
>
> Considering this works as-is in multiple places in the test suite, I'd
> consider it more likely that what has happened is that something has
> corrupted the data in the fragment[0] structure at some point, which
> results in the crash.
>
>> Am I playing with fire here?
>
> There are four strings freed() in that loop.
>
> D->fragment[0].sname and D->fragment[0].ename should be NULL.  (Check --
> are they?)
>
> D->fragment[0].cname is set when the dirfile is first created.  It
> should never change.  Try printing it both when the dirfile is created
> (around open.c:345) and then in close.c just before they're free()d.
> Does it change?
>
> D->fragment[0].ref_name is initially NULL, but then gets set to the name
> of the reference field (the first raw field defined, if you don't
> explicitly set the reference field).  Try printing it just before
> free()ing it.  Does it make sense?
>
> You never mentioned if this is the 0.7.1 tarball you're playing with,
> or SVN.
>
> Good luck,
> -dvw
> --
> D. V. Wiebe
> ge...@ke...
> http://getdata.sourceforge.net/
>



-- 
Ross Williamson
University of Chicago
Department of Astronomy & Astrophysics
773-834-9785 (office)
312-504-3051 (Cell)

Re: [getdata-devel] Flush time + usage philosophy

[D. V. W. <ge...@ke...>]

On Tue, Mar 22, 2011 at 04:34:30PM -0500, Ross Williamson wrote:
> Ok so I took it upon myself to have a go at hacking close.c. I think
> the issue might the counting of fragments.  I have a simple situation
> where there are 4 fragments inside a top level dirfile.  I initially
> looked at the value of D-> n_fragment which returned a value of 5.
> This seemed odd as there are only 4.

You have the top level fragment (the "format" file) plus four
INCLUDEd subfragments?  That's five fragments total.

You can map fragment indices to filenames using gd_fragmentname().  If
the count is wrong, that function should fail for the highest number.

> I also though we might need to deallocate the fragments in reverse
> order - I changed the loop code to (making the mistake that I should
> have done j>=0)
> 
> for (j = D->n_fragment; j>0; j--)

At this point we're just getting rid of everything.  It doesn't matter
what order we do that in.

> Whooo - This worked. Changing it to j>=0 crapped out at index
> fragment[0].  Just to make sure I changed the code to:
> 
> for(j=1; j<D->n_fragment; ++j) and that also worked - so it seems that
> there are one two many fragments in n_fragment and fragment[0] does
> not exist.

Considering this works as-is in multiple places in the test suite, I'd
consider it more likely that what has happened is that something has
corrupted the data in the fragment[0] structure at some point, which
results in the crash.

> Am I playing with fire here?

There are four strings freed() in that loop.

D->fragment[0].sname and D->fragment[0].ename should be NULL.  (Check --
are they?)

D->fragment[0].cname is set when the dirfile is first created.  It
should never change.  Try printing it both when the dirfile is created
(around open.c:345) and then in close.c just before they're free()d.
Does it change?

D->fragment[0].ref_name is initially NULL, but then gets set to the name
of the reference field (the first raw field defined, if you don't
explicitly set the reference field).  Try printing it just before
free()ing it.  Does it make sense?

You never mentioned if this is the 0.7.1 tarball you're playing with,
or SVN.

Good luck,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/