getdata-devel — General discussion of GetData development.

[getdata-devel] GetData 0.4.2 released

[D. V. W. <ge...@ke...>]

GetData 0.4.2 has been released.  It may be downloaded from your local
SourceForge mirror:

http://sourceforge.net/project/showfiles.php?group_id=229287&release_id=640554

This bug-fix release fixes a few bugs found in 0.4.1 relating to slim
and ASCII encoded files, and incorrect protection levels.  This release also
contains fixes to better recreate the legacy API.

Full deltails are provided in the release notes:

http://sourceforge.net/project/shownotes.php?group_id=229287&release_id=640554

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

Re: [getdata-devel] getdata() bug using libslim

[D. V. W. <ge...@ke...>]

On Mon, Nov 17, 2008 at 02:49:23PM -0500, Adam D. Hincks wrote:
> Hi Don,
> 
> In version getdata-0.4.1, _GD_SlimRead returns the number of members read 
> instead of the number of bytes read.  This is because slimread() works 
> like fread() as opposed to read().
> 
> To fix this, I multiply the return value of slimread() by 
> GD_SIZE(data_type).  The modified function is:
> 
> <snip>
> ssize_t _GD_SlimRead(struct _gd_private_entry *entry, void *ptr,
>      gd_type_t data_type, size_t nmemb)
> {
>    return slimread(ptr, GD_SIZE(data_type), nmemb, entry->stream) *
>           GD_SIZE(data_type);
> }
> </snip>

> 
> Can you make this change in the next release?
> 
> Thanks,
> 
> Adam

Done!

-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

[getdata-devel] getdata() bug using libslim

[Adam D. H. <ah...@Pr...>]

Hi Don,

In version getdata-0.4.1, _GD_SlimRead returns the number of members read 
instead of the number of bytes read.  This is because slimread() works 
like fread() as opposed to read().

To fix this, I multiply the return value of slimread() by 
GD_SIZE(data_type).  The modified function is:

<snip>
ssize_t _GD_SlimRead(struct _gd_private_entry *entry, void *ptr,
     gd_type_t data_type, size_t nmemb)
{
   return slimread(ptr, GD_SIZE(data_type), nmemb, entry->stream) *
          GD_SIZE(data_type);
}
</snip>

Can you make this change in the next release?

Thanks,

Adam

Re: [getdata-devel] Need new method...

[D. V. W. <ge...@ke...>]

On Thu, Nov 13, 2008 at 10:53:52AM -0500, Barth Netterfield wrote:
> Hi,
> 
> Kst needs a way to determine what the reference field is, so it can set up a 
> fileWatcher to wait efficiently for changes.
> 
> cbn

That functionality (along with methods to query all global dirfile
metatdata) should be available in the next feature release.

-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

[getdata-devel] Need new method...

[Barth N. <net...@ph...>]

Hi,

Kst needs a way to determine what the reference field is, so it can set up a 
fileWatcher to wait efficiently for changes.

cbn

Re: [getdata-devel] GetData 0.4.0

[Adam D. H. <ah...@Pr...>]

The dirfile standards page (on the SourceForge) doesn't specify if LINTERP 
paths can be relative.  I believe in the original getdata they couldn't be 
-- can they be now? -AH

On Fri, 31 Oct 2008, D. V. Wiebe wrote:

> GetData 0.4.0 has been released.  Thanks to Matthew Truch and Barth
> Netterfield for beta testing it.  It may be downloaded from your
> local SourceForge mirror:
>
> http://sourceforge.net/project/showfiles.php?group_id=229287
>
> Among other things, changes in GetData 0.4.0 include:
> * Support for the slim compression scheme used by ACT
> * The ability to add fields
> * A duplicate field check on open
> * Contents of the DIRFILE struct is now completely private.
> * Support for Dirfile Standards Version 6 (see below)
>
> GetData 0.4.0 also co-incides with yet another update to the Dirfile
> Standards (Version 6). Hopefully this will be the last such update for
> a while.
>
> Highlights of Standards Version 6 are:
> * Both numerical and string scalar constant fields
> * Metafields (ie. sub-fields attached to a parent field)
>
> The release notes contain a complete list of changes from the previous
> release:
>
> http://sourceforge.net/project/shownotes.php?group_id=229287&release_id=637331
>
> Cheers,
> -dvw
> -- 
> D. V. Wiebe
> ge...@ke...
> http://getdata.sourceforge.net/
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Getdata-devel mailing list
> Get...@li...
> https://lists.sourceforge.net/lists/listinfo/getdata-devel
>

[getdata-devel] Getdata 0.4.1 released

[D. V. W. <ge...@ke...>]

GetData 0.4.1 has been released.  It may be downloaded from your local
SourceForge Mirror:

http://sourceforge.net/project/showfiles.php?group_id=229287&release_id=638941

This bug-fix release allows the full stop character "." in field names in
certain situations (which is still a violation of Standards Version 6).  This
change is necessary for GetData to be able to read the BLAST flight data.

This release also contains several minor bug fixes and speed
improvements.  Full deltails are provided in the release notes:

http://sourceforge.net/project/shownotes.php?group_id=229287&release_id=638941

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

[getdata-devel] GetData 0.4.0

[D. V. W. <ge...@ke...>]

GetData 0.4.0 has been released.  Thanks to Matthew Truch and Barth
Netterfield for beta testing it.  It may be downloaded from your
local SourceForge mirror:

http://sourceforge.net/project/showfiles.php?group_id=229287

Among other things, changes in GetData 0.4.0 include:
* Support for the slim compression scheme used by ACT
* The ability to add fields
* A duplicate field check on open
* Contents of the DIRFILE struct is now completely private.
* Support for Dirfile Standards Version 6 (see below)

GetData 0.4.0 also co-incides with yet another update to the Dirfile
Standards (Version 6). Hopefully this will be the last such update for
a while.

Highlights of Standards Version 6 are:
* Both numerical and string scalar constant fields
* Metafields (ie. sub-fields attached to a parent field)

The release notes contain a complete list of changes from the previous
release:

http://sourceforge.net/project/shownotes.php?group_id=229287&release_id=637331

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

[getdata-devel] Getdata 0.3.1 released

[D. V. W. <ge...@ke...>]

Version 0.3.1 of GetData has been released. This release fixes a bug in
the legacy API which prevented calling GetData() on a read-oly dirfile.
For full details, see the release notes:
http://sourceforge.net/project/shownotes.php?group_id=229287&release_id=628542

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

Re: [getdata-devel] feature request: adding entries to format file

[D. V. W. <ge...@ke...>]

On Tue, Sep 16, 2008 at 06:26:00AM -0700, bc...@jp... wrote:
> Hi folks,
> 
> Don asked for some feedback on what should be done with this new  
> getdata library.  Some common operations I do with my own code that  
> aren't yet in getdata are:
> 
> 1. create new dirfiles

The dirfile_open() function can create a new dirfile.  (But since your
item #3 isn't implemented yet, that's of limited usefulness...)

> 2. check for the existence of a field in a dirfile

I think this is possible with get_entry(), since it will return -1 if
the requested field doesn't exist.  Although, it might be useful to have
a more explicit means of doing this.

> 3. add a new field to a dirfile

This is what I meant when I was talking about metadata editing being
on the short-listed ToDo.

> Even though these are fairly straightforward tasks it would be nice to  
> have them as part of the library.

If you can point me to your modified code, I'd love to take a look at it
and see what I can port over.

Cheers,
-dvw
-- 
D. V. Wiebe
ge...@ke...
http://getdata.sourceforge.net/

Re: [getdata-devel] First stand-alone GetData release

[Adam D. H. <ah...@Pr...>]

Hi Don,

I just had a look at the website, and the new version looks great.

I think the main thing that would prevent me at this stage from using it 
is that I need Joe's slimlib to be in it.  Joe has created a getdata 
version that seamlessly integrates slimlib -- i.e., you don't even need to 
tell it whether a dirfile is slimmed or not, it figures it out for you.  I 
have to say that slimlib works really well and I haven't noticed a hit in 
read-in time (though I haven't scrutinised it).

Adam

On Mon, 15 Sep 2008, D. V. Wiebe wrote:

> I've finally released a first version of the stand-alone GetData
> library (versioned somewhat arbitrarily as 0.3.0).  It's being managed
> as a SourceForge project:
>
>  http://getdata.sourceforge.net/
>
> Most of the effort in this new release has gone into creating a new
> interface (API) for the library.  This was done to overcome some
> deficiencies without breaking the old interface (the legacy API).  The
> legacy API is still supported, having been re-implemented as a wrapper
> around the new interface, and programs using the legacy API should be
> able to use this new version without need for modification (re-linking
> might be necessary).
>
> This release does little more than re-create the existing library
> behaviour, but also includes some extensions to the Dirfile Standards
> themselves (which have been formally codified for the first time).  Notes for
> this release are available:
>
> http://sourceforge.net/project/shownotes.php?release_id=626480&group_id=229287
>
> On the ToDo short-list for the next release is metadata manipulation.
> Also outstanding is the slimlib question, which I really haven't looked
> at yet.  Barth has some further Dirfile extensions planned for the near
> future as well, involving optional metadata fields in the format file.
>
> I'm interested in feedback from you guys on the new interface, what
> still needs to be done (if anything) to enable those of you using
> GetData to switch to this version of getdata from your in-house
> version, and any other comments.
>
> Cheers,
> -dvw
> PS. Astute observers may notice that I've also created a getdata mailing
> list: get...@li... which might be a good place
> for this sort of discussion.
> -- 
> Don Wiebe                                   dw...@ph...
> Dept. of Physics/Univ. of Toronto
> 60 St. George St.                           Tele: +1-416-946-0946
> Toronto ON                                  Lab:  +1-416-946-0516
> Canada   M5S 1A7                            http://ketiltrout.net/
>

[getdata-devel] feature request: adding entries to format file

[<bc...@jp...>]

Hi folks,

Don asked for some feedback on what should be done with this new  
getdata library.  Some common operations I do with my own code that  
aren't yet in getdata are:

1. create new dirfiles
2. check for the existence of a field in a dirfile
3. add a new field to a dirfile

Even though these are fairly straightforward tasks it would be nice to  
have them as part of the library.

Thanks again for taking on this project, Don!

Brendan

[getdata-devel] The stand-alone GetData and kst

[D. V. W. <dv...@ke...>]

I've released a stand-alone version of the C GetData library, available
from SourceForge:

http://getdata.sourceforge.net/

My plan (in consultation with Barth) is to replace the getdata code
(getdata.c &c.) in the kst-2.x tree with an external dependency on this
stand-alone library, like many of the kst data sources are.  I will
begin the job of incorporating these changes into the kst-2.x tree soon
(one we're certain the GetData release is reasonably useable.)

Having a stand-alone GetData library is necessary to address the
multiple forks of GetData which have developed at the various
experimental collaborations which use GetData for quick-look and data
analysis.

My questions for the kst developers are these:

Q1:  Should an external dependency to this stand-alone version of GetData
     replace the getdata library in the kst-1.x tree as well?  I see three
     possible answers to this question, none of which I'm particularly
     unhappy with:

A1a: Yes.

A1b: No, but the source code for the new GetData release should be
     copied into the kst-1.x tree.

A1c: No, but bug-fixes made in the new GetData release should be
     back-ported to the kst-1.x tree.



Q2:  How should we encourage the external GetData dependency in kst-2.x
     (and potentially 1.x) to be handled by downstream package creators?
     Is a kst without dirfile support of practical use?  I see two
     posibilities:

A2a: Let the downstream packager worry about it completely.  Encourage
     them to build kst against a GetData library which they would also
     have to package and distribute.  If they don't do this, their kst
     packages will not contain getdata support.

A2b: Have the kst-2.x configure script check for an external GetData
     library, and use it if found, but build an internal version
     distributed with the tarball if it doesn't find one.  This would
     likely result in most downstream packagers creating a kst package
     which also contains getdata.

Of thses, I prefer A2a, since A2b prevents users of the kst package
from upgrading their GetData library separately from kst, although it
does ensure all kst packages will contain GetData support.

-dvw
-- 
Don Wiebe                                   dw...@ph...
Dept. of Physics/Univ. of Toronto
60 St. George St.                           Tele: +1-416-946-0946
Toronto ON                                  Lab:  +1-416-946-0516
Canada   M5S 1A7                            http://ketiltrout.net/

[getdata-devel] First stand-alone GetData release

[D. V. W. <dv...@ke...>]

I've finally released a first version of the stand-alone GetData
library (versioned somewhat arbitrarily as 0.3.0).  It's being managed
as a SourceForge project:

  http://getdata.sourceforge.net/

Most of the effort in this new release has gone into creating a new
interface (API) for the library.  This was done to overcome some
deficiencies without breaking the old interface (the legacy API).  The
legacy API is still supported, having been re-implemented as a wrapper
around the new interface, and programs using the legacy API should be
able to use this new version without need for modification (re-linking
might be necessary).

This release does little more than re-create the existing library
behaviour, but also includes some extensions to the Dirfile Standards
themselves (which have been formally codified for the first time).  Notes for
this release are available:

http://sourceforge.net/project/shownotes.php?release_id=626480&group_id=229287

On the ToDo short-list for the next release is metadata manipulation.
Also outstanding is the slimlib question, which I really haven't looked
at yet.  Barth has some further Dirfile extensions planned for the near
future as well, involving optional metadata fields in the format file.

I'm interested in feedback from you guys on the new interface, what
still needs to be done (if anything) to enable those of you using
GetData to switch to this version of getdata from your in-house
version, and any other comments.

Cheers,
-dvw
PS. Astute observers may notice that I've also created a getdata mailing
list: get...@li... which might be a good place
for this sort of discussion.
-- 
Don Wiebe                                   dw...@ph...
Dept. of Physics/Univ. of Toronto
60 St. George St.                           Tele: +1-416-946-0946
Toronto ON                                  Lab:  +1-416-946-0516
Canada   M5S 1A7                            http://ketiltrout.net/