From: Steve H. <dig...@mi...> - 2003-11-29 21:01:08
|
I'm seriously considering using GRAMPS as my main genealogy software, but am very anxious not to get my data stuck in a given format. Obviously, I know GRAMPS is Free Software, but the data is stored binary and I'm curious if the GRAMPS export function to .GED v5.5 is complete. Are there any fields of data not pushed out through this export? What about images, notes, attributes, and addresses? If I know I'm saving off *everything* to a mostly standard file format, I'll be comfortable backing up via export. Also, I'd appreciate knowing any experiences or advice anyone here can offer regarding the best format to store genealogical data. I've been a Family Origins user since about 1993, but since its abandonment <rant>by the pirates who over-ran the place</rant> I'm less than eager to trust my data to anyone else. FO has been around for a while, as have many others. PAF is often said to be a standard, too, but I wonder how much in these formats is read by other software and how much is proprietary extension that no one else can use. (FO certainly had some.) Has this been hashed here or elsewhere before? Thanks. -- Steve Hall [ dig...@mi... ] |
From: Alex R. <sh...@al...> - 2003-11-29 21:19:03
|
On Sat, Nov 29, 2003 at 04:01:12PM -0500, Steve Hall wrote: >=20 > I'm seriously considering using GRAMPS as my main genealogy software, > but am very anxious not to get my data stuck in a given format. > Obviously, I know GRAMPS is Free Software, but the data is stored > binary ... Well, not really. the data so far is stored in XML format. For one, it's text and it's human readable. If worse comes to worst (although why=20 should it?) you can always extract your data using simple scripts=20 writtent in perl, python, or whatever else you're comfortable with. You=20 also can always use XSLT programming to access the XML data. All that=20 can be sort of last resort if gramps is not in the picture some day when=20 you want to work with your data. > and I'm curious if the GRAMPS export function to .GED v5.5 is > complete. Are there any fields of data not pushed out through this > export?=20 I think that everything is getting saved into GEDCOM.=20 > What about images, notes, attributes, and addresses? If I know > I'm saving off *everything* to a mostly standard file format, I'll be > comfortable backing up via export. Images are kept as separate files, no changes are made to images when=20 saving from gramps XML to GEDCOM. The gramps XML stores only references=20 to the images (or whatever other objects you may have -- sounds, movies,=20 etc). These references will be promptly saved into GEDCOM and the files=20 (images, movies, etc) are placed right by the GEDCOM file. > Also, I'd appreciate knowing any experiences or advice anyone here can > offer regarding the best format to store genealogical data. I can't be of any help out here, sorry :-) Alex --=20 Alexander Roitman http://ebner.neuroscience.umn.edu/people/alex.html Dept. of Neuroscience, Lions Research Building 2001 6th Street SE, Minneapolis, MN 55455 Tel (612) 625-7566 FAX (612) 626-9201 |
From: Richard B. <ra...@xs...> - 2003-11-29 21:39:57
|
Op zaterdag 29 november 2003 22:01, schreef Steve Hall: > Obviously, I know GRAMPS is Free Software, but the data is stored > binary It is a gzipped. Use gunzip to get it in readable format -- Richard Bos Without a home the journey is endless |
From: Steve H. <dig...@mi...> - 2003-11-29 21:53:33
|
On Sat, 2003-11-29 at 16:39, Richard Bos wrote: > Op zaterdag 29 november 2003 22:01, schreef Steve Hall: > > Obviously, I know GRAMPS is Free Software, but the data is stored > > binary > > It is a gzipped. Use gunzip to get it in readable format Ah! Perfect. For a minute there I was thinking Alex was crazy. :) -- Steve Hall [ dig...@mi... ] |
From: Don A. <dal...@us...> - 2003-11-29 22:11:24
|
On Sat, 2003-11-29 at 14:53, Steve Hall wrote: > On Sat, 2003-11-29 at 16:39, Richard Bos wrote: > > Op zaterdag 29 november 2003 22:01, schreef Steve Hall: > > > Obviously, I know GRAMPS is Free Software, but the data is stored > > > binary > > > > It is a gzipped. Use gunzip to get it in readable format > > Ah! Perfect. For a minute there I was thinking Alex was crazy. :) In your preferences, you can set if you want the file to be compressed or uncompressed. It makes no difference from a speed perspective, just from a file size perspective. Even when GRAMPS goes to a real database backend, it will always be able to import and export all of its data to an XML file. Being a long time UNIX guy (since about '85), I understand the importance of text files. GEDCOM is the only interchange format that I am aware of. All the other programs have proprietary formats. The thing you have to watch out for is that GEDCOM is a lousy interchange format. It has quite a few deficiencies, especially when dealing the multiple families and family relationships. Try to use GEDCOM to describe a family where one parent is the biological parent and the other is an adopted parent, and you will end up pulling your hair out. Many programs simply refuse to export this situation. Others, like Family Tree Maker, invent their own extensions to GEDCOM. Of course, exporting a proprietary extension to an interchange format is almost useless. No one else understands it, and you just break compatibility. GRAMPS does its best to understand the extensions of each program, and tries to adapt to each one, but its a far from perfect situation. Don |
From: Steve H. <dig...@mi...> - 2003-11-29 22:33:15
|
From: Don Allingham, Sat Nov 29 17:11:18 2003 > On Sat, 2003-11-29 at 14:53, Steve Hall wrote: > > On Sat, 2003-11-29 at 16:39, Richard Bos wrote: > > > > > > It is a gzipped. Use gunzip to get it in readable format > > > > Ah! Perfect. For a minute there I was thinking Alex was crazy. :) > > In your preferences, you can set if you want the file to be > compressed or uncompressed. It makes no difference from a speed > perspective, just from a file size perspective. Can I place a (ultra-low priority) wish list request that [myfile].gramps and [myfile].gramp.gz be default extension offerings on SaveAs? ;) > Even when GRAMPS goes to a real database backend, it will always be > able to import and export all of its data to an XML file. Being a > long time UNIX guy (since about '85), I understand the importance of > text files. That was what I was looking to hear. :) > The thing you have to watch out for is that GEDCOM is a lousy > interchange format. It has quite a few deficiencies, especially when > dealing the multiple families and family relationships. Try to use > GEDCOM to describe a family where one parent is the biological > parent and the other is an adopted parent, and you will end up > pulling your hair out. Per GEDCOM 6 proposal (http://www.familysearch.org/GEDCOM/GedXML60.pdf), it looks this is being solved. I wonder what else is improved that you know is broken? > Many programs simply refuse to export this situation. Others, like > Family Tree Maker, invent their own extensions to GEDCOM. > > Of course, exporting a proprietary extension to an interchange > format is almost useless. No one else understands it, and you just > break compatibility. This is what Family Origins did. Probably most apps that have been around for a while dealt in different ways. I know FO would change extensions every now and then, requiring data migration at upgrades. It always scared me. > GRAMPS does its best to understand the extensions of each program, > and tries to adapt to each one, but its a far from perfect > situation. This thread has been a good education, but ultimately I'm glad to know GRAMPS is Free, the files text/XML, and the author committed to good design. :)) -- Steve Hall [ dig...@mi... ] |
From: Don A. <dal...@us...> - 2003-11-29 23:33:28
|
On Sat, 2003-11-29 at 15:33, Steve Hall wrote: > Can I place a (ultra-low priority) wish list request that > [myfile].gramps and [myfile].gramp.gz be default extension offerings > on SaveAs? ;) > I like this idea, and have thought about implementing it a couple of times in the past, but I issue I have is what do we do if both a data.gramps and a data.gramps.gz file both exist in the same directory? Don -- Don Allingham <dal...@us...> GRAMPS - Open Source Genealogy |
From: Steve H. <dig...@mi...> - 2003-11-30 01:53:17
|
From: Don Allingham, Sat Nov 29 18:33:22 2003 > On Sat, 2003-11-29 at 15:33, Steve Hall wrote: > > > > Can I place a (ultra-low priority) wish list request that > > [myfile].gramps and [myfile].gramp.gz be default extension > > offerings on SaveAs? ;) > > I like this idea, and have thought about implementing it a couple of > times in the past, but I issue I have is what do we do if both a > data.gramps and a data.gramps.gz file both exist in the same > directory? I guess I'm really foiled by the fact that GRAMPS names/looks for a directory rather than a file. If the package is gzipped, why does it need a container directory? Why not have a single file with whatever structure within? (A la OO.o.) Then the name or presence of compression don't matter. And users like me can rename, email, backup, and restore to heart's content with singular files without also having to deal with directory structure. Or am I missing something? -- Steve Hall [ dig...@mi... ] |
From: Don A. <dal...@us...> - 2003-11-30 04:00:23
|
On Sat, 2003-11-29 at 18:53, Steve Hall wrote: > I guess I'm really foiled by the fact that GRAMPS names/looks for a > directory rather than a file. If the package is gzipped, why does it > need a container directory? Why not have a single file with whatever > structure within? (A la OO.o.) Then the name or presence of > compression don't matter. And users like me can rename, email, backup, > and restore to heart's content with singular files without also having > to deal with directory structure. Or am I missing something? The problem is that a GRAMPS database can consist of more than the data.gramps file. It can also consist of multimedia files, backup files, and versioning files. (When you create a local copy of a multimedia file, GRAMPS makes its own copy, so that if the original is deleted, GRAMSP still has its own copy). I don't like the directory thing either. I'd like a better option. Don |
From: Steve H. <dig...@mi...> - 2003-11-29 22:11:22
|
From: Alex Roitman, Sat Nov 29 15:18:57 2003 > On Sat, Nov 29, 2003 at 04:01:12PM -0500, Steve Hall wrote: > > > > Obviously, I know GRAMPS is Free Software, but the data is stored > > binary ... > > Well, not really. the data so far is stored in XML format. Ah, ok after unzipping... > For one, [XML is] text and it's human readable. XML is great. GEDCOM 6 is going XML supposedly, any chance the two formats will merge some day? > Images are kept as separate files, no changes are made to images > when saving from gramps XML to GEDCOM. The gramps XML stores only > references to the images (or whatever other objects you may have -- > sounds, movies, etc). These references will be promptly saved into > GEDCOM and the files (images, movies, etc) are placed right by the > GEDCOM file. Cool, this is per GEDCOM. Once again, there's nothing to fear and every reason to love Free Software. Thanks for the response. -- Steve Hall [ dig...@mi... ] |
From: Don A. <dal...@us...> - 2003-11-29 22:21:29
|
On Sat, 2003-11-29 at 15:11, Steve Hall wrote: > XML is great. GEDCOM 6 is going XML supposedly, any chance the two > formats will merge some day? > I need to look at the latest GEDCOM 6 specification. The early drafts I saw had a few issues. While the format is pretty flexible, it had some strange forward referencing issues which made it pretty awkward for using it as a database format. Not that it wouldn't work, just that it would be a lot slower than the GRAMPS format. However, since we are moving to a real database backend, being able to import/export GEDCOM 6 is equally valid to importing and exporting the GRAMPS XML format. on |
From: Alex R. <sh...@al...> - 2003-11-30 05:55:31
|
On Sat, Nov 29, 2003 at 08:58:55PM -0700, Don Allingham wrote: > On Sat, 2003-11-29 at 18:53, Steve Hall wrote: > > I guess I'm really foiled by the fact that GRAMPS names/looks for a > > directory rather than a file. If the package is gzipped, why does it > > need a container directory? Why not have a single file with whatever > > structure within? (A la OO.o.) Then the name or presence of > > compression don't matter. And users like me can rename, email, backup, > > and restore to heart's content with singular files without also having > > to deal with directory structure. Or am I missing something? >=20 > The problem is that a GRAMPS database can consist of more than the > data.gramps file. It can also consist of multimedia files, backup files, > and versioning files. (When you create a local copy of a multimedia > file, GRAMPS makes its own copy, so that if the original is deleted, > GRAMSP still has its own copy). Well, we could pack everything into directory and then compress that=20 directory into a single file, just like OOo does it... Alex --=20 Alexander Roitman http://ebner.neuroscience.umn.edu/people/alex.html Dept. of Neuroscience, Lions Research Building 2001 6th Street SE, Minneapolis, MN 55455 Tel (612) 625-7566 FAX (612) 626-9201 |
From: Don A. <dal...@us...> - 2003-11-30 06:28:40
|
On Sat, 2003-11-29 at 22:55, Alex Roitman wrote: > On Sat, Nov 29, 2003 at 08:58:55PM -0700, Don Allingham wrote: > > On Sat, 2003-11-29 at 18:53, Steve Hall wrote: > > > I guess I'm really foiled by the fact that GRAMPS names/looks for a > > > directory rather than a file. If the package is gzipped, why does it > > > need a container directory? Why not have a single file with whatever > > > structure within? (A la OO.o.) Then the name or presence of > > > compression don't matter. And users like me can rename, email, backup, > > > and restore to heart's content with singular files without also having > > > to deal with directory structure. Or am I missing something? > > > > The problem is that a GRAMPS database can consist of more than the > > data.gramps file. It can also consist of multimedia files, backup files, > > and versioning files. (When you create a local copy of a multimedia > > file, GRAMPS makes its own copy, so that if the original is deleted, > > GRAMSP still has its own copy). > > Well, we could pack everything into directory and then compress that > directory into a single file, just like OOo does it... > > Alex This could probably be done. However, it would require that we would need to constantly extract image files and thumbnails from the archive. I'm not sure what type of impact this would have. Then management of temporary files would get to be a problem, especially when someone wants to use an external viewer. If someone opens up an OpenOffice document that has been added to a gallery, we would have to extract it from an archive to a temporary file, allow them to edit the file, somehow know when they were done changing the file, and then re-add it to the archive and delete the temporary file. The other concern is when we move to a real database, can we work on a database that has been added to an archive? Don |
From: Steve H. <dig...@mi...> - 2003-12-01 04:15:35
|
From: Don Allingham, Sat Nov 29 23:26:57 2003 > On Sat, 2003-11-29 at 22:55, Alex Roitman wrote: > > > > Well, we could pack everything into directory and then compress > > that directory into a single file, just like OOo does it... > > This could probably be done. However, it would require that we would > need to constantly extract image files and thumbnails from the > archive. I'm not sure what type of impact this would have. Then > management of temporary files would get to be a problem, especially > when someone wants to use an external viewer. [WARNING: verbose brainstorming ahead] IMO, storing images and other binary globs *within* the GRAMPS file/store is the wrong approach. Ideally, the perfect genealogical application *suite* has a main genealogy app, with supporting apps that manage images, publishing, cemetery info, census, etc., *each* with it's own file format. The main app supports links to data formats of the others, but it's up to the user to re-link if name/structure changes. Most professional graphics layout software works like this. You have a central app that does the layout, but all the globs are links. When they break or are updated, you are prompted. AutoCAD, too, can reference data files of various other mediums. If a link is somehow broken, it first checks the same directory for a file of a matching name and loads it if found. This is very convenient for packaging and transmitting whole data stores since the data integrety is maintained. I'm frankly very fearful of any app that tries to stash globs of different file formats into one big wad. Sure, *we* all know it's just a gzipped directory structure (now!), but I think the differentiation in file extension would be as helpful for managing app development as it is intuitive for the user. Imagine a structure: Hall.gramps <= genealogy db Hall.ged <= exports... report1.txt report1.pdf hall.html ./www/index.html <= exported web site photo1.tif <= images... photo2.png scan3.jpg map4.svg hall.gallman <= image gallery manager (future :) PerryCo.cemman <= cemetery database manager (future :) 1880.censman <= census database manager (future :) book1.pp <= Passepartout GNOME publishing app (http://www.stacken.kth.se/project/pptout/) invite.stw <= OO.o writer document ToDo.sxc <= OO.o spreadsheet Each part does only what it's good at, as small, atomic pieces. And I think this would be clear enough for the average genealogist to manage, too. Better the file system to manage the truly distinct objects than an app interface. I also think an app storing objects within its file that it can't edit is wrong. Unless GRAMPS envisions being able to edit images, layout, or webpages (even using embedded apps) it shouldn't store anything within its file format that it can't fix without a dependent app. > The other concern is when we move to a real database, can we work on > a database that has been added to an archive? You're talking about a file set that properly references singular atoms in a many-to-many relationship, right? Family Origins used a collection of files, all with the same name, but different extensions. The open dialog always filtered for only one of those, so it was clean for the end user. Except when you wanted to move databases around, and then it got VERY complicated. (Imagine databases named "hall", "hall1", "hall-old", "hall-temp", each with 18 files, when you want to exchange names between two dbs.) Wouldn't the bottleneck in having all the files in one tarball be the gzip/tar each read/write? (I can't imagine file I/O or data updates in RAM would be.) Couldn't this happen transparent to the display changes? Surely no one could edit records quickly enough to get ahead of the disk writes. ("Please wait... previous transaction must be processed before completing this one.") Do many small files g(un)zip faster than a few big ones? It might be interesting to investigate the actual data connectiveness that a genealogist really needs. I'm certain individual relationships and source connections to them are db-proper. But I'm not so certain I need photos, cemetery rubbing scans, or birth certificate scans connected as a db would do it. I can barely imagine an instance where one image would be connected to many records. Perhaps an odd photo of several non-direct-family individuals would be referenced by each of their records. But couldn't this just be a single link to a file, and then db references to that link? Well, I'm definitely out after curfew on the above but I thought it might help to elaborate on one user's opinion. I've been doing genealogy for a while and all I really care about is that the record data is clean, the source references accurately maintained, and the report/drawing/export flexible. The rest of the features found in apps like Family Tree Maker I find cutesy appendages useful only for marketing the product but little assistance (and often annoyance) in the practice of quality genealogy. Most of us genealogists keep collections of paper files in cabinets around the house and prefer to go shuffling through it once in a while in hopes of connecting more of it together. So I'd bet conceeding file management on disk to a big db even more of a hurdle for the user than the developer anyway. -- Steve Hall [ dig...@mi... ] |