Re: [Gramps-devel] Current Experiments with Gramps-connect

[Doug B. <dou...@gm...>]

FYI, another update on Gramps-connect.

Now that we are using an exact copy of the BSDDB data format as a
cache in the Django database, things are falling into place. This week
adds the ability for the website to run all reports (except those that
try to open up Gtk windows) in whatever format you want (text, PDF,
etc), and also export directly from the Django databases. That means
you can go to a Gramps-connect site and download an up-to-date version
of the data in Gramps XML or GEDCOM (etc).

What also works now is importing directly into the website. Prior to
this week, if you wanted to populate a Gramps-connect database, one
had to run Gramps GTK and do an Export from there into a Django
Database (sqlite, mysql, postgresql, etc) that you had configured
locally. Now, you can go to the website, upload a GEDCOM or Gramps XML
package (or provide a URL of a family tree) and Gramps-connect will
import this over the web into the database. (Well, this works in code
manually run on the server, but there isn't a web interface to the
importer yet).

Importing directly into the Django tables was an interesting problem
because the import code was not written with a relational database in
mind. In Django, you can't save a person's details until all of the
items you are going to link to have also been entered into their
tables. So, the import code for the Django tables pretends to import
all of the data. At this point you could see an overview of exactly
what is going to happen, before it actually changes any data. When the
standard Gramps import process is complete, a gramps-connect
post-process will go through the data which was attempted to be
imported, and will actually add all of the primary data to their
tables. Then the post-process goes through the data a second time, and
adds all of the linked data. The tests I have done so far shows that
this is working fine (thank you idempotent input/output!) Currently,
all of the imported data is residing in memory before the
post-processing begins, but that would be easy to add a temporary
database to remove that issue.

I think that this two-step method would also work with Gramps Gtk so
that we could have a real merge on import, or perhaps only enter some
of the data from an import, and discard the rest.

I'll hook up the importer on the website (http://gramps-connect.org)
as soon as I can, and people can try importing family trees from
around the web. This will be useful for seeing how Gramps-connect
works under large datasets. Now, I need to write some documentation...

BTW, I've moved trunk/src/web to trunk/src/webapp to make it clear
that everything in this directory is for the webapp. Trunk users will
want to ./autogen.sh and make to get everything redefined.

-Doug