How about if we set use_txn = false if python_version < 2.5 ?
----- Original Message ----
From: Don Allingham <donaldallingham@...>
To: Steve Hall <digitect@...>
Cc: Alex Roitman <shura@...>; gramps-windows@...; Gramps users <gramps-users@...>
Sent: Wednesday, September 13, 2006 10:25:48 PM
Subject: Re: [Gramps-users] [Gramps-windows] Database Loading Issue
So, basically you are staying that we should hold up GRAMPS until all
distributions are shipping 2.5 because databases aren't portable? This
seems a bit severe. Chances are, very few people will actually try to
move a database from one machine to another. Most people only have one
There are several mechanisms that exist that will allow portability
(GRAMPS XML and GRAMPS Package - which is clearly labeled as "portable
XML"), If we hold up GRAMPS for every problem the program has, we will
never release - and this is true of every program.
On Wed, 2006-09-13 at 23:10 -0400, Steve Hall wrote:
> Thanks for the detailed report.
> IMO, it looks like the solution is to simply say that GRAMPS 2.1
> depends on Python 2.5. This new version of Python is already at it's
> second (and reportedly last) release candidate, I don't see that a
> little more waiting is as much of a problem as the non-portability of
> our application files.
> Until then, I guess we're stuck with the export work-around as a
> hiccup of beta testing. It seems prudent to wait for Python for the
> final release though, especially since the current GRAMPS development
> pace is about the same track.
> On Wed, 2006-09-13 at 13:57 -0700, Alex Roitman wrote:
> > Steve,
> > On Wed, 2006-09-13 at 08:48 -0700, Steve Hall wrote:
> > > >
> > > > Gramps databases are NOT portable. This means you can not move
> > > > your familytree.gramps file from linux to windows and expect it
> > > > to be safe. You MUST export to XML from Linux and import the XML
> > > > on Windows.
> > > >
> > > > Not only that, but you can't even move your file from one Linux
> > > > box to another. Any time your gramps file leaves you computer,
> > > > you must export it.
> > > >
> > > > Portability may change with future versions of python, but for
> > > > now, this is what we are stuck with.
> > >
> > > This is a horrible situation, it is unreasonable to require a user
> > > to manage what the application is supposed to do. Can you imagine
> > > the reaction if word processors required this?
> > We learned about this limitation a few months ago, well into
> > developing the 2.1.x branch. It is a horrible problem, I absolutely
> > agree. I've been trying to get people's attention to it for a while.
> > Now that few people actually tried 2.1.x we may get some meaningful
> > discussion on this matter. Before all my calls were just the vapor
> > to everybody :-)
> > > If GRAMPS files can not be exchanged AS IS with other GRAMPS
> > > users, then the program has a fundamental problem.
> > Let me describe the situation in detail.
> > The 2.0.x versions used grdb files that did not use the
> > transactional environment. The new 2.1/2.2 series can work both with
> > and without the transactional environment (TXN for short).
> > The benefits of the transactional environment are:
> > 1. Data can never be corrupted on the low level.
> > Before flaming me for an outright lie, let me explain
> > what this means. The changes to the data are sometimes
> > simple (changing a note will only affect one object,
> > so exactly one piece of data will be modified). But sometimes
> > they are complex (adding a new child to the family (1) adds
> > a new person, (2) adds that person to the family). There
> > are even more complex changes, involving the modifications
> > of several records in several database tables.
> > With the TXN in places, the whole transaction is either
> > committed or abandoned, so the database is always in the
> > consistent state. Even if the program crashes in the middle
> > of the transaction, either the whole thing will be committed
> > or not at all. As many people noticed when testing unstable
> > 2.1.x releases, no amount of crashing in whatever places
> > (well, except initial upgrade from 2.0.x) can damage the data.
> > This is no small matter, and is very important to anybody
> > working with large data files.
> > 2. The performance. The TXN uses logging, and we currently utilize
> > asynchronous transaction logging, so that we don't have to
> > physically write and clear the logs for every transaction.
> > This allows us to transactionally write much faster than we did
> > in 2.0.x before the transactions. For anybody who tried to
> > import huge dataset in 2.0.x and in 2.1.x this performance gain
> > is pretty obvious.
> > BUT, as we learned well into developing 2.1 branch, the TXN-capable
> > databases cannot be simply taken away and opened in another
> > environment (that is, another machine, or even another user on the
> > same machine). Sometimes it works, depending on how the log files
> > are synced and released, but we have no control over that. So we
> > cannot count on this.
> > The workarounds could be either copy the whole environment directory
> > (the ~/.gramps/bsddbenv dir), or try to "clear" the connection
> > between the environment and the datafile. While the bsddb libraries
> > (libdb4.3) have the necessary methods, the python bindings were
> > lacking. Because of the bug reports that we filed the necessary
> > method has been added only recently.
> > So the next version of python (2.5) will allow us to disassociate
> > the database from the environment dir on cleanly closing the db. In
> > fact, the code is already in place, so if the method is available in
> > python then it will be automatically used.
> > Now, what are we to do in the meantime? The following steps are the
> > workarounds of different degree of ugliness:
> > 1. Use XML format natively: open XML files, work with them.
> > Downside: slow to open, the whole data will stay in memory. OK for
> > small files, but impractical for large files.
> > 2. Work in GRDB and export into XML for portability/archiving.
> > 3. Always drag the ~/.gramps/bsddbenv directory along with the file.
> > Downside: very cumbersome and inconvenient.
> > 4. We may disable the TXN functionality, or make it a
> > user-adjustable parameter. This way the users would decide whether
> > they want to trade portability of grdb files for
> > reliability/performance. The TXN-grdb is fast and reliable (until
> > you move it), but not portable.
> > None of this is really good. But we're in a tough situation here: we
> > want TXN and portability. Hopefully, python2.5 will be released soon
> > and solve the problem. However, realistically it will be a while
> > until python2.5 is shipped in every distro.
> > I sincerely welcome any feedback on what to do here. If people want
> > to experiment with portable TXN-less grdbs, right now it can be done
> > by setting the "UseTXN = False" (instead of the current True) in the
> > src/GrampsDb/_GrampsBSDDB.py file in the source tree or in
> > /usr/share/gramps/GrampsDb/_GrampsBSDDB.py in your installed version
> > (or wherever it is installed -- ymmv).
> > If there are any bsddb experts or any database experts that can
> > provide useful advice, we need you now!
> > Alex
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> Gramps-windows mailing list
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
Gramps-users mailing list