Re: [Gramps-users] XML load times

[Joel P. <jj...@ms...>]

I did some more tests.

Again, the two systems (named from my first experience):
"fast" - 2.0 GHz Pentium M, 512 MiB RAM, Windows XP, Gramps 2.2.7
"slow" - 3.06 GHz Pentium 4, 1 GiB RAM, Ubuntu 7.04, Gramps 2.2.8

The file is a Gramps XML file with approx. 700 names. There is a linked
media folder with 67 images of various types totaling 55.1 MiB (several
high-resolution scans).

First, I made identical copies of the base .gramps XML file on each
computer and deleted the media folder, so that all the media links would
be broken.

In this case, both systems took about the same amount of time to load
the file (about 2 minutes).

Second, on the "slow" system, I copied back the media folder and
corrected all the links, so they were all working links to actual media
files, then tried opening the .gramps file again.

In this case, the file load times jumped up into the 15-minute range and
even higher (I killed it after a while). This whole time the CPU was
going at 100%, and the memory usage was creeping higher and higher,
until by the time I killed it, all of my system memory and half of my
1GB swap file was used.

I saw the same behavior on the "fast" system as well. This time it was a
little different, as Gramps loaded into Person view mode in a normal
amount of time, but went crazy with load time and memory when I pressed
the Media button.

Again, to summarize, with no media folder present, both systems loaded
the XML file in a couple minutes. With the media folder present and when
opening into Media view, the load times and memory usage went through
the roof.

What does Gramps try to do with the media during load when it's going
into Media view mode? Does it try to thumbnail all the pictures or load
them into memory or something?

- Joel

On Sat, 2007-08-04 at 09:16 -0600, Don Allingham wrote:
> Something odd is definitely going on here. There was no upgrade between
> 2.2.7 and 2.2.8, so that should not be a problem.
> 
> My guess is the problem is the amount of RAM on the systems. When you
> use the .gramps format, GRAMPS must keep *all* data in memory at once.
> So, if you have more data than you have available RAM, the system will
> dramatically slow down.
> 
> Does the slow machine have less RAM than the faster machine?
> 
> Don
> 
> On Sat, 2007-08-04 at 16:38 +0200, bm...@ca... wrote:
> > Quoting Joel Parker <jj...@ms...>:
> > 
> > > My wife and I are working with a database in Gramps XML format, so we
> > > can share the file without the grdb hassles.
> > >
> > > The database has about 700 individuals, along with maybe 50 media links
> > > (some of which are broken, if that matters). I'm using 2.2.8 on Ubuntu,
> > > and she's using 2.2.7 on Windows XP.
> > 
> > You should use the same version. 2.2.8 corrects not saved privacy settings on
> > save to xml
> > 
> > > On Windows with 2.2.7, she's getting load times of maybe 90 seconds to
> > > open the file (that's after the app has loaded, I do File->Open from
> > > there). On Ubuntu with 2.2.8, I get load times of a full 11 minutes at
> > > 100% CPU, sometimes ending in a "This file is either corrupt or not a
> > > Gramps file" message (and sometimes completing successfully).
> > 
> > I have seen this with .grdb file, but never with .gramps file.
> > If you open an empty grdb file, and do import of xml file, is it faster?
> > 
> > Is this 11 min load times always?
> > 
> > I do not remember, but perhaps there is a database upgrade from 2.2.7 
> > to 2.2.8,
> > you should start gramps from the command line and see if some special upgrade
> > messages pass by. I would however expect database upgrade issues not to play a
> > role when working with xml files....
> > 
> > >
> > > My computer is a 3GHz P4, while hers is a ~1.8GHz Pentium M, but I doubt
> > > that would make a difference that large.
> > 
> > hmm, you said you had the slow load times, so the 3GHz would be slower 
> > than the
> > 1.8, no ? Anyway, there should be no difference between the two.
> > 
> > >
> > > Any idea what's causing this discrepancy?
> > >
> > >
> > >
> > > --
> > > Joel Parker
> > >
> > 
> > Benny
> > 
> > 
> > ----------------------------------------------------------------
> > This message was sent using IMP, the Internet Messaging Program.
> > 
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc.
> > Still grepping through log files to find problems?  Stop.
> > Now Search log events and configuration files using AJAX and a browser.
> > Download your FREE copy of Splunk now >>  http://get.splunk.com/
> > _______________________________________________
> > Gramps-users mailing list
> > Gra...@li...
> > https://lists.sourceforge.net/lists/listinfo/gramps-users
-- 
Joel Parker