Re: [Gramps-devel] Gramps 3.4.2: Very slow Gramps-XML import...

[Benny M. <ben...@gm...>]

2013/2/14 Espen Berg <es...@gm...>

> Hi,
> I just fetched the latest from subversion and compiled from there and
> am now importing from GrampsXML. It went fast up to about 10% but then
> it really slows down the speed. Now it spent 5 minutes from 15% to 16%
> and the speed seems to be decreasing. The CPU is running at 98% and
> memory usage for Gramps is slowly increasing.
>
> Could one reason for the decreased performance be the introduction of
> the separate citation stuff? At least it created over 100K entries in
> my family tree last time I did an import. If I referred to  a source
> e.g. 1000 times, this created 1000 citation entries when the database
> was upgraded. Now I of course recycle the citations, but the existing
> ones still remain. I haven't received motivation to start cleaning up
> yet by merging two and two :D
>

There is an automatic merging tool. But backup before using that.

Benny

>
>
> Cheers
> Espen
>
> 2013/1/27 Gary Burton <bur...@ya...>:
> > Hello Espen
> >
> > I recently fixed a memory leak in the XML import code. You could try
> using
> > the new version from the code repository to see if this improves the
> speed.
> > :
> > You can get a copy of the newversion here:
> >
> >
> http://sourceforge.net/p/gramps/code/21226/tree//branches/maintenance/gramps34/src/plugins/import/ImportXml.py?format=raw
> >
> > Bye
> >
> > Gary
> >
> >
> > ________________________________
> > From: Espen Berg <es...@gm...>
> > To: GRAMPS Development <gra...@li...>
> > Sent: Sunday, 27 January 2013, 21:01
> > Subject: [Gramps-devel] Gramps 3.4.2: Very slow Gramps-XML import...
> >
> > Hi devs,
> > It has been quite a long time since last time I did an import from
> > Gramps-XML but I did a few tries the last week both on my Ubuntu x64 and
> my
> > Windows XP (x64). In both cases the import took about 2 hours to
> complete.
> > My database has about 37K individuals and about 120K events and
> citations.
> > The Gramps-XML file is about 7Mb (but that itself does not say much about
> > the complexity of the data). However, I feel that the import should not
> take
> > that long time. The Gramps-XML file was also exported from Gramps 3.4.2
> so
> > the citations are present in the file and should not need to be created
> on
> > the fly from a Gramps-XML file from an older version where the citations
> > were not present.
> >
> > Do you have any ideas of why the import takes so long time? Since it
> took so
> > long for both systems on different computer architectures, the reason
> seems
> > to be in the import algorithm. Any others experiencing the same
> behaviour?
> >
> >
> > Cheers
> > Espen
> >
> >
> ------------------------------------------------------------------------------
> > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> > MVPs and experts. ON SALE this month only -- learn more at:
> > http://p.sf.net/sfu/learnnow-d2d
> > _______________________________________________
> > Gramps-devel mailing list
> > Gra...@li...
> > https://lists.sourceforge.net/lists/listinfo/gramps-devel
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
> > MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
> > with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
> > MVPs and experts. ON SALE this month only -- learn more at:
> > http://p.sf.net/sfu/learnnow-d2d
> > _______________________________________________
> > Gramps-devel mailing list
> > Gra...@li...
> > https://lists.sourceforge.net/lists/listinfo/gramps-devel
> >
>
>
> ------------------------------------------------------------------------------
> Free Next-Gen Firewall Hardware Offer
> Buy your Sophos next-gen firewall before the end March 2013
> and get the hardware for free! Learn more.
> http://p.sf.net/sfu/sophos-d2d-feb
> _______________________________________________
> Gramps-devel mailing list
> Gra...@li...
> https://lists.sourceforge.net/lists/listinfo/gramps-devel
>