GEDCOM import stalls

  • Jules

    Jules - 2011-11-28

    PhpGedView V. 4.2.4
    PHP 5.3.8
    MySQL UTF8 only
    Win XP SP3

    I upgraded my smoothly working PhpGedView to these new PHP and MySQL versions.

    The nasty consequence is that import of large GEDCOM files (7800 and 14000KB)  stalls at about 50% and 20%.

    In PHP-CGI-FCGI.INI I increased "memory_limit" to a whooping = 960M.

    Are there any other parameters to tweak to get a positive result?

    Thanks EJF

  • ggpauly

    ggpauly - 2011-11-28

    Hi EJF,

    Try setting your execution time limit to 60 seconds.

  • Gerry Kroll

    Gerry Kroll - 2011-11-28

    Try using a different browser.  Some versions of Internet Explorer don't handle the continuous updating of the Import Progress bar properly.  It's an incompatibility with Microsoft's implementation of JavaScript.

    Generally, the other browsers work much better.  I've had no problems with FireFox, Opera, Safari, or Chrome on my Windows XP system.

    When a PhpGedView display seems to stall, you can sometimes get useful information when you inspect the page source of that stalled page.  When you inspect that source, look for some sort of error message at the bottom the page source.

    The advice to set your execution time limit lower is good.  You need to understand that there are actually two time limits that come into play.  The lower of the two limits is the one that's effective.  One limit is set by the system administrator in the php.ini configuration file.  The other is set by you when you configure or run PhpGedView.

    The time limit in php.ini is an absolute one:  When it runs out, PhpGedView is killed with NO warning.  The limit set in PhpGedView is more flexible.  When you're importing, PGV watches the amount of time used, and when the programmed limit is close to running out, PGV will interrupt the Import in such a way that pressing the Continue button either manually or automatically will restart the Import at the point of interruption.

    The point of the previous paragraph is that you should set the Import time limit to be less than the absolute limit set in the php.ini configuration, otherwise PGV will not be able to suspend the Import before the time limit runs out.

  • Jules

    Jules - 2011-12-02

    Thank you both!

    Unfortunately the advices did not help. It is certainly not dending of the browser. I normally use Chrome and have the others at hand. But the result is the same.

    Thanks again

  • Gerry Kroll

    Gerry Kroll - 2011-12-02

    Did you check the page source at the time of the stalled Import?

    There has to be an error message of some sort, either in the system log or in the page source.

    Your GEDCOM sizes are not outrageously large.  There are quite a few successful users of PGV with GEDCOMs much larger than that.

    You could also inspect the copy of the GEDCOM that's created during the Import.  This copy would reflect a smaller size than the incoming GEDCOM, and the last person or family in it would be the last one that was imported successfully.  You would have to look at the next incoming person or family for a problem of some sort - perhaps a loop such as a person recorded as being a child of himself.

    If necessary, you could e-mail me ZIP copies of your GEDCOMs, and I can try the Import on my off-line installation of PGV.  If it works on my system, that would prove there's something wrong with yours.  If it doesn't, I'll be able to figure out what's wrong with your file.  gkroll at keldine dot ca

  • Stephen Arnold

    Stephen Arnold - 2011-12-02

    Did you try setting the import time to as little as 10 to 20 seconds? If you are borderline sufficient memory, you may have to have it go this slowly, in these tiny little bites of processor time.  Did you confirm the memory was reflected on both MASTER and LOCAL values?

  • Jules

    Jules - 2011-12-06

    Hello helpers,
    In the case of the first Gedcom file setting the import time to as little as 20 seconds was the successful solution. Thanks!

    The second file still stalls with the following error:

    SQLSTATE: String data, right truncated: 1406 Data too long for column 'n_sort' at row 1

    Frei-Meier.ged: 10

    0 @S3161@ SOUR
    1 _CAT 1
    1 TITL Als Stammvater aller Basler Merian gilt ein Theobald Merian (um 1415 – um 1505), bischöflich-baslerischer Meier oder Amtmann in Lüttelsdorf/Courroux bei Delsberg/Delémont. Von diesem Theobald sollen die beiden Basler Familienzweige abstammen. Die ältere Linie geht auf den Sohn Theobald Merian (um 1465 – 1544) zurück, der um 1480 als Schiffmann von Lüttelsdorf nach Basel zog und später eine Sägerei im rechtsrheinischen Kleinbasel betrieb. Die jüngere Linie begründete Johann Peter Merian (um 1468/1478 – nach 1519), der als bischöflich-baslerischer Meier in Lüttelsdorf ebenfalls nach Basel kam. Ob es sich bei Theobald (Sohn) und bei Johann Peter tatsächlich um Brüder und damit um Söhne des Theobald (Vater) gehandelt hat, wie die Überlieferung behauptet, ist urkundlich nicht belegt.
    1 CHAN
    2 DATE 26 JUL 2011

    This MySQL error didn't show up with the previous version. There is something that must have changed in the field definitions, I think.

    Thanks, EJF.

  • Stephen Arnold

    Stephen Arnold - 2011-12-06

    You are mistaken about any change. The parameters on the media table for TITL have always been:
    m_titl varchar(255)

    You will have to use the NOTE section of the tag to provide a more lengthy field, restricting the title to 255 characters.

  • Gerry Kroll

    Gerry Kroll - 2011-12-06

    As Stephen notes, you are mis-using the "Title" field of the Source entry. 

    The title is supposed to be a very short description.  For example:  "Stammvater aller Basler Merian".  The full text should be a Note or other text field reserved for the purpose.

    I question your use of the Source entry in this particular case.  You're not actually referring to any official documents that can be verified by other genalogists.

  • Jules

    Jules - 2011-12-10

    As a matter of fact I use another genealogy program to record my findings. I import a GEDCOM file produced by this other program into phpGedView. The TITL errror is due to this other routine. I must check there. The text in question is stored there as NOTE and the document source in SOUR.

    But: An old backup GEDCOM file that worked perfectly well with my old configuration cannot be imported and produces import errors. I plan to go back to my old configuration that worked without hiccups.



Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.

No, thanks