Import Speed and Descendency Report Failures

Help
2012-10-26
2013-05-30
  • Thomas Ronayne
    Thomas Ronayne
    2012-10-26

    I had to install PGV from phpgedview-phpGedView.tar.gz (because the stable release wound not work) in Linux Kernel 3.2.29 #2 SMP Mon Sep 17 14:19:22 CDT 2012 x86_64 Intel(R) Core(TM)2 Duo CPU     E7600  @ 3.06GHz GenuineIntel GNU/Linux, Apache (httpd-2.4.3-x86_64-1), and PHP (_php-5.4.7-x86_64-1. The main problem with the Stable release was that I could not import any GEDCOM files (blank screen), so I updated. This new version takes forever and week to import a GEDCOM file - I'm talking about a half-hour to import a 2,000 individual GEDCOM. I've allocated 1G (of 8G) to PHP and in PGV. Is there some HTTPD/PHP/PGV configuration setting I've missed? I tail an import (the index/filename.ged.new file while it's importing) and, you know, this is a 64-bit Linux box and I can actually read the individuals being loaded; i.e., sllloooowwww.

    Attempting a Descendency Report with any of the four GEDCOMS installed results in this:
    _

    ERROR 2: Creating default object from empty value
    0 Error occurred on line 3729 of file class_reportbase.php in function PGVRRelativesSHandler
    1 called from in function call_user_func
    2 called from line 1438 of file class_reportbase.php in function startElement
    3 called from in function xml_parse
    4 called from line 387 of file reportengine.php
    Warning: Creating default object from empty value in /var/www/htdocs/phpgedview/includes/classes/class_reportbase.php on line 3729
    ERROR 2: Creating default object from empty value
    0 Error occurred on line 3729 of file class_reportbase.php in function PGVRRelativesSHandler
    1 called from in function call_user_func
    2 called from line 1438 of file class_reportbase.php in function startElement
    3 called from in function xml_parse
    4 called from line 387 of file reportengine.php
    Warning: Creating default object from empty value in /var/www/htdocs/phpgedview/includes/classes/class_reportbase.php on line 3729
    

    _However, other reports work.

    Have I missed something in Apache, PHP, PGV?

    Thanks._

     
  • Gerry Kroll
    Gerry Kroll
    2012-10-26

    That sounds more like a database problem.

     
  • Thomas Ronayne
    Thomas Ronayne
    2012-11-13

    Well, OK, what sort of data base problem?

    This is the SQL code I have used to create and grant permissions to the data base (I seem to remember there was a guide to doing this on the PhpGedView web site but I can't find it):

    create  database if not exists phpgedview;
    grant   select, insert,
            update, delete, index, alter, create, lock tables,
            create temporary tables, drop, references on phpgedview.*
    to      ged@localhost identified by 'gedpassword';
    grant   select, insert,
            update, delete, index, alter, create, lock tables,
            create temporary tables, drop, references on phpgedview.*
    to      ged@'%' identified by 'gedpassword';
    flush   privileges;
    

    I also grant all privileges to "me."

    Something wrong or missing  with those?

     
  • Stephen Arnold
    Stephen Arnold
    2012-11-13

    Why not zip the GEDCOM and email it to me. I'll load it in my offline PGV version and see if I can view any problems with your data.

    What was the formatting for import? Was it all in UTF-8 so that there was no conversion necessary for import?
    A 2,000 INDI gedcom should take no more than about 10 minutes (5 minutes or less on my webtrees installation).

    Then we will know if you have a gedcom issue or an issue with software or hardware.
    -Stephen   mailto:   GEDCOM at MyArnolds dot com

     
  • Thomas Ronayne
    Thomas Ronayne
    2012-11-16

    I figured out the problem with slow loading - I\'m using MySQL 5.5.27 which replaces MySQL 5.1.56 is a previous distribution release. MySQL 5.5 defaults to InnoDB tables rather than the previous default to MyISAM. InnoDB, for all practical purposes, is a transaction that gets committed immediately and when loading a large data set takes a while; thus the extremely slow loading. Adding

    # Set the default storage engine to MyISAM
    default-storage-engine=myisam
    

    in the mysqld section of /etc/mysql/my.cnf eliminates that problem. Alternatively, editing every create table directive in PGV to specify MyISAM would be better but a whole lot of work as would modifying PGV to not commit until a load is complete (also a lot of work).

    I should note that the previous distribution release included php-5.3.17 where the current release is php-5.4.7. 5.4.x requires actually setting the default time zone (in /etc/httpd/php.ini) for dates to display properly; i.e., you cannot use the default system time without getting a ton of error messages.

     
  • Gerry Kroll
    Gerry Kroll
    2012-11-17

    It seems to me that there should be a way of specifying the myisam engine when the database is created, instead of doing so at the table level.  It doesn't make sense to mix engine types in the same database.

     
  • Thomas Ronayne
    Thomas Ronayne
    2012-11-17

    All I can do is refer you to http://dev.mysql.com/doc/refman/5.5/en/create-database.html, Section 13.1.10. CREATE DATABASE Syntax, of the MySQL 5.5 Reference Manual.

    In part,

    A database in MySQL is implemented as a directory containing files that correspond to tables in the database. Because there are no tables in a database when it is initially created, the CREATE DATABASE statement creates only a directory under the MySQL data directory and the db.opt file. Rules for permissible database names are given in Section 9.2, “Schema Object Names”. If a database name contains special characters, the name for the database directory contains encoded versions of those characters as described in Section 9.2.3, “Mapping of Identifiers to File Names”.

    It doesn't look to me like you can specify the type of data base in the CREATE DATABASE syntax.

    On the other hand, the CREATE TABLE syntax (see http://dev.mysql.com/doc/refman/5.5/en/create-table.html) does explain that InnoDB is the default type which can be overridden in the CREATE TABLE directive or by setting the system default.

    In any event, InnoDB tables do work in PGV but they're slower than molasses on a winter's day loading without turning off commits for every row loaded; again, this is something that would need embedding in the PGV table creation or loading area (and I just don't want to edit an application to add such things).

    I still have a problem with descendency reports failing with error messages but at least the data loads in a few seconds instead of many, many minutes.