Thread: RE: [Phpgedview-talk] Some general questions from a newbie
Brought to you by:
canajun2eh,
yalnifj
From: John F. <Joh...@by...> - 2005-03-18 19:18:22
|
Hi James, > At first I wasn't going to consider PGV, as the first material that I found > lauded it as not requiring mySQL, but I see that mySQL is an option. The next version of PGV 3.3 will support MySQL, SQLite, and Postgres, as well as the database free index mode. Index mode will likely be phased out as PHP 5 become more ubiquitous. Technically Oracle should also be available, but it has not been tested. > In brief, I'd like to provide a service much like the One Great Family > website, but either for free or for almost free. Also, I don't care for > their merging interface, nor their free-for-all merging resolution. It > seems like they allow pairs of submitters to merge against each other and > to modify that pair's data. I'd like to have a master file and have > submitters's data merged against it. Merge disputes would be arbitrated. Your complaints about One Great Family are the same ones that I have, along with the fact that they don't keep much more information than birth and death and that there is no privacy in their system. > OK, that said, and knowing that if I knew how really difficult this would > be to implement, I'd give up the silly idea in a second, I'd still like to > ask those who know the software to fast track me about the obvious > obstacles. I am a programmer, but not too advanced with php and still > pretty clueless about OOP. I've learned just enough to build a dynamic site > with some authentication, session control, etc. I've recently installed > mediaWiki (same as used by WikiPedia) which uses php/mySQL, and have been > able to use it's "extension" markup tags to call php functions which pass > back HTML containing tables of data from my main site's mySQL. But let's > pretend that I have a team of competent php gurus in my back pocket. How > suitable is PGV as a starting place for modifying into what I describe? How > close is it already? I'll install it and play with it during the next month > regardless, but it would help if I knew in advance some of the boundaries. PGV is already pretty close to that. What you don't currently have is the ability for non-root users to upload their own gedcoms or manage their own users. But once a gedcom is uploaded administrative privileges can be extended to other users. =20 =20 > Here are some of my initial fears: > 1) there is a practical limit to the amount of data that can be managed. The practical limits for what you want to do will probably be based on hardware constraints (ie available RAM, available hard disk, CPU power, etc). And then software constraints such as limits in MySQL. I imagine that you would be able to hold 2-3 million people (ie 500 average size gedcoms) before you would have to start looking into clustering your database and hardware. > 2) there are too many fundamental design structures that would have to be > changed to allow the extra features All you should have to do is make a few database changes to the users table. > 3) GEDCOM upload is manual? (If so, a module would have to be developed to > allow grandma to submit.) Upload can be done online (depending again on server constraints) and is a pretty simple process... though I don't think the average grandma could do it without some training and instruction. If you want to support such basic users it would be better to have them upload their gedcom to a temporary location and then you import it for them. > 4) robustness issues with importing GEDCOMs from every possible source and > export scenario. If so, an import filter would need to be developed. I > believe that this is already an issue, with a common solution to run the > file through a dependable application like Brother's Keeper to massage the > data into a more troublefree form? PGV will handle almost all possible gedcoms that are close to the GEDCOM spec. The PGV philosophy has been to leave the original gedcom intact as it comes in and then build all displays from the gedcom records. This has produced a system that is extremely compatible with the GEDCOM spec. The real problems come from people not inputting valid data such as dates and places. Invalid data is still shown, but it can't be used in any calculations such as the calendar. Family Tree Maker is also notorious for producing awkward gedcoms. PGV already has a cleanup script as part of the import process which will fix FTM gedcoms and some other common gedcom problems. You could extend this to do anything that you wanted. > 5) I would probably need to add data in a non-GEDCOM form. I assume that > uploading it directly into the mySQL tables wouldn't be too difficult? It > depends on how much the data is spread around in different tables, and how > relationships are managed. This data would probably be mostly events, > without much connection to other records, like a list of births from a > church book, eg. PGV requires gedcom form since it stores the gedcom in the tables and uses the gedcom record to build the displays. The gedcom data is stored in 4 tables, pgv_individuals, pgv_families, pgv_sources, and pgv_other. So you will need to convert any data to gedcom first (not an impossible task for most genealogy data). You should definitely download the latest beta version 3.3 and play around with it. I'll be happy to answer any other questions you may have. --John =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D John Finlay PhpGedView Project Manager ya...@us... |
From: John F. <Joh...@by...> - 2005-03-18 21:32:37
|
Hi John, > Can you point me to a description of what munging it does to FTM gedcoms? >=20 I don't have a description written up already, but I can give you an idea here. FTM is really bad about putting data in the PLAC field when it shouldn't be there. For example, FTM will export a SSN record like this: 1 SSN 2 PLAC 123-45-6789 This would put all of the SSNs and other personal descriptor tags in the place data. PGV will clean this up during the import so that it looks like this: 1 SSN 123-45-6789 The script will also do some date validation. For example it will convert dates in the form 2005-01-01 to their valid gedcom dates 01 JAN 2005. There are some other general cleanup things that it will do. > Is this module accessible so that I could take the fixed up gedcom and > download it? Or even update it for my personal requirements? >=20 The PGV files that do this are validategedcom.php and function_tools.php. Yes, you could upload a file to PGV to have it cleanup the gedcom, and then download the file again. Yes you could update the scripts to do other cleanup routines. > I am trying to move some fairly large family files from FTM to Legacy, and > I need to do some processing over and above what special FTM to Legacy > conversions that Legacy will do on an gedcom import. I have also written other gedcom process scripts for PGV that are not included in the project to do things like convert source citations or name records. I can send you some of these if you want to use them as starting points. --John =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D John Finlay PhpGedView Project Manager ya...@us... |
From: John L. <ge...@tq...> - 2005-03-18 22:42:53
|
John, Excellent news that gedcoms can be processed and returned. Very useful. 1/ Do you handle prefix/suffix? FTM has this horribly confused. 2/ Do you handle relationship of child to each parent. I think that Legacy handles this now. 3/ FTM is able to handle some "BETWEEN" date formats that Legacy cannot (eg, when the detail on one side is different than on the other, eg, "between 1 jan 1800 and 1809". 4/ Events and custom events are a challenge. Can you handle event comments and event notes? FTM as you note puts all of that in a PLAC tag ... I would want to be able to decide where to place it. 5/ I have some custom events - I may wish to parse their PLAC tag between place, comment and note. Yes, I would like to see the other conversion routines that you might have. Thanks, john. At 04:32 PM 3/18/2005, John Finlay wrote: >Hi John, > > > Can you point me to a description of what munging it does to FTM >gedcoms? > > > >I don't have a description written up already, but I can give you an >idea here. FTM is really bad about putting data in the PLAC field when >it shouldn't be there. For example, FTM will export a SSN record like >this: >1 SSN >2 PLAC 123-45-6789 > >This would put all of the SSNs and other personal descriptor tags in the >place data. PGV will clean this up during the import so that it looks >like this: >1 SSN 123-45-6789 > >The script will also do some date validation. For example it will >convert dates in the form 2005-01-01 to their valid gedcom dates 01 JAN >2005. > >There are some other general cleanup things that it will do. > > > Is this module accessible so that I could take the fixed up gedcom and > > download it? Or even update it for my personal requirements? > > > >The PGV files that do this are validategedcom.php and >function_tools.php. Yes, you could upload a file to PGV to have it >cleanup the gedcom, and then download the file again. Yes you could >update the scripts to do other cleanup routines. > > > I am trying to move some fairly large family files from FTM to Legacy, >and > > I need to do some processing over and above what special FTM to Legacy > > conversions that Legacy will do on an gedcom import. > >I have also written other gedcom process scripts for PGV that are not >included in the project to do things like convert source citations or >name records. I can send you some of these if you want to use them as >starting points. > >--John > >============================= >John Finlay >PhpGedView Project Manager >ya...@us... |
From: John L. <ge...@tq...> - 2005-03-22 14:32:56
|
Hi, I am just installed 3.2.2 to begin trying out phpGedView and am having a few problems that maybe someone can help with. I am no expert with Linux ... 1/ I downloaded the software as a .zip file. I extracted the file on my PC. I then uploaded the files with ws_ftp to a linux server at my hosting company. I note that the .htaccess file in the /index folder does not show up when the ftp is complete in the directory on the server. Is this important? Is this an issue with ws_ftp or is my hosting company not allowing these files to be uploaded? 2/ Stumbling along, I uploaded my gedcom (about 1.7 MB) that I created from Legacy 5 using all defaults with ws_ftp. I set permissions as required and they seem to be correct. I configured to use index files to start. I added the gedcom assigned it a name. But when it goes to Step 2 of 4: Configure PHPGedView + Gedcom File, it displays the message after the field "Default person for pedigree and descendency charts" : Unable to find individual with id (in red) and will not let me make any changes beyond that. I do not seem to be able to get beyond that. What do I need to do? john. |
From: John L. <ge...@tq...> - 2005-03-22 18:30:37
|
All Solved... for now... 1/ Hidden files are just not displayed unless you ask to see them... Not being a Unix/Linux person, I did not know to ask. 2/ I moved the index out of my www space and was able to get it to work. I am not sure exactly what was going on. It would be helpful if the instructions on the media files would explain in better detail how a folder is to be structured if you want to add a level. I ran into a problem that I did not know how to easily resolve without just blowing away the config.php file. I made an error in entering the index folder information and I could not figure out how to get back to configuration to fix it. thanks, john. At 09:25 AM 3/22/2005, John Lisle wrote: >Hi, > >I am just installed 3.2.2 to begin trying out phpGedView and am having a >few problems that maybe someone can help with. I am no expert with Linux ... > >1/ I downloaded the software as a .zip file. I extracted the file on my >PC. I then uploaded the files with ws_ftp to a linux server at my hosting >company. > >I note that the .htaccess file in the /index folder does not show up when >the ftp is complete in the directory on the server. > >Is this important? Is this an issue with ws_ftp or is my hosting company >not allowing these files to be uploaded? > >2/ Stumbling along, I uploaded my gedcom (about 1.7 MB) that I created >from Legacy 5 using all defaults with ws_ftp. I set permissions as >required and they seem to be correct. I configured to use index files to start. > >I added the gedcom assigned it a name. But when it goes to Step 2 of 4: >Configure PHPGedView + Gedcom File, it displays the message after the >field "Default person for pedigree and descendency charts" : Unable to >find individual with id (in red) and will not let me make any changes >beyond that. > >I do not seem to be able to get beyond that. What do I need to do? > >john. > > > > >------------------------------------------------------- >SF email is sponsored by - The IT Product Guide >Read honest & candid reviews on hundreds of IT Products from real users. >Discover which products truly live up to the hype. Start reading now. >http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click >_______________________________________________ >Phpgedview-talk mailing list >Php...@li... >https://lists.sourceforge.net/lists/listinfo/phpgedview-talk |
From: Jon A. <jon...@he...> - 2005-03-22 19:30:02
|
John, The .htaccess file is hidden. The dot causes that in UNIX. I'm not familiar with ws_ftp but there is probably an option to show hidden files somewhere. Or if it lets you issue a command, try "ls -a" in that directory---you will see .htaccess near or at the top of the list. Jon John Lisle wrote: > Hi, > > I note that the .htaccess file in the /index folder does not show up > when the ftp is complete in the directory on the server. > |
From: John L. <ge...@tq...> - 2005-03-28 19:42:08
|
John, A week or so ago you offered me some additional gedcom cleanup routines. I am interested in having them. Please send them to me off-list. I wanted to take this opportunity to describe the adventure I had trying to install and upload a large gedcom. Maybe my experience will help someone else and maybe it will help suggest ways to make the process easier. Maybe someone will tell me something I did wrong that will make the next time much easier! __________ I have finally succeeded in installing 3.3 and using it. I am quite impressed with what it can do. But I have to tell you that it was a major struggle to install one of my large (c. 20 MB) gedcoms. I was continually fighting limits. IMHO, it is imperative to find a way to upload a large gedcom to a MySQL database without having to change standard php limits. Here is what I found I needed to do when phpGedView timed out trying to import my gedcom into MySQL. 1/ I installed EasyPHP 1.8 from easyphp.org on my Windows XP Home PC. This is a combo package that includes apache, php, mysql, and phpmyadmin. I am not going to say that making it work was easy. Their web site is mostly in French... I found that I needed to re-boot and stop/start servers to get it to work. I edited the php.ini to increase limits. 2/ I copied 3.3 into my localhost folder and moved the gedcom. Sidebar: I would really like to see the config file have a way to point to the where the code lives so the code and folders that do not need to change or have permissions change, etc. can sit anywhere. One of the things I saw from this exercise is that I wanted to put different gedcoms in different MySQL databases. In part, this was going to be because I would want to dump them individually. In part, to be able to retain some separation. And as the software SEEMS to be configurable, each MySQL database now requires its own copy of all of the code. 3/ I configured the database, configured the gedcom, validated the gedcom, and imported the gedcom into my local MySQL database. This took FOREVER. First, I needed to increase the timeout limit on the gedcom. I started by increasing from 60 to 600 seconds. This barely got me through the first pass of the import where it does names. Then it went to the second pass and ran out of time doing places. I increased it again to 1800 seconds. And it barely made it on that pass but then crashed in the last pass where it load the married names. I finally needed to increase it to 4800 seconds to make it through the last pass. In each case, it had to re-run the passes that it had successfully done before. A valuable enhancement would be to make it possible to re-run starting where you left off. Another would be to ignore time limit while importing a gedcom. I have difficulty understanding 2 performance issues - why does this process take so long? I have a quick enough little PC. and why did each go around on the same pass take less time? The first time I did pass one it took about 630 seconds. The 4th time it was just over 500 seconds. Same data. Same PC. Nothing else running on the PC at the time. Bizarre... 4/ OK. Now I have my GEDCOM in a phpGedView MySQL database. I then went into phpMyAdmin and dumped the database into a gzipped SQL file to re-import through phpMyAdmin into my MySQL database on my host. Export was easy. Import was a bear! Of course, my server at hosting company timed out trying to upload and unzip and process a 56 MB SQL file. (about 7+ MB compressed). I then exported the tables individually (I did combine the small tables into one export...) and then pasting the SQL into the SQL window a bit at a time, I was able to process all of the SQL. But even that was not a small task. (My placelinks table was about 225K records that I pasted in 25K records at a time.) In the end, it is working... I think ... but I will need to go through the whole process again once I get a routine that will fix the FTM prefix titles. (FTM title becomes an event. It should be a 2 NPFX tag under the person's name rather than a 1 TITL event tag.) (Of course, if someone knows a clever SQL query that I can run on the database to change this, please advise...) Other PROBLEMS... so far... A/ I note that the config.php file (as distributed) claims this is version 3.2. B/ I get an internal server error when trying to do a Given Name/Last name search on my hosted database. On my local database it works. Any ideas how to diagnose? (Could this be problem with how I imported the Gedcom or with some other install limit?) C/ I would like to be able to add standard facts. I have a series of custom facts that I use constantly. I would like to be able to access them as I would for the regular facts. In part for ease, but also for consistency so that they are spelled the same whenever someone does an edit. In looking at individual.php, that list looks like it is hardcoded. ---- I hope this is useful to someone. I see this tool as an important way to advance the collaborative work I am leading. If anyone wants to see what I have so far... http://www.gojileader.com/stedman Thanks for listening, John Lisle Nashua, NH www.stedmanfamily.org |
From: Rodney H. <rm...@rm...> - 2005-03-30 13:16:57
|
Having installed v3.3 in a temporary directory to test - http://rmhh.co.uk/tree3/index.php?command=gedcom -, I'm having a couple of problems:- (URLs shortened) 1. The top of the Welcome Page shows "Notice: Undefined index: admin in .../tree3/blocks/html_block.php on line 32 Warning: Cannot modify header information - headers already sent by (output started at .../tree3/blocks/html_block.php:32) in .../tree3/includes/functions_print.php on line 562" 2. The MyGedView Portal Page shows:- "Notice: Undefined index: admin in .../tree3/blocks/html_block.php on line 32 Warning: Cannot modify header information - headers already sent by (output started at .../tree3/blocks/html_block.php:32) in .../tree3/index.php on line 75" 3. Clicking on any of the help buttons results in the help popup with multiple lines of "Notice:-Undefined variable: UpArrow in .../tree3/help_text_vars.php on line 47" before the help text. If I may at this point also mention that it would be helpful in that help popup to add that when the user enters the new search ID, they should hit 'Enter' to go to that person. 4. The language selection icons seem to be missing, as in v3.2.2 as at http://rmhh.co.uk/tree2/index.php?command=gedcom Can anyone please tell me what I'm doing wrong? -- Rodney HALL Heywood, Lancashire Suaviter sed fortiter Agreeably but powerfully ~~~~~~~~~~~~~~ rm...@rm... http://rmhh.co.uk/ http://rmhh.org.uk/ ~~~~~~~~~~~~~~~~~~~ |
From: =James B. <j.b...@ve...> - 2005-03-31 23:57:26
|
I'm sending this in case you don't get a more knowledgable reply. I've seen similar messages when trying to do Sessions in php, I think the trick was to move the commands to an earlier postion. Not sure why you'd get that scenaria with working code though. Perhaps a path isn't config'd right and something's not "Included" (called) properly? The first error is probably the key... At 02:16 PM 3/30/05, you wrote: >Having installed v3.3 in a temporary directory to test - >http://rmhh.co.uk/tree3/index.php?command=gedcom -, I'm having a couple of >problems:- >(URLs shortened) > >1. The top of the Welcome Page shows "Notice: Undefined index: admin in >.../tree3/blocks/html_block.php on line 32 > >Warning: Cannot modify header information - headers already sent by (output >started at .../tree3/blocks/html_block.php:32) in >.../tree3/includes/functions_print.php on line 562" James Birkholz admin, Posen-L mailing list and website http://www.Posen-L.com |
From: Rodney H. <rm...@rm...> - 2005-04-01 10:40:22
|
James, Many thanks for your reply. I have found that the problem is the custom Theme that I have previously used not being compatible with the new build. I shall have to go back through the files all over again to get the look that I want! :o( The only strange thing I see on the default style is that language change is now a drop-down box and only English is offered! Regards -- Rodney HALL Heywood, Lancashire Suaviter sed fortiter Agreeably but powerfully ~~~~~~~~~~~~~~ rm...@rm... http://rmhh.co.uk/ http://rmhh.org.uk/ ~~~~~~~~~~~~~~~~~~~ > -----Original Message----- > From: php...@li... > [mailto:php...@li...] On > Behalf Of =James Birkholz= > Sent: Friday, April 01, 2005 12:57 AM > To: php...@li... > Subject: Re: [Phpgedview-talk] 3.3 install Problems > > I'm sending this in case you don't get a more knowledgable reply. > I've seen similar messages when trying to do Sessions in php, > I think the > trick was to move the commands to an earlier postion. > Not sure why you'd get that scenaria with working code > though. Perhaps a > path isn't config'd right and something's not "Included" > (called) properly? > The first error is probably the key... > > At 02:16 PM 3/30/05, you wrote: > >Having installed v3.3 in a temporary directory to test - > >http://rmhh.co.uk/tree3/index.php?command=gedcom -, I'm > having a couple of > >problems:- > >(URLs shortened) > > > >1. The top of the Welcome Page shows "Notice: Undefined > index: admin in > >.../tree3/blocks/html_block.php on line 32 > > > >Warning: Cannot modify header information - headers already > sent by (output > >started at .../tree3/blocks/html_block.php:32) in > >.../tree3/includes/functions_print.php on line 562" > > James Birkholz > admin, Posen-L mailing list and website > http://www.Posen-L.com |
From: John L. <ge...@tq...> - 2005-03-18 21:02:50
|
John - You caught my attention when you said... At 02:18 PM 3/18/2005, John Finlay wrote: >The real problems come from people not inputting valid data such as >dates and places. Invalid data is still shown, but it can't be used in >any calculations such as the calendar. Family Tree Maker is also >notorious for producing awkward gedcoms. PGV already has a cleanup >script as part of the import process which will fix FTM gedcoms and some >other common gedcom problems. You could extend this to do anything that >you wanted. >--John > >============================== >John Finlay >PhpGedView Project Manager >ya...@us... Can you point me to a description of what munging it does to FTM gedcoms? Is this module accessible so that I could take the fixed up gedcom and download it? Or even update it for my personal requirements? I am trying to move some fairly large family files from FTM to Legacy, and I need to do some processing over and above what special FTM to Legacy conversions that Legacy will do on an gedcom import. thanks, john. |