I am attempting to obtain consistency with the location hierarchy for each individual. I've been editing on each individual's page.
To do this more quickly, I would like to edit either the Gedcom file or the mySQL database directly. But I don't understand how modifying one will update the other. And, of course, I want both to be updated with the same information.
Can someone explain that process to me or direct me to where such information is documented?
Thank you,
Frank
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Frank
You can export the gedcom and use a text editor (search and replace), you could Export and use a GEDCOM place utility like Nigel/Kiwi has previously suggested in other threads, you could use the batch update utility under the Admin menu and select search and replace, or you could do it via search in phpMyAdmin (db only) HOWEVER, that would NOT be a good choice. I assume you do not use the SYNC to GEDCOM feature (you probably shouldn't). You can also use the LISTS > PLACES and drill down, cleaning up as you go from Country to Locale (link states first, then counties within each state).
Assuming the GEDCOM is manageable (perhaps less than 25k INDIs), a fast machine and plenty of memory, I'd choose the batch utility. If memory is an issue, use a text editor. -Stephen
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Stephen has told you how. In answer to your more fundamental question, you edit the GEDCOM. When you accept the changes, the database is updated. That is the flow of processing, not the reverse.
Doug.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just to add a little as to how I understand the process to work for you. Export your Gedcom. This Gedcom is created from your database. Edit the Gedcom with a suitable text editor. Then import your edited Gedcom which will update your database. That is, exporting your Gedcom creates a new Gedcom from the database. Importing the Gedcom creates a new database. If you follow this process then at the time of importing your Gedcom do not select to retain any image links.
Stuart
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have around 12,700 INDIs now. Most (99.5%) were obtained from importing a GEDCOM. However the places have no consistency. Since there are several countries involved, I am using USA as the Level 0 point for USA states. Therefore I am adding USA to most entries, and am deleting the counties that appear randomly. I hope I am on the right track.
I hadn't imported a second GEDCOM since importing the first. You have explained to me that I can export the GEDCOM, which is created from the mySQL database, I then search and replace, I then import it back to PGV, which will replace all the data therein. I assume by not retaining image links, they will be replaced with the links in the new imported GEDCOM.
Thank You!
Frank
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Frank
Yes and NO to all of this.
Please understand that the GEDCOM text document, once imported in PGV, has little relevance as any changes you make using the GUI are written to the database. PGV has an option to SYNC, but as I said, you should disable this feature with a 12k INDI gedcom since it can really slow the processing of changes and it can be corrupted easily with constant read/write I/O to your hard drive.
I suggested two scenarios for export and revisions outside PGV and two using the tools inherent within the program. I again state that, if you know of a lot of changes that are identical, you can use the Batch admin function. However, if you wish to add USA to all the states, you might be better off using GREP and a text editor.
Check on your imported PLAC configuration before you delete any counties. Counties are usually quite important and the standard PLAC configuration is Local (city/twp), County, State, Country - a four level PLAC structure.
Frankly Frank, I think you need to study the program a bit more, familiarize yourself with all the existing functions and then strategize your goals. THEN, start changing your PLAC structure. Make changes to individual records along the way so that they conform to your data entry conventions, and later change those that don't yet conform. Creating an FAQ and deciding extraly which conventions you wish to use (and almost everyone's is slightly different - abbreviate States or don't, abbreviate countries or don't, abbreviate the word county or use the entire word or don't use it at all, etc. And this is just the PLAC conventions. Every data fact should have an established convention. Do you abbreviate NSFX like Dr, Prof, Col, Maj, Rev, or not? Do you use periods or not? etc… See what I mean? Do it now and you won't be sorry - VERY sorry, later. Also, don't get in the habit of using others work (importing GEDCOMs), especially if they don't meet your SOUR criteria. Absolutely necessary or you end up with garbage in - garbage out and again, you'll be VERY sorry later. -Stephen
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank You Stephen. For the most part, I know what you mean. I've made 100's of edits… (many hundred). And along the way, I've been struggling to determine which standards to stick with. I've also read many posts here in the forum with people asking the same questions I've been having.
I am not syncing the GEDCOM to the mySQL database. I just thought the program would perform better without doing that.
I have set standards such as: Don't use periods in the locations, don't use counties in the USA (that standard doesn't work perfectly for some of the older Places), Use two letter abbreviations for states, Don't use the word county nor country. Use the full word for countries, including USA for USA. However, more currently, I am considering using the state abbrev. for level 0 in the USA, and dropping USA. That is mainly because I have probably over 10,000 names with the states as two letter abbreviations at level 0 and each name may have three locations. And to do a search on 50 state abbreviations just to add ", USA" to end of each one could be herindous. But I can search and delete, ", USA" quickly. I will tryout the editors you mentioned.
I truly appreciate the help you folks provide the newcomers like myself. I really like the program and it is very stable. I've been able to get the memory limit of my server moved up to 64 mB. But that isn't enough for some of the features.
Frankly Frank
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
PS - I meant to add: I'm not sure it is wise to leave the USA county off. That could be something I will truly regret. I read about leaving a space for it between commas if it is unknown. I am struggling with the idea of using "unknown" at the USA county level and including that "location" below the state level for each USA state. And use the coordinates of the state for "unknown".
This is one of those situations Stephen mentions that I could be very sorry about later if I make the wrong decision now.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Some additional questions etc. Statement: I got into trouble when I downloaded my GEDCOM, made changes to be uniform with locations, and uploaded it. My problem was that the text editor left out a lot of carriage returns or whatever it is that separated one individual gedcom from the next. I didn't know how to restore correctly. I think I was also trying to upgrade.
QUESTION: I and all the folks who add to my data are synced to the GEDCOM. Are you people saying that I should change this and then be verifying each change? I have 10K individual names in one of my gedcoms. QUESTION: How can I import just one, or two, individuals that got "lost" as noted in my statement above? Can I import a person with a specific ID #?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Frank
Thanks for the thanks. BUT …Don't remove the USA and consider adding the full states. Really easy with the batch utility or a search and replace in the text editor outside PGV. just 50 actions:
first field; ", AL"
second field: ", Alabama, USA"
etc. It would take less than about 1/2 to 3/4 of an hour to change them all and you'll be golden.
Also, I'm at a loss to understand why the "no counties'. That would be a most unusual setting as many, many times, it is only the county that is specified in the older SOUR index references (IN, OH, TX births and deaths, etc). Also, you really have to be sure your GEDCOM specified PLAC settings match your actual PLAC entries.
Good luck, -Stephen
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
amdrew
If the records are no longer recorded in GEDCOM text format, but rather lengthy strings, there is little you can do. If it is only the CR between records, you could possibly do a Search & Replace for all "0" level records, something like:
find: "0 "
replace: "\r 0 "
but this would only work if the CR are missing between the major level 0 records. If it removed all the CR, it would be a bigger chore. Go back to your original backup copy and then do your PLAC replace again without wiping out the CR's. -Stephen
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks. I will leave USA and continue inputting them. I will also leave the counties. And add counties to new entries.
And if you've learned that whole state name should be used rather than just the abbreviation, I'll do that as well.
I just needed some guidance in that area. I am using the Google module and leaving the counties out, seemed to help that process. I've been manually entering every location into Google. Perhaps I should be importing the state data that has the city and county information already.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I reverted back to a 5-day old GEDCOM, which was before I mutilated it by removing a bunch of counties. I used the internal Batch editor and converted all states to full names and added USA to them.
Worked Great!
Now, I need to start building the Google module places to conincide with the state names.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I used the List - Place Hierarchy to drill down and clean up the top-level locations, which happens to be countries in my case. It looks great. Then I went to Edit Geographic Place Locations, which is under the map on the Place Hierarchy page. The list shown there matches the list I had just cleaned up until I click on "Click here to show inactive places:" Then a bunch of non-top level locations show up.
Does it make sense that I need to clean those up too? Does that list come from a different database than the first one does?
Thank you for the help, I'm still trying to understand all the features and data structure.
Frank
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Frank
Not exactly hijacking your own thread, and not exactly an entirely new subject, but IMHO, different enough that you should start a new thread for your GM question. Before you do so, please read the extensive description in the WIKI and search these forums for previous postings nearly identical to your issues. -Stephen
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've researched this for hours but have had no luck finding a clear answer. Perhaps I'm not looking in the right place. I haven't found anything in Wiki that gets as technical as what I am asking. And I've read a lot of posts that are indeed technical, but I haven't seen that they address this particular issue.
On the bright side, I have found answers to most of my questions online here or in WIKI.
If the answer to my question is online, I'll look some more before starting a new thread.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Frank
I think your answers are there, but it depends more on how you wish to use GM. If you wish to have your places generally conform to the GM locations, then you can simply delete your placelocation (empty it) table and import places from the data provided as additional locations, or you can use the GEDCOM for your places by using the import from gedcoms. I would empty the table first so you can eliminate all those 'dead' locations.
Others may have different solutions and I'm not an expert on GM module. I'm not sure anyone here has been involved in its development and it is also now deprecated as it is v2 API and GM is now onto v3. Only the webtrees program will be upgrading soon to v3 of the API. -Stephen
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am attempting to obtain consistency with the location hierarchy for each individual. I've been editing on each individual's page.
To do this more quickly, I would like to edit either the Gedcom file or the mySQL database directly. But I don't understand how modifying one will update the other. And, of course, I want both to be updated with the same information.
Can someone explain that process to me or direct me to where such information is documented?
Thank you,
Frank
Frank
You can export the gedcom and use a text editor (search and replace), you could Export and use a GEDCOM place utility like Nigel/Kiwi has previously suggested in other threads, you could use the batch update utility under the Admin menu and select search and replace, or you could do it via search in phpMyAdmin (db only) HOWEVER, that would NOT be a good choice. I assume you do not use the SYNC to GEDCOM feature (you probably shouldn't). You can also use the LISTS > PLACES and drill down, cleaning up as you go from Country to Locale (link states first, then counties within each state).
Assuming the GEDCOM is manageable (perhaps less than 25k INDIs), a fast machine and plenty of memory, I'd choose the batch utility. If memory is an issue, use a text editor.
-Stephen
Stephen has told you how. In answer to your more fundamental question, you edit the GEDCOM. When you accept the changes, the database is updated. That is the flow of processing, not the reverse.
Doug.
Frank,
Just to add a little as to how I understand the process to work for you. Export your Gedcom. This Gedcom is created from your database. Edit the Gedcom with a suitable text editor. Then import your edited Gedcom which will update your database. That is, exporting your Gedcom creates a new Gedcom from the database. Importing the Gedcom creates a new database. If you follow this process then at the time of importing your Gedcom do not select to retain any image links.
Stuart
Thanks so much people.
I have around 12,700 INDIs now. Most (99.5%) were obtained from importing a GEDCOM. However the places have no consistency. Since there are several countries involved, I am using USA as the Level 0 point for USA states. Therefore I am adding USA to most entries, and am deleting the counties that appear randomly. I hope I am on the right track.
I hadn't imported a second GEDCOM since importing the first. You have explained to me that I can export the GEDCOM, which is created from the mySQL database, I then search and replace, I then import it back to PGV, which will replace all the data therein. I assume by not retaining image links, they will be replaced with the links in the new imported GEDCOM.
Thank You!
Frank
Frank
Yes and NO to all of this.
Please understand that the GEDCOM text document, once imported in PGV, has little relevance as any changes you make using the GUI are written to the database. PGV has an option to SYNC, but as I said, you should disable this feature with a 12k INDI gedcom since it can really slow the processing of changes and it can be corrupted easily with constant read/write I/O to your hard drive.
I suggested two scenarios for export and revisions outside PGV and two using the tools inherent within the program. I again state that, if you know of a lot of changes that are identical, you can use the Batch admin function. However, if you wish to add USA to all the states, you might be better off using GREP and a text editor.
Check on your imported PLAC configuration before you delete any counties. Counties are usually quite important and the standard PLAC configuration is Local (city/twp), County, State, Country - a four level PLAC structure.
Frankly Frank, I think you need to study the program a bit more, familiarize yourself with all the existing functions and then strategize your goals. THEN, start changing your PLAC structure. Make changes to individual records along the way so that they conform to your data entry conventions, and later change those that don't yet conform. Creating an FAQ and deciding extraly which conventions you wish to use (and almost everyone's is slightly different - abbreviate States or don't, abbreviate countries or don't, abbreviate the word county or use the entire word or don't use it at all, etc. And this is just the PLAC conventions. Every data fact should have an established convention. Do you abbreviate NSFX like Dr, Prof, Col, Maj, Rev, or not? Do you use periods or not? etc… See what I mean? Do it now and you won't be sorry - VERY sorry, later. Also, don't get in the habit of using others work (importing GEDCOMs), especially if they don't meet your SOUR criteria. Absolutely necessary or you end up with garbage in - garbage out and again, you'll be VERY sorry later.
-Stephen
Thank You Stephen. For the most part, I know what you mean. I've made 100's of edits… (many hundred). And along the way, I've been struggling to determine which standards to stick with. I've also read many posts here in the forum with people asking the same questions I've been having.
I am not syncing the GEDCOM to the mySQL database. I just thought the program would perform better without doing that.
I have set standards such as: Don't use periods in the locations, don't use counties in the USA (that standard doesn't work perfectly for some of the older Places), Use two letter abbreviations for states, Don't use the word county nor country. Use the full word for countries, including USA for USA. However, more currently, I am considering using the state abbrev. for level 0 in the USA, and dropping USA. That is mainly because I have probably over 10,000 names with the states as two letter abbreviations at level 0 and each name may have three locations. And to do a search on 50 state abbreviations just to add ", USA" to end of each one could be herindous. But I can search and delete, ", USA" quickly. I will tryout the editors you mentioned.
I truly appreciate the help you folks provide the newcomers like myself. I really like the program and it is very stable. I've been able to get the memory limit of my server moved up to 64 mB. But that isn't enough for some of the features.
Frankly Frank
PS - I meant to add: I'm not sure it is wise to leave the USA county off. That could be something I will truly regret. I read about leaving a space for it between commas if it is unknown. I am struggling with the idea of using "unknown" at the USA county level and including that "location" below the state level for each USA state. And use the coordinates of the state for "unknown".
This is one of those situations Stephen mentions that I could be very sorry about later if I make the wrong decision now.
Some additional questions etc. Statement: I got into trouble when I downloaded my GEDCOM, made changes to be uniform with locations, and uploaded it. My problem was that the text editor left out a lot of carriage returns or whatever it is that separated one individual gedcom from the next. I didn't know how to restore correctly. I think I was also trying to upgrade.
QUESTION: I and all the folks who add to my data are synced to the GEDCOM. Are you people saying that I should change this and then be verifying each change? I have 10K individual names in one of my gedcoms. QUESTION: How can I import just one, or two, individuals that got "lost" as noted in my statement above? Can I import a person with a specific ID #?
Frank
Thanks for the thanks. BUT …Don't remove the USA and consider adding the full states. Really easy with the batch utility or a search and replace in the text editor outside PGV. just 50 actions:
first field; ", AL"
second field: ", Alabama, USA"
etc. It would take less than about 1/2 to 3/4 of an hour to change them all and you'll be golden.
Also, I'm at a loss to understand why the "no counties'. That would be a most unusual setting as many, many times, it is only the county that is specified in the older SOUR index references (IN, OH, TX births and deaths, etc). Also, you really have to be sure your GEDCOM specified PLAC settings match your actual PLAC entries.
Good luck, -Stephen
amdrew
If the records are no longer recorded in GEDCOM text format, but rather lengthy strings, there is little you can do. If it is only the CR between records, you could possibly do a Search & Replace for all "0" level records, something like:
find: "0 "
replace: "\r 0 "
but this would only work if the CR are missing between the major level 0 records. If it removed all the CR, it would be a bigger chore. Go back to your original backup copy and then do your PLAC replace again without wiping out the CR's.
-Stephen
Stephen.
Thanks. I will leave USA and continue inputting them. I will also leave the counties. And add counties to new entries.
And if you've learned that whole state name should be used rather than just the abbreviation, I'll do that as well.
I just needed some guidance in that area. I am using the Google module and leaving the counties out, seemed to help that process. I've been manually entering every location into Google. Perhaps I should be importing the state data that has the city and county information already.
I reverted back to a 5-day old GEDCOM, which was before I mutilated it by removing a bunch of counties. I used the internal Batch editor and converted all states to full names and added USA to them.
Worked Great!
Now, I need to start building the Google module places to conincide with the state names.
I used the List - Place Hierarchy to drill down and clean up the top-level locations, which happens to be countries in my case. It looks great. Then I went to Edit Geographic Place Locations, which is under the map on the Place Hierarchy page. The list shown there matches the list I had just cleaned up until I click on "Click here to show inactive places:" Then a bunch of non-top level locations show up.
Does it make sense that I need to clean those up too? Does that list come from a different database than the first one does?
Thank you for the help, I'm still trying to understand all the features and data structure.
Frank
Frank
Not exactly hijacking your own thread, and not exactly an entirely new subject, but IMHO, different enough that you should start a new thread for your GM question. Before you do so, please read the extensive description in the WIKI and search these forums for previous postings nearly identical to your issues.
-Stephen
I've researched this for hours but have had no luck finding a clear answer. Perhaps I'm not looking in the right place. I haven't found anything in Wiki that gets as technical as what I am asking. And I've read a lot of posts that are indeed technical, but I haven't seen that they address this particular issue.
On the bright side, I have found answers to most of my questions online here or in WIKI.
If the answer to my question is online, I'll look some more before starting a new thread.
Frank
I think your answers are there, but it depends more on how you wish to use GM. If you wish to have your places generally conform to the GM locations, then you can simply delete your placelocation (empty it) table and import places from the data provided as additional locations, or you can use the GEDCOM for your places by using the import from gedcoms. I would empty the table first so you can eliminate all those 'dead' locations.
Others may have different solutions and I'm not an expert on GM module. I'm not sure anyone here has been involved in its development and it is also now deprecated as it is v2 API and GM is now onto v3. Only the webtrees program will be upgrading soon to v3 of the API.
-Stephen