After a few headaches trying to get PGV to install the csv's for the US (the ones that came with PGV) I discovered that I had to break them down into smaller packets. After doing this I figure I won't be the only one who wants these csv's and I won't be the only one with this problem so I've zipped them and if you want them you can download them from www.waltonandninnes.id.au/USA.zip
Each csv contains 1500 lines or less and is less than 100 Kb in size.
All of these locations contain county information which is great for those that like historical accuracy but can be unfortunate for anyone with a more modern outlook, eg:
I have a birth listed as "St Joseph's Hospital, Augusta, Georgia, USA" but when I look to the map it just puts the flag in the middle of Georgia, 100 miles from Augusta. This is because the place of birth listed doesn't list the County. Looking the place up in google shows you the correct location but also doesn't tell you which county Augusta is in. It took me half an hour to find a map that shows the county boundaries for Georgia so I now know that Augusta is in Richmond, Georgia. This is going to be a painful excercise for the other 2000 odd relatives that come fron the US.
Anyway, hope that zip helps someone.
Cheers,
Andrew.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Where can i find information about how these files are used? I.e. where are they supposed to be copied to?
I could use the same information for Norway, is there a description somewhere saying how they shall be built?
John Morten
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The files are quite easy to generate, just take a look at one of the files.
The contents of the CVS-files is:
1. Level: 1=province/stae, 2=county, 3=place
2. Country
3. province/state
4. Place (leave empty if level=2)
5. Longitude
6. Latitude
7. Zoom level (usually 12 or 13)
8. Icon (optional)
The icon can contain for example a flag of a city, county or province.
You can generate the files using a spreadsheet program, like Excel.
The best way to start is by getting hold of a list of all places with their longitude and latitude (that is how I generated to Dutch and US files). Try some government site, like the geological survey organization, they might have some files available.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2009-09-12
Andrew, can you elaborate on "a few headaches trying to get PGV to install the csv's for the US (the ones that came with PGV) I discovered that I had to break them down into smaller packets" please.
The original files are not large. Maximum size for any state is 584KB, and most are much smaller than that. I've never know before of any problems uploading these as required. Your statement may worry some users thinking of using them, and I'm sure thats not necessary.
John, there is extensive information about the GM module and the extra files in the WIKI:
The "headaches" seem to be a slow server with time out problems. I kept getting PDO exceptions with "error: 2006 MySQL has gone away".
Lots of googling and reading suggested to me that this is a recently fixed bug in PDO that only affected installations on Windows machines, but when I begged bluehost to upgrade PDO they assured me that it was only a time out error.
So I broke down the csv's into little packets and got very frustrated because they'd load in my own apache server without problems but still refused to load on bluehost. Then the penny dropped - I'm using Linux and the default in my text editor (kwrite) is for Unix end of line characters. I changed that to DOS/Win end of line and was back in business. But I still got MySQL taking a holiday on many of the files, trial and error proved to me that 1500 lines was about as many as I could feed it in one go.
A couple of things about this exercise still confuse me though. My own server was happy with the Unix end of lines, does this mean BlueHost are running Apache on a Microtoss machine?
Also, when I first set up PGV google maps I added all of Australia and England csv's without a hitch, New South Wales csv has over 3000 lines and is 170 Kb in size and went in with no trouble. Why does it time out on smaller files now? Does the database get checked for duplicate entries as more entries are added? There's a lot of entries in there to check against now.
I certainly don't mean to put anyone off of trying out those csv's, having them already there to use is a real boon. Manually entering the location for each and every relative would take forever.
John Morten, if you have a look inside the /modules/googlemap/extra/USA folder
you'll find a ReadMe on how the locations were extracted from a U.S. Geological Survey data file and a copy of the script (generate.awk) that was used to do this. I whish the author had included their name in the ReadMe so I could say thanks.
Cheers,
Andrew.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I can solve that last mystery: that was me. It was part of my original development of the Googlemap module. I did have quite a large number of relatives in the US, and as I am lazy i did not want to check the GPS coordinates of all the locations.
I found the US Geological Survey data and decided that I could use this to generate all the data I needed.
I found that this is the best way: find some government office that tracks locations and convert their data into data for the Googlemap module. I also did that for the Netherlands.
Regards,
Johan
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for that thomasadkins, very handy indeed. Exactly the tool I needed.
Andrew.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2009-09-16
Are you both aware that the Google Map module in PGV does a pretty good job at finding lat/long coordinates itself, if used correctly? Its always been a feature, ever since borkhuis' original introduction, but has improved over time. I rarely need anything else these days.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Your map module does a superb job at finding coordinates and it's easy and intuitive to use, If you only have relatives in 2 or 3 towns of a particular country that is definitely the way to do it.
But that can get a bit tedious when you want to list 500 or so towns in one country. Also, using a csv to create your initial layout with proper place hierarchy to start with does make the job a bit easier for later on.
I also really like the Place Check feature in the Admin page, it really shows up all the spelling errors that have been made in the gedcom over the years.
There is one small bug with the module though (v 4.2.2) where if you click on "Import from current GEDCOM" it loads places from your notes as well as places. This can be quite entertaining if you've got a bit of spare time, I had a country called "Alzheimers - choked on a piece of sponge cake" amongst others. This bug also manifests in the Place Check function.
And I've made a few more csv's for anyone that wants them:
www.waltonandninnes.id.au/NewZealand.csv
www.waltonandninnes.id.au/Ireland.csv
www.waltonandninnes.id.au/Denmark.csv
Denmark is post 2006
Cheers,
Andrew.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2009-09-17
Andrew, the New Zealand csv is a good start, but you do realise you've missed out the regions completely don't you - Northland, Auckland, Waikato, Bay of Plenty, Taranaki, Whanganui, Gisborne, Hawkes Bay, Wellington, Tasman, Marlborough, West Coast, Canterbury, Otago, Southland. There is already a file of these in the Patches section here (under the Tracker menu above), which is where you should really be posting these files anyway.
Your Irish file is inconsistent. It is important to decide whether you are including the County level or not (I believe you should). If you are, then you should ALWAYS include it. Its also questionable whether counties should be referenced as "County Clare" or just "Clare". Personally I use the latter, but even the Irish themselves don't always agree on this.
On your NOTES issue - can you give us a snippet of the GEDCOM that is being included. My understanding is that only PLAC tags are read. You shouldn't have any of those in your data.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
With the csv's, I left the region information out of New Zealand deliberately as when ever I've spoken to anyone from New Zealand they've never mentioned regions, in fact if you ask usually the most you get is "it's on the South Island". This tied in with the info we have on relatives as well.
Ireland was directly ripped from a data set from GeoNames.org and the naming convention was left as is. I've since discovered a bit of a mixup with their data sets in the regional information, eg: in the GB set Pudsey isn't in Yorkshire, it's in Greater London, in fact a great many places seem to be in Greater London. I tried a data set from NGA and it's got almost identically the same errors.
As for the errors with places in my gedcom - you're right and I do apologise, that gedcom is a real mess with all sorts of things showing up in the PLAC tags.
Thanks for the help,
Andrew.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2009-09-18
Andrew, you'll have to update your "whenever I've spoken to anyone from New Zealand " comment - why do you think I'm called 'kiwi" :-))
Seriously though, to make the best use of both the GM module and PGV's standard Place Heirarchy, it does pay to use multi-levels. In the module, if you don't have coordinates loaded for a specific town, the map can at least find a region, which is better than just a country.
It can be harder to train users to get it right, but trust me, consistency really does pay off in this area.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Boundaries do change over time - especially if you take the electoral ones!
If you are using the Wises then it gives you a **town**.
If you are using the Electoral Rolls then you get the **electorate** - whatever that is at the time.
Or you use our current day understanding of regions which, who knows, may change in years to come but on what grounds would you know a region?
I don't think that anything is hard and fast but I for one would be unlikely to use 'regions' - but rather towns or electorates as there are SOURCES for those!!! :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
After a few headaches trying to get PGV to install the csv's for the US (the ones that came with PGV) I discovered that I had to break them down into smaller packets. After doing this I figure I won't be the only one who wants these csv's and I won't be the only one with this problem so I've zipped them and if you want them you can download them from www.waltonandninnes.id.au/USA.zip
Each csv contains 1500 lines or less and is less than 100 Kb in size.
All of these locations contain county information which is great for those that like historical accuracy but can be unfortunate for anyone with a more modern outlook, eg:
I have a birth listed as "St Joseph's Hospital, Augusta, Georgia, USA" but when I look to the map it just puts the flag in the middle of Georgia, 100 miles from Augusta. This is because the place of birth listed doesn't list the County. Looking the place up in google shows you the correct location but also doesn't tell you which county Augusta is in. It took me half an hour to find a map that shows the county boundaries for Georgia so I now know that Augusta is in Richmond, Georgia. This is going to be a painful excercise for the other 2000 odd relatives that come fron the US.
Anyway, hope that zip helps someone.
Cheers,
Andrew.
Google is a great help when you're trying to figure out the counties.
Search for "augusta georgia" and you'll get lots of hits. If there's a Wiki entry, go for that.
Where can i find information about how these files are used? I.e. where are they supposed to be copied to?
I could use the same information for Norway, is there a description somewhere saying how they shall be built?
John Morten
The files are quite easy to generate, just take a look at one of the files.
The contents of the CVS-files is:
1. Level: 1=province/stae, 2=county, 3=place
2. Country
3. province/state
4. Place (leave empty if level=2)
5. Longitude
6. Latitude
7. Zoom level (usually 12 or 13)
8. Icon (optional)
The icon can contain for example a flag of a city, county or province.
You can generate the files using a spreadsheet program, like Excel.
The best way to start is by getting hold of a list of all places with their longitude and latitude (that is how I generated to Dutch and US files). Try some government site, like the geological survey organization, they might have some files available.
Andrew, can you elaborate on "a few headaches trying to get PGV to install the csv's for the US (the ones that came with PGV) I discovered that I had to break them down into smaller packets" please.
The original files are not large. Maximum size for any state is 584KB, and most are much smaller than that. I've never know before of any problems uploading these as required. Your statement may worry some users thinking of using them, and I'm sure thats not necessary.
John, there is extensive information about the GM module and the extra files in the WIKI:
http://wiki.phpgedview.net/en/index.php?title=Google_Map_module
Hi Kiwi,
The "headaches" seem to be a slow server with time out problems. I kept getting PDO exceptions with "error: 2006 MySQL has gone away".
Lots of googling and reading suggested to me that this is a recently fixed bug in PDO that only affected installations on Windows machines, but when I begged bluehost to upgrade PDO they assured me that it was only a time out error.
So I broke down the csv's into little packets and got very frustrated because they'd load in my own apache server without problems but still refused to load on bluehost. Then the penny dropped - I'm using Linux and the default in my text editor (kwrite) is for Unix end of line characters. I changed that to DOS/Win end of line and was back in business. But I still got MySQL taking a holiday on many of the files, trial and error proved to me that 1500 lines was about as many as I could feed it in one go.
A couple of things about this exercise still confuse me though. My own server was happy with the Unix end of lines, does this mean BlueHost are running Apache on a Microtoss machine?
Also, when I first set up PGV google maps I added all of Australia and England csv's without a hitch, New South Wales csv has over 3000 lines and is 170 Kb in size and went in with no trouble. Why does it time out on smaller files now? Does the database get checked for duplicate entries as more entries are added? There's a lot of entries in there to check against now.
I certainly don't mean to put anyone off of trying out those csv's, having them already there to use is a real boon. Manually entering the location for each and every relative would take forever.
John Morten, if you have a look inside the /modules/googlemap/extra/USA folder
you'll find a ReadMe on how the locations were extracted from a U.S. Geological Survey data file and a copy of the script (generate.awk) that was used to do this. I whish the author had included their name in the ReadMe so I could say thanks.
Cheers,
Andrew.
Andrew,
I can solve that last mystery: that was me. It was part of my original development of the Googlemap module. I did have quite a large number of relatives in the US, and as I am lazy i did not want to check the GPS coordinates of all the locations.
I found the US Geological Survey data and decided that I could use this to generate all the data I needed.
I found that this is the best way: find some government office that tracks locations and convert their data into data for the Googlemap module. I also did that for the Netherlands.
Regards,
Johan
For identifying the lat-longs or county of a city, town or township, the best resource I've found is www.epodunk.com
Thanks for that thomasadkins, very handy indeed. Exactly the tool I needed.
Andrew.
Are you both aware that the Google Map module in PGV does a pretty good job at finding lat/long coordinates itself, if used correctly? Its always been a feature, ever since borkhuis' original introduction, but has improved over time. I rarely need anything else these days.
Your map module does a superb job at finding coordinates and it's easy and intuitive to use, If you only have relatives in 2 or 3 towns of a particular country that is definitely the way to do it.
But that can get a bit tedious when you want to list 500 or so towns in one country. Also, using a csv to create your initial layout with proper place hierarchy to start with does make the job a bit easier for later on.
I also really like the Place Check feature in the Admin page, it really shows up all the spelling errors that have been made in the gedcom over the years.
There is one small bug with the module though (v 4.2.2) where if you click on "Import from current GEDCOM" it loads places from your notes as well as places. This can be quite entertaining if you've got a bit of spare time, I had a country called "Alzheimers - choked on a piece of sponge cake" amongst others. This bug also manifests in the Place Check function.
And I've made a few more csv's for anyone that wants them:
www.waltonandninnes.id.au/NewZealand.csv
www.waltonandninnes.id.au/Ireland.csv
www.waltonandninnes.id.au/Denmark.csv
Denmark is post 2006
Cheers,
Andrew.
Andrew, the New Zealand csv is a good start, but you do realise you've missed out the regions completely don't you - Northland, Auckland, Waikato, Bay of Plenty, Taranaki, Whanganui, Gisborne, Hawkes Bay, Wellington, Tasman, Marlborough, West Coast, Canterbury, Otago, Southland. There is already a file of these in the Patches section here (under the Tracker menu above), which is where you should really be posting these files anyway.
Your Irish file is inconsistent. It is important to decide whether you are including the County level or not (I believe you should). If you are, then you should ALWAYS include it. Its also questionable whether counties should be referenced as "County Clare" or just "Clare". Personally I use the latter, but even the Irish themselves don't always agree on this.
On your NOTES issue - can you give us a snippet of the GEDCOM that is being included. My understanding is that only PLAC tags are read. You shouldn't have any of those in your data.
With the csv's, I left the region information out of New Zealand deliberately as when ever I've spoken to anyone from New Zealand they've never mentioned regions, in fact if you ask usually the most you get is "it's on the South Island". This tied in with the info we have on relatives as well.
Ireland was directly ripped from a data set from GeoNames.org and the naming convention was left as is. I've since discovered a bit of a mixup with their data sets in the regional information, eg: in the GB set Pudsey isn't in Yorkshire, it's in Greater London, in fact a great many places seem to be in Greater London. I tried a data set from NGA and it's got almost identically the same errors.
As for the errors with places in my gedcom - you're right and I do apologise, that gedcom is a real mess with all sorts of things showing up in the PLAC tags.
Thanks for the help,
Andrew.
Andrew, you'll have to update your "whenever I've spoken to anyone from New Zealand " comment - why do you think I'm called 'kiwi" :-))
Seriously though, to make the best use of both the GM module and PGV's standard Place Heirarchy, it does pay to use multi-levels. In the module, if you don't have coordinates loaded for a specific town, the map can at least find a region, which is better than just a country.
It can be harder to train users to get it right, but trust me, consistency really does pay off in this area.
kiwi-pgv
Boundaries do change over time - especially if you take the electoral ones!
If you are using the Wises then it gives you a **town**.
If you are using the Electoral Rolls then you get the **electorate** - whatever that is at the time.
Or you use our current day understanding of regions which, who knows, may change in years to come but on what grounds would you know a region?
I don't think that anything is hard and fast but I for one would be unlikely to use 'regions' - but rather towns or electorates as there are SOURCES for those!!! :)
I have been having the same problem as Andrew - some of the states hit a time out value during import. His split files work well. Thanks Andrew!
BTW, this is with my spiffy new host, lunarpages. They have the latest and greatest of everything compared to my old fleabitten host.