which prevents from recalulating the common_surnames for every page but uses the admin-set common_names_add instead.
What about other tuning methods ?
Wouldn't it make sense to rewrite these functions ?
In fact actually it takes much more time calculating than transmitting the pages. I'm very unhappy with this :-(.
Another possible cause I've read about is that the ISDEAD-status gets calculated the first time an Indi is requested. This also would slow down the page after reimporting. I'd prefer a longer import (which lasts only 2 seconds for me) but a much slower page generation time (pedigree charts with 4 generations last more than 20 seconds!).
Peter
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Up to now, i've used PGV on my webserver, which runs in MySQL mode. I experienced no speed-problem whatsoever.
Yesterday I installed PGV on my laptop in index mode and now I can understand your feelings fully!
The On this day.... block and Upcoming events block take ages to calculate......
I can't get the common surnames right. The higher I set the threshold, the more names appear on the screen.
Since all works well in MySQL mode, I think removing calculations etc. would only be a work around for the real problem (which may be best at this time). I hope one of the developers will spend some time on this issue to get it right. Maybe tuning won't be neccessary any more.
Boudewijn.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Maybe I also have to talk to my admin, as he set php time limit to 30s - I don't know how much memory he has spent.... perhaps not enough.
Would you please tell me if my site is significantly slower than yours ?
THanks
Peter
PS:
perhaps the following lines explain the behaviour of surnames threshold:
$surnames = get_common_surnames($COMMON_NAMES_THRESHOLD);
if (count($surnames)<10) $surnames = get_common_surnames($COMMON_NAMES_THRESHOLD/2);
So perhaps when increasing threshold you reached a point where less than 10 names would have been found so it started again with threshold/2 and resulted in MORE names than before.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I checked your welcome page and it gives a responsetime of about 7 or 8 secs. Which is very good compared with the 30-60 I have on my laptop.
My own site has a responsetime of 1-3 secs, but that is without the transmissiontime, as I run the site here at home on my webserver.
I'll send you an URL, username and pass separately, so you can check my site's speed from your place and compare a bit more. It might be handy to switch om the execution time display on your site, that way you don't have to count manually ;-)
I will do some more testing with the common surnames and will remove the divisionline. That way it should do what it's told. I will send these results later.
Regards,
Boudewijn.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I removed the division factor from the two files and the common names list has become (much more) predictable. It's now like it's descibed in the helptext too.
The changed files are upped to CVS.
Bye
Boudewijn
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It seems that it depends on the site I call if your or mine server is faster. Though some things like indilist "all" is impossible on mine in cause of the php execution time > 30s limit.
Nevertheless I think even your site is too slow. From here the pedigree lasts 15-18 s (one time 25s!) and the welcom page takes 5s (which is nearly acceptable) multimedialist takes 7s.
So probably all of our team (like you and me) have tested on their own local server and nobody has mentioned that calculation time increased significantly on 3.x
Peter
PS: I think this common surnames divisionline was a (still badly documented) FEATURE by John Finlay and is not a bug...
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I agree with you that some of the responsetimes are long. I experimented on that with Roland a while ago and he managed to get the time down from 47 to some around 20. Due to adding the handling of different languages, it has increased to about 30 secs again. But I'm not convinced it was better before 3.0.
The division factor was indeed part of the feature, but I think it brought some confusion to the admins. I removed it from the coding, but it can be put back easily.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Index mode is always going to have speed issues because in order to search you have to iterate across all of the individuals and because it takes longer to parse the entire index array into memory. So the more individuals you have the more time it is going to take.
You can check my index test site at http://john.lib.byu.edu/pgv_index/
It is running my GEDCOM with 6900 individuals and it can create the home page in about 4 seconds. This is a 1.5Ghz P4 with over 300MB of RAM running XP Pro.
Speed is greatly affected by the privacy settings. You'll notice that when you login as an administrator the pages should render about twice as fast.
Speed is also very system dependent. For example it is dependent upon the processor speed, the amount of memory, and the system cache. Cache and Memory should have a significant iimpact on index mode because it is so memory intensive. I would hate to see the performance ratings runing it on a celeron :P Slow network connections have also been shown to give you slower performance output.
Speed is also dependent upon the operating system. Win2000 or XP Pro should give you a lot better performance than Win 98 or regular XP. Linux should out perform windows, but I've never done a true performance evaluation.
Peter, if you would like, you can send me your gedcom file and I can test it on my systems to see if there is something unique about your gedcom that could be causing the problem.
The feature that cuts the common names threshold in half was added to ensure that there was always at least 10 names in the list. I added this because in the currently released code there is not an easy way to edit the threshold. This feature can and should be removed in the CVS because the CVS code has a way to edit the threshold parameters in the gedcom configuration.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Those who are running PGV on web hosts may be able to solve their problem by finding a speedy host. My host, JaguarPC costs under $85 per year and has always shone best at speed, both transfer speed and processing speed. That said, I don't have very big Gedcoms.
Peter, maybe you could send me your files to try them as a non-wizard at my site also http://www.hawsedc.com/phpGedView/ . Contact me there.
Tom Haws
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've updated the CVS code so that it no longer needs to calculate the common surnames on every page. This will improve overall speed of the site. The common surnames are now stored in the $GEDCOMS array in the index/gedcoms.php file.
If you upgrade to this code and are hosting multiple gedcoms, you may need to edit and save each gedcom configuration for the change to take place.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just as a procedural thing - when you do a major change like this do you recommend that it be installed from CVS - and then should we also apply all other changes to other modules since the previous full file update (in case of dependancies) ?
Jim
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
John has updated the code for the common surnames AND he optimised the algorith for checking media which are attached as OBJE-records within the gedcom.
So everybody who uses many medias in OBJE-records should find INDEX-mode much faster now.
In fact for several pages (at least on my server, and with my special Gedcom) the INDEX-mode seems to be faster than MySQL mode...
(By the way, also my webadmin's php settings have been optimized - has been set to 8M and 30s before I asked him)
So thanks again for all the improvements in velocity!
Peter
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
i did only partially and have some notes:
result probably depends on number of indis, number of fams, number of media (and how they are connected: inside INDI-record or by link to an OBJE-record).
For 2500 indis and 120 medias in separate OBJE tags I can tell on a ~350MHz PII running apache, MySQL, PHP 4.3.2 the index-mode beats the MYSQL mode in welcom-page(3s), pedigree-tree(4s), medialist(2s). Only in individual list [ALL] the mysql is slightly faster (70s).
Probably on larger gedcoms the mySQL mode will show its points....
Regards
Peter
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Mark wrote:
"am I reading "70s" correctly as 70 seconds ... is that the absolute time or the time difference between index & MySQL?"
yes it has been 70 seconds in total (here the index mode in fact took about 78 seconds, while MySQL did it in abuot 70 seconds - the other sites have been faster in Index-mode for my configuration/gedcom).
"My gedcom in MySQL mode with 7102 individuals does
indilist.php?show_all=yes&surname_sublist=yes
Total Execution time: 18.040 sec. Total Database Queries: 16
indilist.php?show_all=yes&surname_sublist=no
Total Execution time: 36.595 sec. Total Database Queries: 16"
so probably your webserver is much faster than a Dual-Pentium II 350MHz (or is running less other services).
I assume that my provider's machine is one of the slower ones out there at the moment....
Peter
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
There is still a speed issue with the MyGedView page for logged-in users. The upcoming events block is very slow in index mode. I have been trying to fix this, but still haven't figured out a better way to do it.
--John
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
as the latest version seems to very slow I'd like to ask how to "tune" PGV.
Ways which have been mentioned before/which I found myself are:
- remove "on this day" block
- remove "random picure" block
- change functions_print.php lines 599 and 738 from
$surnames = get_common_surnames($COMMON_NAMES_THRESHOLD);
to
$surnames = $COMMON_NAMES_ADD;
which prevents from recalulating the common_surnames for every page but uses the admin-set common_names_add instead.
What about other tuning methods ?
Wouldn't it make sense to rewrite these functions ?
In fact actually it takes much more time calculating than transmitting the pages. I'm very unhappy with this :-(.
Another possible cause I've read about is that the ISDEAD-status gets calculated the first time an Indi is requested. This also would slow down the page after reimporting. I'd prefer a longer import (which lasts only 2 seconds for me) but a much slower page generation time (pedigree charts with 4 generations last more than 20 seconds!).
Peter
Hi Peter,
Up to now, i've used PGV on my webserver, which runs in MySQL mode. I experienced no speed-problem whatsoever.
Yesterday I installed PGV on my laptop in index mode and now I can understand your feelings fully!
The On this day.... block and Upcoming events block take ages to calculate......
I can't get the common surnames right. The higher I set the threshold, the more names appear on the screen.
Since all works well in MySQL mode, I think removing calculations etc. would only be a work around for the real problem (which may be best at this time). I hope one of the developers will spend some time on this issue to get it right. Maybe tuning won't be neccessary any more.
Boudewijn.
Hi Boudewijn,
I've switched to MySQL inbetween but I think it's still slow.
Would you please have a look at
http://ahnen.pluntke.com/script/phpp/index.php.
Maybe I also have to talk to my admin, as he set php time limit to 30s - I don't know how much memory he has spent.... perhaps not enough.
Would you please tell me if my site is significantly slower than yours ?
THanks
Peter
PS:
perhaps the following lines explain the behaviour of surnames threshold:
$surnames = get_common_surnames($COMMON_NAMES_THRESHOLD);
if (count($surnames)<10) $surnames = get_common_surnames($COMMON_NAMES_THRESHOLD/2);
So perhaps when increasing threshold you reached a point where less than 10 names would have been found so it started again with threshold/2 and resulted in MORE names than before.
Hi Peter,
I checked your welcome page and it gives a responsetime of about 7 or 8 secs. Which is very good compared with the 30-60 I have on my laptop.
My own site has a responsetime of 1-3 secs, but that is without the transmissiontime, as I run the site here at home on my webserver.
I'll send you an URL, username and pass separately, so you can check my site's speed from your place and compare a bit more. It might be handy to switch om the execution time display on your site, that way you don't have to count manually ;-)
I will do some more testing with the common surnames and will remove the divisionline. That way it should do what it's told. I will send these results later.
Regards,
Boudewijn.
Hi Peter,
I removed the division factor from the two files and the common names list has become (much more) predictable. It's now like it's descibed in the helptext too.
The changed files are upped to CVS.
Bye
Boudewijn
Hi Boudewijn,
thanks for the user/pass.
It seems that it depends on the site I call if your or mine server is faster. Though some things like indilist "all" is impossible on mine in cause of the php execution time > 30s limit.
Nevertheless I think even your site is too slow. From here the pedigree lasts 15-18 s (one time 25s!) and the welcom page takes 5s (which is nearly acceptable) multimedialist takes 7s.
So probably all of our team (like you and me) have tested on their own local server and nobody has mentioned that calculation time increased significantly on 3.x
Peter
PS: I think this common surnames divisionline was a (still badly documented) FEATURE by John Finlay and is not a bug...
Hi Peter,
I agree with you that some of the responsetimes are long. I experimented on that with Roland a while ago and he managed to get the time down from 47 to some around 20. Due to adding the handling of different languages, it has increased to about 30 secs again. But I'm not convinced it was better before 3.0.
The division factor was indeed part of the feature, but I think it brought some confusion to the admins. I removed it from the coding, but it can be put back easily.
Index mode is always going to have speed issues because in order to search you have to iterate across all of the individuals and because it takes longer to parse the entire index array into memory. So the more individuals you have the more time it is going to take.
You can check my index test site at http://john.lib.byu.edu/pgv_index/
It is running my GEDCOM with 6900 individuals and it can create the home page in about 4 seconds. This is a 1.5Ghz P4 with over 300MB of RAM running XP Pro.
Speed is greatly affected by the privacy settings. You'll notice that when you login as an administrator the pages should render about twice as fast.
Speed is also very system dependent. For example it is dependent upon the processor speed, the amount of memory, and the system cache. Cache and Memory should have a significant iimpact on index mode because it is so memory intensive. I would hate to see the performance ratings runing it on a celeron :P Slow network connections have also been shown to give you slower performance output.
Speed is also dependent upon the operating system. Win2000 or XP Pro should give you a lot better performance than Win 98 or regular XP. Linux should out perform windows, but I've never done a true performance evaluation.
Peter, if you would like, you can send me your gedcom file and I can test it on my systems to see if there is something unique about your gedcom that could be causing the problem.
The feature that cuts the common names threshold in half was added to ensure that there was always at least 10 names in the list. I added this because in the currently released code there is not an easy way to edit the threshold. This feature can and should be removed in the CVS because the CVS code has a way to edit the threshold parameters in the gedcom configuration.
--John
Those who are running PGV on web hosts may be able to solve their problem by finding a speedy host. My host, JaguarPC costs under $85 per year and has always shone best at speed, both transfer speed and processing speed. That said, I don't have very big Gedcoms.
Peter, maybe you could send me your files to try them as a non-wizard at my site also http://www.hawsedc.com/phpGedView/ . Contact me there.
Tom Haws
I've updated the CVS code so that it no longer needs to calculate the common surnames on every page. This will improve overall speed of the site. The common surnames are now stored in the $GEDCOMS array in the index/gedcoms.php file.
If you upgrade to this code and are hosting multiple gedcoms, you may need to edit and save each gedcom configuration for the change to take place.
--John
Just as a procedural thing - when you do a major change like this do you recommend that it be installed from CVS - and then should we also apply all other changes to other modules since the previous full file update (in case of dependancies) ?
Jim
So many things have been done now.
John has updated the code for the common surnames AND he optimised the algorith for checking media which are attached as OBJE-records within the gedcom.
So everybody who uses many medias in OBJE-records should find INDEX-mode much faster now.
In fact for several pages (at least on my server, and with my special Gedcom) the INDEX-mode seems to be faster than MySQL mode...
(By the way, also my webadmin's php settings have been optimized - has been set to 8M and 30s before I asked him)
So thanks again for all the improvements in velocity!
Peter
Did anyone ever run a PGV benchmark to test speed differences between versions?
i did only partially and have some notes:
result probably depends on number of indis, number of fams, number of media (and how they are connected: inside INDI-record or by link to an OBJE-record).
For 2500 indis and 120 medias in separate OBJE tags I can tell on a ~350MHz PII running apache, MySQL, PHP 4.3.2 the index-mode beats the MYSQL mode in welcom-page(3s), pedigree-tree(4s), medialist(2s). Only in individual list [ALL] the mysql is slightly faster (70s).
Probably on larger gedcoms the mySQL mode will show its points....
Regards
Peter
am I reading "70s" correctly as 70 seconds ... is that the absolute time or the time difference between index & MySQL?
My gedcom in MySQL mode with 7102 individuals does
indilist.php?show_all=yes&surname_sublist=yes
Total Execution time: 18.040 sec. Total Database Queries: 16
indilist.php?show_all=yes&surname_sublist=no
Total Execution time: 36.595 sec. Total Database Queries: 16
Mark
Mark wrote:
"am I reading "70s" correctly as 70 seconds ... is that the absolute time or the time difference between index & MySQL?"
yes it has been 70 seconds in total (here the index mode in fact took about 78 seconds, while MySQL did it in abuot 70 seconds - the other sites have been faster in Index-mode for my configuration/gedcom).
"My gedcom in MySQL mode with 7102 individuals does
indilist.php?show_all=yes&surname_sublist=yes
Total Execution time: 18.040 sec. Total Database Queries: 16
indilist.php?show_all=yes&surname_sublist=no
Total Execution time: 36.595 sec. Total Database Queries: 16"
so probably your webserver is much faster than a Dual-Pentium II 350MHz (or is running less other services).
I assume that my provider's machine is one of the slower ones out there at the moment....
Peter
does tthe calculation include "wire time" of sending so much data (the entire indi list)? would the server's connection speed play a role?
Whereas another site that I know which uses v2.51
indilist.php?show_all=yes&surname_sublist=no
Total Execution time: 7.907 sec. Total Database Queries: 2.
indilist.php?show_all=yes&surname_sublist=yes
Total Execution time: 9.114 sec. Total Database Queries: 2.
Although I do notice that this version doesn't actually sublist the surnames - you always get the full list.
Not sure whether this one is in index mode or MySQL, and he might have privacy turned off and the GEDCOM only contains dead people.
Looking forward to 3.00.2 (or 3.01 or 3.1) whichever comes out.
Mark
There is still a speed issue with the MyGedView page for logged-in users. The upcoming events block is very slow in index mode. I have been trying to fix this, but still haven't figured out a better way to do it.
--John