Menu
▾
▴
ids-devel
|
From: Jeff D. <je...@ba...> - 2002-12-31 18:52:54
|
I think the main "lack of speed" most people are experiencing in IDS is
the amount of time it takes ImageMagick to create the thumbs and the various
image sizes. The speed at which this operation occurs can only be improved
by a faster CPU and possibly a very small amount by a faster hard drive.
FastCGI and modperl are used when a site is expecting a tremendous
amount of traffic, perhaps on the order of one dynamic page requested per
second or higher. That is the point when the script and interpreter startup
overhead starts to load the server down. Implementing FastCGI and/or
modperl support doesn't make any noteable speed increase to a single user.
It just prevents the system from buckling when you get 1,000 visitors /
20,000 page views an hour.
Jeff
|
|
From: Scott S. <ss...@sc...> - 2002-12-31 19:42:52
|
I would concur with that assessment. I also notice that for the most part people only look at the 512 sizes of the images. If the previewmaker could create those as well then that would solve a lot of the speed issues that I see. I do have though over probably 10k images at this point and that will eat up a lot of disk space. The whole speed for space problem. -Scott On Tue, 2002-12-31 at 11:48, Jeff Dafoe wrote: > > I think the main "lack of speed" most people are experiencing in IDS is > the amount of time it takes ImageMagick to create the thumbs and the various > image sizes. The speed at which this operation occurs can only be improved > by a faster CPU and possibly a very small amount by a faster hard drive. > FastCGI and modperl are used when a site is expecting a tremendous > amount of traffic, perhaps on the order of one dynamic page requested per > second or higher. That is the point when the script and interpreter startup > overhead starts to load the server down. Implementing FastCGI and/or > modperl support doesn't make any noteable speed increase to a single user. > It just prevents the system from buckling when you get 1,000 visitors / > 20,000 page views an hour. > > > Jeff > > > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > IDS-devel mailing list > IDS...@li... > https://lists.sourceforge.net/lists/listinfo/ids-devel > -- Scott Sawyer o: 928 226 0404 Owner c: 602 920 0083 Scott Sawyer Photography http://www.scottsawyer.org Capturing today's moments, forever |
|
From: Piet B. <pie...@ya...> - 2002-12-31 20:19:03
|
I generally do a 'wget' to do the dirty work of generating those 512 byte wide images ahead of time, instead of when the visitor gets to the site. This may take a long time if you have a lot of images, or a lot of big images, especially on a slow computer. At least the images get stored in the image-cache directory. The command to recursively get up to 8 levels of links from a site is: wget -r -np -l 5 URL (where "URL" is your home page) (You can use the --delete-after option of wget to have the images and files deleted immediately after download) --- Scott Sawyer <ss...@sc...> wrote: > I would concur with that assessment. > > I also notice that for the most part people only look at the 512 sizes > of the images. If the previewmaker could create those as well then that > would solve a lot of the speed issues that I see. > > I do have though over probably 10k images at this point and that will > eat up a lot of disk space. The whole speed for space problem. > > -Scott __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com |
|
From: Ashley M. K. <as...@pc...> - 2002-12-31 20:24:07
|
Piet Barber wrote:
>I generally do a 'wget' to do the dirty work of generating those 512 byte wide
>images ahead of time, instead of when the visitor gets to the site.
>
This can easily be done with the same previewmaker.pl script, by the
way.
--
W | I haven't lost my mind; it's backed up on tape somewhere.
+--------------------------------------------------------------------
Ashley M. Kirchner <mailto:as...@pc...> . 303.442.6410 x130
IT Director / SysAdmin / WebSmith . 800.441.3873 x130
Photo Craft Laboratories, Inc. . 3550 Arapahoe Ave. #6
http://www.pcraft.com ..... . . . Boulder, CO 80303, U.S.A.
|
|
From: Scott S. <ss...@sc...> - 2002-12-31 20:26:18
|
For those of us that are coding challenged (me). How is that? -Scott On Tue, 2002-12-31 at 13:23, Ashley M. Kirchner wrote: > Piet Barber wrote: > > >I generally do a 'wget' to do the dirty work of generating those 512 byte wide > >images ahead of time, instead of when the visitor gets to the site. > > > This can easily be done with the same previewmaker.pl script, by the > way. > > -- > W | I haven't lost my mind; it's backed up on tape somewhere. > +-------------------------------------------------------------------- > Ashley M. Kirchner <mailto:as...@pc...> . 303.442.6410 x130 > IT Director / SysAdmin / WebSmith . 800.441.3873 x130 > Photo Craft Laboratories, Inc. . 3550 Arapahoe Ave. #6 > http://www.pcraft.com ..... . . . Boulder, CO 80303, U.S.A. > > > > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > IDS-devel mailing list > IDS...@li... > https://lists.sourceforge.net/lists/listinfo/ids-devel > -- Scott Sawyer o: 928 226 0404 Owner c: 602 920 0083 Scott Sawyer Photography http://www.scottsawyer.org Capturing today's moments, forever |
|
From: Ashley M. K. <as...@pc...> - 2002-12-31 20:33:48
|
Scott Sawyer wrote:
>For those of us that are coding challenged (me). How is that?
>
The default method is to generate only one preview, 100 pixels:
print "Beginning preview generation...\n";
foreach $file (@FilesToPreview) {
createDisplayImage($previewMaxDimension, '', $file);
print "Created preview for \"$file\"\n";
}
If you want the others, you can augment that. I wanted to have 512,
640 and 800 created, so my code looks like:
foreach $file (@FilesToPreview) {
createDisplayImage($previewMaxDimension, '', $file);
createDisplayImage(512, '', $file);
createDisplayImage(640, '', $file);
createDisplayImage(800, '', $file);
print "Created preview for \"$file\"\n";
}
Please note that I added an extra size on my site. I have both a
'tiny' and a 'small'.
--
W | I haven't lost my mind; it's backed up on tape somewhere.
+--------------------------------------------------------------------
Ashley M. Kirchner <mailto:as...@pc...> . 303.442.6410 x130
IT Director / SysAdmin / WebSmith . 800.441.3873 x130
Photo Craft Laboratories, Inc. . 3550 Arapahoe Ave. #6
http://www.pcraft.com ..... . . . Boulder, CO 80303, U.S.A.
|
|
From: Anthony A. D. T. <aa...@ve...> - 2003-01-01 12:17:20
|
>I generally do a 'wget' to do the dirty work of generating those 512 byte >wide images ahead of time, instead of when the visitor gets to the site. I do this too, and have been amazed that wget actually seems to navigate the whole site. I've had miserable experience trying to get it to leech arbitrary web sites - it tends to arbitrarily not follow some links, despite being instructed to recurse. |
|
From: Kevin P. <pfe...@iu...> - 2002-12-31 20:29:54
|
Scott Sawyer writes: > I also notice that for the most part people only look at the 512 sizes > of the images. If the previewmaker could create those as well then that > would solve a lot of the speed issues that I see. Ashley mentioned doing this; it's quite easy. Previewmaker gets its max preview size from the ids.config. Just to try this I made a copy of both, adding -512 to the names, removing most config items from the ids-512.config file and of course changing "previewMaxDimension" to 512. This works fine. I think it's just as easy though to modify the loop in a copy of previewmaker (the one that runs at 3:00 am) to do both image sizes at the same time. Maybe Ashley already has this handy. BTW, the site he mentioned is very fast; I think my problem is the platform I'm using. It cost almost nothing and new machines are cheap these days, so I'll try a faster server before "mucking about" further. :-) -K > I do have though over probably 10k images at this point and that will > eat up a lot of disk space. The whole speed for space problem. -- Kevin Pfeiffer <pfe...@iu...> - www.iu-bremen.de Online Editorial Services - International University Bremen "The rule still applies. However, this rule will not be enforced." |
|
From: mike b. <wm...@ar...> - 2002-12-31 20:03:29
|
the tiny patch at the bottom of this mail significantly speeds up album page generation for IDS 0.8 and 0.81, and looks like it would work for 0.82 as well. the problem is that fillDateHash is called in both generateAlbum and sortItems, but it really only needs to be called in sortItems. further, generateAlbum always calls fillDateHash regardless of sort type, whereas sortItems is smarter and only calls fillDateHash when sorting by timestamp. i caught this using the DProf perl profiler, which showed IDS was spending most of its time in Image::TIFF getting timestamps from photos, even when not sorting by timestamp. time trials for before and after the patch is applied, using the default intelligent sort method : rezrov ~/public_html/reality % dprofpp albumpage.dpp Total Elapsed Time = 4.123205 Seconds User+System Time = 4.013175 Seconds rezrov ~/public_html/reality % dprofpp albumpage-optimized.dpp Total Elapsed Time = 1.486758 Seconds User+System Time = 1.486758 Seconds these were taken on an old AMD-K6 300mhz machine, so the improvement is somewhat magnified. it's still worthwhile on a faster machine, however. sorting by timestamp speed is improved as well, since fillDateHash was being called twice per file when sorting by timestamp previously, and now is only called once. apologies if this has already been brought up - i poked through the mailing list archives and piet's patches website and didn't spot anything that looked similar. i had submitted this directly to mo...@mu... about a year ago and had never heard anything back - i imagine it got lost in the shuffle someplace. ---mike biesele http://real.ity.org/ rezrov ~/ids-0.81 % diff -c index.cgi index-opt.cgi *** index.cgi Fri Oct 5 17:15:28 2001 --- index-opt.cgi Sun Dec 16 23:56:22 2001 *************** *** 379,388 **** $imagecounter = 0; my($startcounter) = 0; my ($commentedItems) = 'n'; ! ! foreach my $fileName (@itemsToDisplay) { ! fillDateHash($fileName); ! } @itemsToDisplay = sortItems(@itemsToDisplay); # generate album HTML --- 379,390 ---- $imagecounter = 0; my($startcounter) = 0; my ($commentedItems) = 'n'; ! ! # redundant - fillDateHash is called in sortItems when necessary already ! # ! # foreach my $fileName (@itemsToDisplay) { ! # fillDateHash($fileName); ! # } @itemsToDisplay = sortItems(@itemsToDisplay); # generate album HTML > > I think the main "lack of speed" most people are experiencing in IDS is >the amount of time it takes ImageMagick to create the thumbs and the various >image sizes. The speed at which this operation occurs can only be improved >by a faster CPU and possibly a very small amount by a faster hard drive. > FastCGI and modperl are used when a site is expecting a tremendous >amount of traffic, perhaps on the order of one dynamic page requested per >second or higher. That is the point when the script and interpreter startup >overhead starts to load the server down. Implementing FastCGI and/or >modperl support doesn't make any noteable speed increase to a single user. >It just prevents the system from buckling when you get 1,000 visitors / >20,000 page views an hour. > > >Jeff > > > > > >------------------------------------------------------- >This sf.net email is sponsored by:ThinkGeek >Welcome to geek heaven. >http://thinkgeek.com/sf >_______________________________________________ >IDS-devel mailing list >IDS...@li... >https://lists.sourceforge.net/lists/listinfo/ids-devel > |
|
From: happyman <hap...@co...> - 2003-01-02 00:32:40
|
Hi all,
I am using ids+fastcgi. The most importnat thing is fastcgi is a =
single process,
css2 happyman/public_html> ps ax |grep ids
32158 ? S 32:04 /usr/bin/suidperl -U =
/dev/fd/3//home/happyman/public_html/ids/index.fcgi
I made ids runing as its owner using suidperl, so the thumbnails =
too. I don't really care about thumbnail creation time.
It handles very well on my site with 2.7G photos (including =
image-cache). You can link your image-cache dir to
tmpfs (shared-memory filesystem) or install a reverse proxy to increase =
speed. (I did)]
visit my pages on http://photos.happyman.idv.tw/
regs,
Eric Chiu
----- Original Message -----=20
From: "Jeff Dafoe" <je...@ba...>
To: <ids...@li...>
Sent: Wednesday, January 01, 2003 2:48 AM
Subject: [Ids-devel] IDS speed and modperl/fcgi
I think the main "lack of speed" most people are experiencing in IDS =
is
the amount of time it takes ImageMagick to create the thumbs and the =
various
image sizes. The speed at which this operation occurs can only be =
improved
by a faster CPU and possibly a very small amount by a faster hard drive.
FastCGI and modperl are used when a site is expecting a tremendous
amount of traffic, perhaps on the order of one dynamic page requested =
per
second or higher. That is the point when the script and interpreter =
startup
overhead starts to load the server down. Implementing FastCGI and/or
modperl support doesn't make any noteable speed increase to a single =
user.
It just prevents the system from buckling when you get 1,000 visitors /
20,000 page views an hour.
Jeff
-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
IDS-devel mailing list
IDS...@li...
https://lists.sourceforge.net/lists/listinfo/ids-devel
|
|
From: Jeff D. <je...@ba...> - 2003-01-02 02:29:37
|
> I am using ids+fastcgi. The most importnat thing is fastcgi is a
single process,
It's a single process as long as your concurrency is a single request.
It's identical to regular CGI in that regards. Unless you modified IDS to
be multithreaded.
Jeff
|
|
From: happyman <hap...@co...> - 2003-01-02 04:02:49
|
Yes. Thanx for Jeff's correction..
Only dynamic fastcgi server process is single.=20
however, we can speed ids up a little by share some program blocks.
My patch is located:
http://210.59.224.194/~happyman/ids-fastcgi-big5.patch
Most of my efforts is make IDS handle Chinese BIG5 correctly.=20
Big5 is double-byte and ranges as below:
my $BIG5 =3D '[\xa1-\xf9](?:[\x40-\x7e]|[\xa1-\xfe])';
the 2nd byte contains special chars. like '_' '\' (have problem with =
regular expression)
BTW, The dirty big5 search and match function in idsShared.pm is powered =
by my school bro. Sywang. :)
Enjoy it!
Eric Chiu
----- Original Message -----=20
From: "Jeff Dafoe" <je...@ba...>
To: <ids...@li...>
Sent: Thursday, January 02, 2003 10:24 AM
Subject: Re: [Ids-devel] IDS speed and modperl/fcgi
> I am using ids+fastcgi. The most importnat thing is fastcgi is a
single process,
It's a single process as long as your concurrency is a single =
request.
It's identical to regular CGI in that regards. Unless you modified IDS =
to
be multithreaded.
Jeff
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X