From: Jesse M. <jm...@vi...> - 2008-07-30 18:37:52
|
I hacked together APC-based caching (which still does all filesystem writes but doesn't read from them unless there is an APC miss) and it is pretty fast. It seems to me that we could modularize the caching mechanism such that a user could run with no caching per-entity file per-entity APC per-entity memcache per-entity db caching (a fast single select for a serialized hunk of data might be faster than a half dozen queries and associated logic to build a complete object from the raw data store) aggregate file caching etc. Some of this might be abetted PHP5 magic like the __autoload handlers (instead of storing the list of required classes AND the serialized data) On Wed, Jul 30, 2008 at 1:00 PM, Alec Myers <al...@al...> wrote: > I did some quick testing, based on a single album-page with 48 public... > ----- Original Message ----- > From: "Andy Staudacher" <an...@gm...> > A single cache file per album, or per decimal subtree as you suggest might > work. Maybe it needs more work for very large galleries where the subtree > (or album) gets too large. > > But do you really want to think about this level of optimization? > Ideally, we want a simple solution that is also fast. As for all > optimizations, every aspect that complicates the solution should be > justified with rigorous benchmarking. > > I'd really like to see improvements in the caching department. But beyond > trying to get rid of the disk based entity cache it's all rather blury and > requires a lot of work and discussion. Not sure if it's worth it. If it's > something that you'd like to work on, please do! If it's just one of many > things you'd like to improve, I would rather focus on one of them. > > Also, there's this task noone tackled yet: Providing benchmarking > methodologies and tools for G2. It would make specific optimization tasks > much easier if we had that. > > - Andy > > On Wed, Jul 30, 2008 at 3:16 PM, Alec Myers <al...@al...> wrote: > >> I don't particularly want extra options either. Removing entity cacheing >> would take away another three files (in my example) leaving five per >> image. >> That would be a great improvement. >> >> Fast-download: you could reduce the inode overhead by 99% by having just >> one -fast file per cache/entity/x/x *directory* - "switch ($itemId) >> {case:...}" and "if (GalleryFastDownload($itemId) {...}". Keeping such a >> file up to date would be only marginally more complex than at present. >> Executing it would be only marginally slower. It might even be faster >> since >> consecutive items in albums are likely to have near consecutive numbering, >> and the server file-storage may cache the file for subsequent reads. Is >> that >> a really silly idea? >> >> >> >> >> ----- Original Message ----- >> From: "Andy Staudacher" <an...@gm...> >> To: "Alec Myers" <al...@al...> >> Cc: <gal...@li...> >> Sent: Wednesday, July 30, 2008 1:52 PM >> Subject: Re: [Gallery-devel] How many files does gallery create per >> photo!!? >> >> >> Lots of files indeed. And we had previous discussions about the merit of >> the >> caching. >> - We need to keep the -fast caches to ensure that core.DownloadItem >> requests >> have low overhead. >> - Entity cache on disk often doesn't make much sense. We'll probably get >> rid >> of it. >> >> Having options about enabling specific disk caching, I'm rather -1. G2 >> needs >> to get simpler, not more complex. >> >> - Andy >> >> On Wed, Jul 30, 2008 at 12:59 PM, Alec Myers <al...@al...> wrote: >> >> > Hi Guys, >> > >> > I did some digging today in response to a forum thread and came up with >> > the >> > following info about how many files (and hence inodes) Gallery uses. It >> > works out like this (you probably all know this, but I was really >> > surprised): >> > >> > -One for the original image. >> > -Two for the cached version of each derivative -thumbnail, resize, >> > watermark - reduced to one per derivative in G2.3 >> > Additionally one or two extra per image or derivative for cached entity >> > information. >> > >> > In my case for every single photo uploaded with a watermark (and no >> > resizes) >> > there are a total of 10 files! For example for one photo id 56919 >> > (watermarked derivative 56920, thumbnail 56997) there are the following >> > files: >> > >> > -The original jpg >> > >> > in cache/entity/5/6/: >> > >> > 56919.inc >> > 56920.inc >> > 56920-fast.inc >> > 56997.inc >> > 56997-fast.inc >> > >> > in cache/derivative/5/6: >> > >> > 56919.dat >> > 56919-meta.dat >> > 56997.dat >> > 56997-meta.dat >> > >> > That's a total of 10 files for each photo, the information from 7 of >> which >> > is just duplicating stuff in the db. (For G2.3 that would be 8 files, >> with >> > 5 >> > duplicating stuff in the db) so I'm wondering if it wouldn't be worth >> > including the option to switch-off entity-caching for those with limited >> > inodes? I realise that caching is important for performance, but if your >> > hosting provider pulls the site, that trumps slow loading. >> > >> > >> > >> > >> > >> > >> > >> > ------------------------------------------------------------------------- >> > This SF.Net email is sponsored by the Moblin Your Move Developer's >> > challenge >> > Build the coolest Linux based applications with Moblin SDK & win great >> > prizes >> > Grand prize is a trip for two to an Open Source event anywhere in the >> > world >> > http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> > __[ g a l l e r y - d e v e l ]_________________________ >> > >> > [ list info/archive --> http://gallery.sf.net/lists.php ] >> > [ gallery info/FAQ/download --> http://gallery.sf.net ] >> > >> > >> >> >> >> ------------------------------------------------------------------------- >> This SF.Net email is sponsored by the Moblin Your Move Developer's >> challenge >> Build the coolest Linux based applications with Moblin SDK & win great >> prizes >> Grand prize is a trip for two to an Open Source event anywhere in the >> world >> http://moblin-contest.org/redirect.php?banner_id=100&url=/ >> __[ g a l l e r y - d e v e l ]_________________________ >> >> [ list info/archive --> http://gallery.sf.net/lists.php ] >> [ gallery info/FAQ/download --> http://gallery.sf.net ] >> >> > > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > __[ g a l l e r y - d e v e l ]_________________________ > > [ list info/archive --> http://gallery.sf.net/lists.php ] > [ gallery info/FAQ/download --> http://gallery.sf.net ] > > |