From: Andy S. <an...@gm...> - 2008-07-30 21:21:36
|
On Wed, Jul 30, 2008 at 10:47 PM, Alec Myers <al...@al...> wrote: > > With due respect to all, the file count of an installation is also an > important metric for many hosting providers. I'm now looking at a Gallery > installation with 16k images, and 164,000 files in the data directory, > which > seems totally crazy! Moving to G2.3 will reduce that by 32,000, and > reducing > the number of -fast files as suggested will remove roughly another 24,000. > I'm happy to work on that and see where it goes, but if Valiant or anyone > else rules that the architecture isn't acceptable there's no point me > spending time on it. Just checking if we count the same number of files: 3 files per derivative or data item (entity cache + fast download + data file) - It doesn't make sense to not have separate data files. So that's a given. - We still need a fast download file. It could be an aggregate for multiple entities though. I don't see how an aggregate would improve performance though. And if you do an aggregate, you'll have fun with locking when writing to that aggregate. So it might impair performance. - The entity cache file is disputed. An aggregate file could improve things. If the aggregate isn't item or album oriented, but decimal subtree oriented, then I don't see how it could improve performance either. I'd be interested in experiments where we just drop the entity file cache. Either way, I'm all for improvements. I just want to emphasize that the solutions should introduce a lot of complexity and that the improvements should be clearly measurable and reproducible on different platforms. - Andy > > > Also worth considering: allow an entity type to consider whether it should > be cached or not. Entities which are loaded rarely don't need extra disc > space in the cache. > > ------ Original Message ----- > From: "Andy Staudacher" <an...@gm...> > To: "Jesse Mullan" <jm...@vi...> > Cc: <gal...@li...> > Sent: Wednesday, July 30, 2008 9:08 PM > Subject: Re: [Gallery-devel] How many files does gallery create per > photo!!? > > > On Wed, Jul 30, 2008 at 12:30 PM, Jesse Mullan <jm...@vi...> wrote: > > > (dang it, this went just to any by accident) > > > > On Wed, Jul 30, 2008 at 2:30 PM, Jesse Mullan <jm...@vi...> wrote: > > > That's 2 queries plus a LOT of complex joins followed by various PHP > > > manipulations of the data, right? If one indexed query doesn't yield > > > much speedup, then why would there be speedup for file-based caching? > > > > Joins based on the integer primary key column are complex? It's more > expensive than select from a single table on PK, but I think that our ORM > isn't that heavy when just loading entities. And the PHP code that follows > isn't heavy either. > > Anyhow, that's just a detail. I think that this specific caching backend > "new cache table with single column for serialized entity object replacing > GalleryStorage::loadEntities()" is not where you can get the biggest bang > for the buck. I'd be happy to get convinced by numbers. :) > > We should focus on removing the disk based entity cache for > loadEntitiesById() calls with more than 1 id. Not sure if we can remove the > disk baed entity cache completely. > We need to focus on adding memcached/APC support. > We need to find out why our page cache isn't that great yet (it's pretty > heavy for some users). > We might have to kill the session based permission cache. > If you want to work on the DB, the imageblock code is still a bit heavy. Or > the itemAttributesMap (view counting) could benefit from changes. > > - Andy > > > > > > > > > > On Wed, Jul 30, 2008 at 2:07 PM, Andy Staudacher <an...@gm...> > > wrote: > > >> @per entity db cache: > > >> An entity can be fetched with 2 queries. > > >> 1. get entity type from store. > > >> 2. single select with lots of table joins to get the whole data set > for > > a > > >> single entity. > > >> > > >> I doubt that this is an area where we need to spend time for > > optimizations. > > >> > > >> As for modularizing the cache layer and adding memcached/APC/... > > support: > > >> Sure, we all agree on that. > > > > > > > ------------------------------------------------------------------------- > > This SF.Net email is sponsored by the Moblin Your Move Developer's > > challenge > > Build the coolest Linux based applications with Moblin SDK & win great > > prizes > > Grand prize is a trip for two to an Open Source event anywhere in the > > world > > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > __[ g a l l e r y - d e v e l ]_________________________ > > > > [ list info/archive --> http://gallery.sf.net/lists.php ] > > [ gallery info/FAQ/download --> http://gallery.sf.net ] > > > > > > > > > -------------------------------------------------------------------------------- > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > > > > -------------------------------------------------------------------------------- > > > __[ g a l l e r y - d e v e l ]_________________________ > > [ list info/archive --> http://gallery.sf.net/lists.php ] > [ gallery info/FAQ/download --> http://gallery.sf.net ] > > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > __[ g a l l e r y - d e v e l ]_________________________ > > [ list info/archive --> http://gallery.sf.net/lists.php ] > [ gallery info/FAQ/download --> http://gallery.sf.net ] > > |