[Apc-aa-coders] Re: Caching - are you able to help a bit

[Honza M. <hon...@ec...>]

Hi Mitra,

  I'm sending it also in the list, since other developers could have
some suggestions, too.

On =DAt, 2003-03-04 at 22:25, Mitra wrote:
> At 3:38 PM +0100 4/3/03, Honza Malik wrote:
> >Hi Mitra,
> >
> >On =DAt, 2003-03-04 at 00:59, Mitra wrote:
> >>  Honza
> >>
> >>  I'm looking at some of the caching problems - are you able to help =
a
> >>  bit, with understanding things that seem strange to me?
> >>
> >>  I'm particularly looking at this with the site module.
> >>
> >>  The first odd thing i notice .....
> >>
> >>  Store gets called a lot which is fine, BUT lastClearTime is always =
0,
> >>  so  it calls purge every time,  that makes for 6 extra "DELETE FROM
> >>  pagecache WHERE stored < ..." calls on each of my site pages.
> >
> >You are right - the concept of lastClearTime is not good - you never
> >know the lastClearTime.
> >
> >>  It looks like this is because each call is with a "new PageCache",
> >>  some in GetView and the others in get_output_cached
> >>
> >>  I see two solutions to this
> >>  1: Easy: Delete the purge from the store, really purge should be
> >>  happening at the retrieval side, not at the storing!
> >>
> >>  2: Remove the "new PageCache" from everywhere, do it once and keep =
it
> >>  in a global variable $GLOBALS[pagecache]
> >
> >I think it does not help. If one script deletes the cache and do not
> >store the time to some table, noone can know, when it was. We can stor=
e
> >it into log table.
>=20
> This might be ideal, but the problem is much worse than this - at the=20
> moment purge is called MANY times by ONE script, which means we could=20
> remember it within a script.

If we will purge() cache from special script, then no calling in other
scripts is needed. If we won't use special script, you are right.

> >The best solution, I think, is to call purge() from AA cron. The cache
> >was written in time, then AA cron wasn't in AA, so there was no such
> >possibility. Now we can move it here.
> >
> >I think it should not be problem. Anyway. I plan to write script
> >db_maintain.php3 which will be runed each 5 minutes from AA cron. Ther=
e
> >I want to move managing count Hits (Item display count). Marek found
> >some inefficiecy in count hit - if a user searches in AA, noone can vi=
ew
> >fulltext!!! - it waits until the searching is done and after that it
> >updates display_count in item table. After that the fulltext is shown.
> >It is quite bad.
>=20
> I don't think its a good idea to rely on Cron, it requires access to=20
> the system that many cases won't have - I find this when I'm doing=20
> work for some organization who are hosting their site on some ISP,=20
> not on their own machine. Often this hosting is a gift by the ISP to=20
> the organization, so they don't always give it top priority.

That's right but I do not see better solution. What about to create
special service (say at Econnect or SourceForge) which will create HTTP
request to cron.php3 on remote AA installs for such organizations (each
5 minutes)?

(I'm sure there must be such service on the web already.)

> I find the search/fulltext issue quite surprising, can you say more=20
> about where this lock happens - is it in MySQL or in PHP ?  And what=20
> has this got to do with Counting Hits?  Or do you mean that it is=20
> because the fulltext view is trying to update the table, and can't do=20
> this until the Search finishes?

Yes, exactly. It is MySQL issue. If MySQL do the SELECT from item joined
with content table, the tables are locked (at least in MySQL <4) and all
UPDATEs must wait until the SELECT is executed.

> >We can fix it, if we write the hit to another table. We plan to use lo=
g
> >table, where we log the hit and then we run db_maintain.php3 from cron.
> >The script will count all loged hits and then update database
> >(item.display_count). There is no problem, if the script will call als=
o
> >purge() for cache. The purge events could be written into log database
> >as well, so we will know, when last purge() was run.
>=20
> Another solution MIGHT be to do the hitcount update AFTER all page=20
> output has been done, that way we don't care if the script waits a=20
> second or two?

No way. We can't use it with shtml files, because SSI includes MUST wait
after whole script is executed. After then it can continue display next
part of shtml file.

Also, MySQL do not have any UPDATE DELAYED command. It has only REPLACE
DELAYED, which we can't use, because the autoincremented short_id is
changed on each REPLACE.

> >  > What do you think - I think I prefer version 2, although its more
> >>  work ... if we did it in just the retrieval side (i./e. ignoring it
> >>  in /admin/) it would fix most of the inefficiencies.
> >>
> >>  The next thing I notice is that even if nocache=3D1 it is still sto=
ring
> >>  stuff in the cache (again from get_output_cached and GetView)
> >
> >This is designed behavior - mostly you want just renew chache content
> >and show it. Maybe just the name 'nocache' is not good.
>=20
> Ok - this is fine, I thought nocache meant not using the cache at all.
>
> - Mitra

   Honza