Re: [SMW-devel] Query result caching and invalidation

[Jeroen De D. <jer...@gm...>]

Hey,

Not sure what "just store all properties" means. I was arguing for the
> opposite: not to store the properties again, since the printouts can easily
> be fetched from DB in the (relatively rare) cases that the parser cache
> needs to be rebuild. Mirroring all printout properties in the query cache
> would require more frequent updates to it and make it more specific to one
> single page.
>

Ok, seems like I misunderstood you then. You're saying to only cache which
objects match the query and none of their associated property values,
correct?

Don't think such rebuilds are rare - they happen whenever an edit is made,
whenever the cache of a page expires, and whenever we invalidate it because
the query results changed. But I agree this is less important then getting
the page invalidation right. Maybe we should initially just focus that and
not care to much on what we cache (or even not cache at all)?

> A property-based cache invalidation would kill most of the query caches
on almost every property edit

In situations as you described, I don't see how you can avoid this.
Assuming that those queries in templates will then typically also use a
property in at least one condition.

> Storing results for (sub)conditions as an exhaustive list could allow a
much more fine-grained control of cache invalidation. The challenge is to
keep these sets small. Maybe there are other approaches as well, such as
singling out certain "selective" subqueries for this purpose.

Yes, having that would be good.

One thing that just occurred to me is that in order to avoid caching things
only used once or twice that then potentially gets invalided a lot, not
benefiting much performance wise, is to keep a count of (sub) query usage
and only actually cache these that are used several/many times.

Cheers

--
Jeroen De Dauw
http://www.bn2vs.com
Don't panic. Don't be evil.
--