[Exist-open] R: R: delete item performance issue

[Salvo N. <sa...@ti...>]

Michael wrote:

>I think you may not have appreciated the extent to which indexation and
storage are tighly=20
>meshed in eXist's design. You can't just store/delete by day then index =
in
the dead of night as=20
>you conceivably could with some other systems which are wrapped around =
the
filestore.=20

OK, I try to think to an explicit partition algorithm.

Anyway, excuse me, but my english is a little poor...

I have not understand why you say "You can't just store/delete by day =
then
index in the dead of night"... Is a configuration limitation issue (can =
I
stop index process) or you think to another issue ? What do you mean =
"you
conceivably could with some other systems which are wrapped around the
filestore" ?=20

Tnx for your time...

Salvo



-----Messaggio originale-----
Da: exi...@li...
[mailto:exi...@li...] Per conto di Michael
Beddow
Inviato: venerd=EC 20 luglio 2007 12.57
A: exi...@li...
Oggetto: Re: [Exist-open] R: delete item performance issue

Salvo wrote:

> I'm sure we have no time to reorganize book collection in sub=20
> collection because I'm involved in developing new operational function =

> and I have no enough time.

Well, you know your own data, but I find it hard to see what would be so
time-consuming about devising and implementing a partitioning scheme.
Assuming that you keep the name of your current all-inclusive collection =
as
your parent collection and place all your partitions below it, there =
would
be no need to alter any of your querying code. All you need is an =
algorithm
to determine what documents go in what subcollection and a component in =
your
store and delete routines that encapsulates that algorithm so that =
documents
go to and are deleted from the correct partition. Then you do a one-off
batch run to split your current big collection into the required =
partitions.
Your partitioning criterion need not be any aspect of the data that =
clients
or users are concerned about, so there's no need to consult them.

Of course, designing a *generic* on-the-fly partitioning method that =
will
fit all the highly heterogeneous data structures and contents that eXist =
has
to cope with is a vastly more complex task, as Wolfgang's response makes
plain. But a local modification to transparently partition well-known =
local
data is far more manageable. I would certainly think it preferable in =
every
respect to trying to interfere with eXist's current =
index-on-store/reindex
on delete behaviour, which is what you seem to be thinking of as an
alternative.

I think you may not have appreciated the extent to which indexation and
storage are tighly meshed in eXist's design. You can't just store/delete =
by
day then index in the dead of night as you conceivably could with some =
other
systems which are wrapped around the filestore.

Michael Beddow



-------------------------------------------------------------------------=

This SF.net email is sponsored by: Microsoft Defy all challenges.
Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Exist-open mailing list
Exi...@li...
https://lists.sourceforge.net/lists/listinfo/exist-open