Re: [Iamb-dev-internal] streed

[Ken R. <kr...@qx...>]

Rich Bowen wrote:
> 
> Matt Cashner wrote:
> >
> > the decision i've made for the internals of the stree daemon is to store
> > the strees in an internal hash ref.

A hash ref? A scalar? Do you mean that you would create a hash,
and both the hash and a ref to it would get stored? Could you
re-create the hash from the database from just the ref?

Something is really not clicking here.

> > then every n seconds a snapshot of
> > that internal queue will be taken and written to the
> > database. modification of the hash should be fast enough to avoid data
> > collisions (extensive testing will verify this). i'm probably going to
> > need another process to farm off the database saving too. DBI blocks IO
> > and as such, while DBI is doing a insert or whatever, the whole server has
> > to sit and spin until the IO is done. very silly and very nasty. and if we
> > plan on this being used in a high traffic situation, that sort of blocking
> > could be very ugly indeed. any thoughts folks?

Clearly, it depends on two things: the value of n and the length
of time the data is blocked. We'll probably have to do some fine
tuning in real life.

Since the value of n is a function of the stability of the server
(and the platform), maybe we should say in the docs that the
default value of n needs to be divided by 10 for Win*. :-)

> Is this DBI's fault, or the particular DBD that we happen to be using?
> Would another database alleviate this problem? (I seem to recall that
> this is DBI, but I was not sure.)

More specifically, is it the MySQL DBI, and not another one
that we could possibly use? Are there any serious alternatives:
free, stable, easily obtained, and standardized?

> But, yes, it would be pretty cool to have a dbi cache thingy that you
> send data to, and it queues up inserts/updates for later processing.
> This could be a very cool thing just of itself - that is, completely
> apart from IAMB.

I also am interested to see how this works out. Glad to see that
you are back at this!

-- Ken