Re: [Firebird-devel] AutoOptimize

[Vlad H. <hv...@us...>]

> Vlad Horsun wrote:
> > ....
> >     Ideally this threads must run with such priority to not harm other threads.
> > If we have IO subsystem which can handle, say, N IO requests per second,
> > and constant work load with M IO requests per second (M < N), we can safely
> > let background (or low IO priority) threads perform N - M IO requests per second.
> > Am i wrong ?
> > Problem is that we don't know N and M
> >
> >   
> >> Certainly, we can invite 
> >> a lot - let single request pass after N high-priority requests provided 
> >> total requests number is less or equal to M. Having M and N settable 
> >> from firebird.conf:). Do we need such overcomplicated things?
> >>     
> >
> >     If it will work than yes we need it ;) But i doubt such algorithm will work well
> >  
> > ...
> >   
> Can you explain please why you think that this will not work well? 

    Because each IO requests have different cost. More - the same IO request have
one cost at one time and another cost at another time. Operating with only requests
amount is not enough and give no control. 

    Example : we have need to read page number 1 and write page number 100. If we 
first read and then write we have this operations in physical order. If we do write first 
then we have backorder and this two operations will run much slower. Also we can't
predict OS or HW caching effects. Again, because of careful writes we don't know how
may pages we will write, writing some particular page (we must write dependent 
pages first). Therefore we need something more real than just a numbers of an queued 
requests to rule threads priority. IMO, of course

> Because similar algorithms (ie. prioritized queues) are the basis on 
> today networks. IMHO, the scope is the same prioritizing/managing access 
> to a slow resource. For ex. see 
> http://en.wikipedia.org/wiki/Quality_of_service as a start.
> 
> Also, IMHO, is very good to have a look at 
> http://en.wikipedia.org/wiki/Background_Intelligent_Transfer_Service - 
> we need something like this, but for disks, right?

    No. I see nothng common (except of common words 'queue', 'priority', etc)

> As an aside it seems a little bit odd that while we have a resource 
> hungry feature triggering in the way in which GC triggers 

    What do you mean under "way in which GC triggers" ? Do you talk
about auto-sweep or about background GC ?

> which can 
> reach to the point of exhausting server memory (see the thread 'Firebird 
> restarting' from the support list) 

    I participated in that thread and i'm sure - GC itself have nothing common 
with exhausting server memory.

> and we are a little bit 'scared' about statistics auto updating. 

    Who are 'scared' ? We already discussed with Dmitry needs of auto-update
statistics (years ago) and have the same conclusion as i wrote in prior letter. 
In FB2 we don't need it much, period. There are only way to be in trouble after 
migrating to FB2 - not perform recommended backup\restore circle but create 
new database from script and pump it with data from old database. If people so 
smart to do such non-obvious step than they must be smart enough to update 
statistics. This is mine POV

Regards,
Vlad