Re: [Firebird-devel] AutoOptimize

[Vlad H. <hv...@us...>]

> >>>> I.e., if we mark some thread as low IO priority, we may delay IO requests from it 
> >>>
> >>>     How much delay ? This is a central point
> >>>
> >> As a first approach we may delay them as long as there are other active 
> >> disk requests.
> > 
> >     This is not good, imo. Such low-priority thread will always sleep if we have even 
> > small but regular IO load, i guess. 
> >  
> 
> Certainly. But as far as I understand, it must sleep in such cases. Or 
> there is no reason to make it low IO priority. 

    Ideally this threads must run with such priority to not harm other threads.
If we have IO subsystem which can handle, say, N IO requests per second,
and constant work load with M IO requests per second (M < N), we can safely
let background (or low IO priority) threads perform N - M IO requests per second.
Am i wrong ?
Problem is that we don't know N and M

> Certainly, we can invite 
> a lot - let single request pass after N high-priority requests provided 
> total requests number is less or equal to M. Having M and N settable 
> from firebird.conf:). Do we need such overcomplicated things?

    If it will work than yes we need it ;) But i doubt such algorithm will work well
 
...
> >> seems it will not break whole system throughput as much as bad plans do 
> > 
> >     It depends. Yes, our indices are very dense and we need smaller IO to read 
> > leaf level compared to other DBMS, but it is very CPU intensive. 
> > 
> 
> BTW, when we add (or delete) record to index block, it seems we can 
> easily detect, do we add unique or duplicated record? Is not this 
> information enough to adjust selectivity dynamically?

    selectivity = 1.0 / (all_keys - duplicate_keys)

how can you incrementally recalculate it ? And when do you store it on disk ?
 
> >> now. Anyway for future versions (remember prepared statements cache ?) 
> >> it will require more common solution.
> > 
> >     My current worries is cache_writer and garbage_collector threads
> > 
> 
> You've called thread "AutoOptimize" :)

    Me ? Not :) I just raised problem here because it is touched by another peoples ;)
 
> But what's a problem with this threads? 


    I want to allow background threads to do as much as they can do without affecting
worker threads much 

> Do you want to make them low IO priority? Not good idea, I'm afraid.

    Why ? Explain please 

Regards,
Vlad