Re: [fetchmail-devel] 24h comparison of old and new UID handling code

[Matthias A. <mat...@gm...>]

Am 28.05.2010, 02:34 Uhr, schrieb Rainer Weikusat:

> The numbers contained in this mail come from two 24h 'observations' of
> the old and new UID handling code done via oprofile. They are based on
> querying 119 POP3 accounts. Some statistics about the .fetchids files
> (everything in lines):
>
> 	total:		35004
>         max:		8019
>         min:		1
>        average:	294.15
>         median:		46
>         std. dev:	823.87

D'oh. A std dev' that's thrice the average. There's a considerable chance  
for a negative number. :-}
The actual problem is that the model doesn't fit, you don't have a Normal  
distribution here, confirmed by the table I'm not quoting.

> UID handling is still the dominant operation (processor-time wise)
> performed by fetchmail.

No it's not, see below -- you're not grouping other code properly. If  
oprofile samples call stacks so that it can cumulate per caller (like  
du(1) does) you'll see it's not. Otherwise, we'll resort to the preclusion  
reasoning below.

> The relative amount of time is down from
> 93.46% to 34.36%. During the measurement interval, 1,540,732 event
> samples were collected for the old code and 63,092 for the new
> code: The trie/ array-based UID db has used about 4.09% of the CPU
> time the list-based one did use.

The relative times of the functions among themselves are hard to  
interpret, particularly if you don't group the other functions properly.

The total sample count is far more interesting, and there you're down from  
1.52 million samples (out of 1.65 million) to 0.07 million (out of 0.18  
million), where the other stuff remains more or less the same. IOW, there  
is a base amount of ~0.12 million samples NOT related to UID handling.  
Where the old code in your load scenario was more than 12x that of the  
other code (i. e. "dominated"), it's now roughly half the effort, which is  
no longer "dominating".

> I hope to be able to produce something like a final patch at some time
> late evening tomorrow.

No urge here, I'm not sure if I'll have the time to merge that stuff over  
the week-end. May well turn mid to end of next week before I get to it.

Thank you.

-- 
Matthias Andree