Re: [Bacula-devel] Alternative DB proposal

[Kern S. <ke...@si...>]

On Tuesday 14 October 2008 09:39:27 John Huttley wrote:
> So the modified version is actually a bit faster?

That is what I understood too, but I wanted to get a confirmation.

If it is indeed the case that the new case runs faster, it is indeed odd, and 
I would say the tester has fallen into a very trap that is very common in 
performance analysis.  

That is to get reliable test results on a memory caching machine such as 
Linux, you must run each test 10 times, and throw out the two with the times 
that differ the most from the average, then re-compute the average based on 
the remaining 8 samples which should not vary more than a few percent.

Another aid is to run one set of 10 tests, then the other set of 10 tests, 
then the first set of 10 tests again, and make sure the two runs of the first 
set generate the same results (using the methodology mentioned above).
If you want to have a better measure of disk times, you can sprinkle "sync" 
shell commands between each of the 10 test runs.

Regards,

Kern

>
> Thats odd.
>
> are you running  64Bit?
> Whats your Ram?
> Version of Mysql?
>
>
> I'll run it on my system also.
>
> Regards,
>
> john
>
> Yuri Timofeev wrote:
> > Hi
> >
> > 2008/10/7 John Huttley <Jo...@mi...>:
> >> Hi
> >> My first test are complete.
> >> To aid in getting all the questions/answers and results in one place,
> >> I've made a wiki entry.
> >>
> >> http://wiki.bacula.org/doku.php?id=wiki:playground
> >
> > My first test are complete. ;)
> >
> > Summary:
> >
> > MySQL, ProLiant DL180G5, CentOS 5.2
> >
> > 1. Batch inserts 10M unique! records in table 'File' (DB schema not
> > changed) : 39 hours (23+16), DB file size 15,4 Gb
> > 2. Batch inserts 10M unique! records in table 'File' (NEW DB schema
> > see also create_mysql_test2.sh) : 35 hours (19+16), DB file size 15,2
> > Gb
> >
> > Source code attached.
> >
> > Please note that the length of time (16 hours) insert records from the
> > table 'batch' to the table 'File' the same.
> > This indicates the stability of results.
> >
> >
> >
> >
> >
> > Full report:
> >
> > # ./test
> > Date: Oct 10 2008 11:27:37
> > Host: Localhost via UNIX socket
> > Use database: 'bacula_test'.
> > Temporary table 'batch' created.
> > Start filling tables 'batch'(temporary), 'Path', 'Filename' ...
> > End.
> > 10000000 records inserts.
> > *** CPUs time used : 375.880000
> > *** Elapsed time (wall clock) : 85334.000000
> >
> > Start inserts from temporary table 'batch' to 'File' ...
> > End inserts to 'File' ...
> > *** CPUs time used : 0.000000
> > *** Elapsed time (wall clock) : 58838.000000
> >
> >
> >
> >
> > # ./test2
> > Date: Oct 10 2008 11:27:37
> > Host: Localhost via UNIX socket
> > Use database: 'bacula_test2'.
> > Temporary table 'batch' created.
> > Start filling tables 'batch'(temporary), 'Path', 'Filename' ...
> > End.
> > 10000000 records inserts.
> > *** CPUs time used : 384.650000
> > *** Elapsed time (wall clock) : 71670.000000
> >
> > Start inserts from temporary table 'batch' to 'File' ...
> > End inserts to 'File' ...
> > *** CPUs time used : 0.000000
> > *** Elapsed time (wall clock) : 58564.000000
> >
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------------
> >
> > -------------------------------------------------------------------------
> > This SF.Net email is sponsored by the Moblin Your Move Developer's
> > challenge Build the coolest Linux based applications with Moblin SDK &
> > win great prizes Grand prize is a trip for two to an Open Source event
> > anywhere in the world
> > http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Bacula-devel mailing list
> > Bac...@li...
> > https://lists.sourceforge.net/lists/listinfo/bacula-devel