[cweb-CVS] bigdata perf.txt,NONE,1.1 .cvsignore,1.10,1.11

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Update of /cvsroot/cweb/bigdata
In directory sc8-pr-cvs4.sourceforge.net:/tmp/cvs-serv15000

Modified Files:
	.cvsignore 
Added Files:
	perf.txt 
Log Message:
Updated the UML model and added a ZIP containing an HTML presentation of the model.
Working on partitioned index support.

Index: .cvsignore
===================================================================
RCS file: /cvsroot/cweb/bigdata/.cvsignore,v
retrieving revision 1.10
retrieving revision 1.11
diff -C2 -d -r1.10 -r1.11
*** .cvsignore	9 Feb 2007 18:56:59 -0000	1.10
--- .cvsignore	8 Mar 2007 18:14:05 -0000	1.11
***************
*** 10,11 ****
--- 10,12 ----
  *.tmp
  test_*
+ com.bigdata.journal.StressTestConcurrent.comparison.csv

--- NEW FILE: perf.txt ---

1. Transactions per second target of 8k.  Implement TCP A and C.
   Implement group commit.  See
   http://blogs.oracle.com/charleslamb/2006/09/22#a14.  Is the disk
   write cache enabled or disabled?  Are transactions really durable?
   http://www.soliddata.com/pdf/TimesTen.pdf discusses write-through
   RAID vs write-behind RAID vs solid state for log files.  Since
   journals are size limited, a very small (.5 - 1G) solid state drive
   would sufficient.  Old journals are just transferred to normal
   disk.  Note that solid state drives do eventually degrade with use
   and the journal will cause repeated overwrites of data on the solid
   state disk.

   Consider and minimize startup costs for an index.  When the index
   is isolated by a transaction and we are seeking high TPS rates,
   then those startup and run costs need to be optimized to reduce
   startup latency and also run latency for short transactions.

MROW performance test: 2/20/07.

Note that AIO support is not implemented.  The performance is CPU
bound for all cases except Disk only.  However, note that this is with
concurrent clients.  When using a single client, direct mode
performance might be IO bound with a single client since the Direct
mode does not use AIO.  The disk queue length is short except on
sustained writes (pre-writes before the readers start).

Significantly less performance is obtained when using a single client.

Transient(heap)  : #clients=20, ntrials=100000, nok=39691, ncancelled=60309, nerrors=0 in 10094ms (3932 reads per second); nwritten=2728
Transient(direct): #clients=20, ntrials=100000, nok=41027, ncancelled=58973, nerrors=0 in 10297ms (3984 reads per second); nwritten=2739
Direct(direct)   : #clients=20, ntrials=100000, nok=38776, ncancelled=61224, nerrors=0 in 10047ms (3859 reads per second); nwritten=2716
Direct(heap)     : #clients=20, ntrials=100000, nok=39030, ncancelled=60970, nerrors=0 in 10063ms (3878 reads per second); nwritten=2739
Mapped           : #clients=20, ntrials=100000, nok=38801, ncancelled=61199, nerrors=0 in 10094ms (3843 reads per second); nwritten=2731
Disk		 : #clients=20, ntrials=100000, nok=12671, ncancelled=87329, nerrors=0 in 10047ms (1261 reads per second); nwritten=2784

Note: Default VM args using JRockit Jrockit R26.4.0 (jdk1.5.0_06)

--------------------

Concurrent client transaction stress test: 2/21/07

JVM: Jrockit R26.4.0 (jdk1.5.0_06)

VM ARGS: -server -Xms1g -Xmx1g -XX:MaxDirectMemorySize=256M

Test at clients in { 1, 2, 10, 20, 100, 200 }.

mode=Transient, #clients=1, nops=100, ntx=10000, ncomitted=3989, naborted=444, nfailed=0, nuncommitted=6011 in 30219ms (132 tps)
mode=Transient, #clients=2, nops=100, ntx=10000, ncomitted=4244, naborted=457, nfailed=12, nuncommitted=5744 in 30000ms (141 tps)
mode=Transient, #clients=10, nops=100, ntx=10000, ncomitted=4316, naborted=483, nfailed=11, nuncommitted=5673 in 30032ms (143 tps)
mode=Transient, #clients=20, nops=100, ntx=10000, ncomitted=4344, naborted=506, nfailed=11, nuncommitted=5645 in 30015ms (144 tps)
mode=Transient, #clients=100, nops=100, ntx=10000, ncomitted=4060, naborted=486, nfailed=11, nuncommitted=5929 in 30016ms (135 tps)
mode=Transient, #clients=200, nops=100, ntx=10000, ncomitted=4044, naborted=494, nfailed=13, nuncommitted=5943 in 33390ms (121 tps)

mode=Direct, #clients=1, nops=100, ntx=10000, ncomitted=601, naborted=70, nfailed=0, nuncommitted=9399 in 30000ms (20 tps)
mode=Direct, #clients=2, nops=100, ntx=10000, ncomitted=628, naborted=64, nfailed=0, nuncommitted=9372 in 30000ms (20 tps)
mode=Direct, #clients=10, nops=100, ntx=10000, ncomitted=633, naborted=81, nfailed=1, nuncommitted=9366 in 30000ms (21 tps)
mode=Direct, #clients=20, nops=100, ntx=10000, ncomitted=633, naborted=77, nfailed=1, nuncommitted=9366 in 30031ms (21 tps)
mode=Direct, #clients=100, nops=100, ntx=10000, ncomitted=639, naborted=91, nfailed=0, nuncommitted=9361 in 30016ms (21 tps)
mode=Direct, #clients=200, nops=100, ntx=10000, ncomitted=643, naborted=103, nfailed=0, nuncommitted=9357 in 30015ms (21 tps)

mode=Disk, #clients=1, nops=100, ntx=10000, ncomitted=604, naborted=59, nfailed=0, nuncommitted=9396 in 30015ms (20 tps)
mode=Disk, #clients=2, nops=100, ntx=10000, ncomitted=586, naborted=62, nfailed=0, nuncommitted=9414 in 30000ms (19 tps)
mode=Disk, #clients=10, nops=100, ntx=10000, ncomitted=660, naborted=89, nfailed=0, nuncommitted=9340 in 30000ms (22 tps)
mode=Disk, #clients=20, nops=100, ntx=10000, ncomitted=629, naborted=67, nfailed=0, nuncommitted=9371 in 30016ms (20 tps)
mode=Disk, #clients=100, nops=100, ntx=10000, ncomitted=665, naborted=106, nfailed=0, nuncommitted=9335 in 30016ms (22 tps)
mode=Disk, #clients=200, nops=100, ntx=10000, ncomitted=623, naborted=83, nfailed=0, nuncommitted=9377 in 30000ms (20 tps)

w/ Options.FORCE := No

mode=Direct, #clients=10, nops=100, ntx=10000, ncomitted=3337, naborted=350, nfailed=3, nuncommitted=6660 in 30062ms (111 tps)
mode=Disk, #clients=200, nops=100, ntx=10000, ncomitted=3242, naborted=416, nfailed=7, nuncommitted=6751 in 30015ms (108 tps)

w/ Options.FORCE := Force (data, but not metadata)

mode=Disk, #clients=200, nops=100, ntx=10000, ncomitted=635, naborted=92, nfailed=1, nuncommitted=9364 in 30015ms (21 tps)

Note: Page faults per second is quite high for this stress test and
that needs to be looked at futher.  I was running MS Outlook as well
at the time that these measures were collected, so there might be
memory contention on the host (Dell laptop; Windows XP/Pro. 2G ram.)

Note: TPS is basically constant for a given combination of the buffer
mode and whether or not commits are forced to disk.  This means that
the #of clients is not a strong influence on performance.  The big
wins are Transient and Force := No since neither conditions synchs to
disk.  This suggests that the big win for TPS throughput is going to
be group commit.