Re: [Postgres-xc-general] Performance when adding new datanode

[Bartłomiej W. <bar...@tu...>]

How many clients/threads/processes did you have in the test? Just one? 
It seems like it based on that not many cores were being used.

Just one client:  psql -d test < insert.sql (100000 inserts) 
sequentially with psql -d test < select.sql (100000 selects)

Was the test being run on the same server as the database?

Yes, it was.

Also, all of the "nodes" are sharing the same storage device, not 
dedicated for each?

All of the "nodes" are local. Storage device the same but it is very 
large throughput  ~Gbit/s

For a better test, you should use multiple clients and ideally multiple 
servers, and preferably  dedicated storage.

I wanted to test it on localhost and check WRITE performance when adding 
datanodes local ( *see the read performance but why not write ?**??* )


Regards!



W dniu 2013-09-16 14:43, Mason Sharp pisze:
>
>
>
> On Mon, Sep 16, 2013 at 6:34 AM, Bartłomiej Wójcik 
> <bar...@tu... 
> <mailto:bar...@tu...>> wrote:
>
>     Hello,
>
>     Today I am testing the following simple cluster:  Host1(vm1, IP1,
>     6cores 4GB ram, disk storage(very large throughput  ~Gbit/s)):    1
>     gtm    +    1 coordinator    +    (1,2,3) datanode
>
>     Loading the database 100000 inserts and I received the time:      
>     3m1s
>     and    2m57s     (2 of 6 cores work mainly) -    100% of the data at
>     each datanode (checked)
>     Then I do simple selects on that data and I received the time:
>     28m29s  and  28m25s    (2 of 6 cores work mainly)
>
>     I refresh the database (drop and create database)
>     Then I add second datanode to the cluster, do the same test(inserts,
>     selects) and I received:
>
>     2m59s    and  2m57s  (inserts)    (2 of 6 cores work mainly) - 50% of
>     the data at each datanode (checked)
>     15m2s  and  13m41s  (selects)    (2 of 6 cores work mainly)
>
>     I refresh the database (drop and create database again)
>     Then I add third datanode and I received:
>
>     3m1s    and    2m58  (inserts)    (2 of 6 cores work mainly)    -
>     33% of
>     the data at each datanode (checked)
>     9m35s  and  10m5s (selects)    (2 of 6 cores work mainly)
>
>
>     Is this results ok ? Why did not increase write performance(inserts)
>     after adding a datanode? Why did not benefit more from other cores ?
>
>
>     The results are reproducible (as you can see)
>
>
> How many clients/threads/processes did you have in the test? Just one? 
> It seems like it based on that not many cores were being used.
>
> Was the test being run on the same server as the database?
>
> Also, all of the "nodes" are sharing the same storage device, not 
> dedicated for each?
>
> For a better test, you should use multiple clients and ideally 
> multiple servers, and preferably  dedicated storage.
>
>
>
>     Regards!
>
>     BW
>
>
>     ------------------------------------------------------------------------------
>     LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
>     1,500+ hours of tutorials including VisualStudio 2012, Windows 8,
>     SharePoint
>     2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power
>     Pack includes
>     Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13.
>     http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
>     _______________________________________________
>     Postgres-xc-general mailing list
>     Pos...@li...
>     <mailto:Pos...@li...>
>     https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>
> Regards,
>
> -- 
> Mason Sharp
>
> StormDB - http://www.stormdb.com
> The Database Cloud
> Postgres-XC Support and Services