Re: [Postgres-xc-general] Performance when adding new datanode

[Ashutosh B. <ash...@en...>]

On Mon, Sep 16, 2013 at 7:28 PM, Bartłomiej Wójcik <
bar...@tu...> wrote:

>  How many clients/threads/processes did you have in the test? Just one?
> It seems like it based on that not many cores were being used.
>
> Just one client:  psql -d test < insert.sql (100000 inserts) sequentially
> with  psql -d test < select.sql (100000 selects)
>
>  Was the test being run on the same server as the database?
>
> Yes, it was.
>
>
Try to insert the rows in parallel using multiple sessions (and if there
are more than one coordinators, use multiple coordinators.)


>  Also, all of the "nodes" are sharing the same storage device, not
> dedicated for each?
>
> All of the "nodes" are local. Storage device the same but it is very large
> throughput  ~Gbit/s
>
>  For a better test, you should use multiple clients and ideally multiple
> servers, and preferably  dedicated storage.
>
> I wanted to test it on localhost and check WRITE performance when adding
> datanodes local ( *see the read performance but why not write ?**??* )
>
>
> Regards!
>
>
>
> W dniu 2013-09-16 14:43, Mason Sharp pisze:
>
>
>
>
> On Mon, Sep 16, 2013 at 6:34 AM, Bartłomiej Wójcik <
> bar...@tu...> wrote:
>
>> Hello,
>>
>> Today I am testing the following simple cluster:    Host1(vm1, IP1,
>> 6cores 4GB ram, disk storage(very large throughput  ~Gbit/s)):    1
>> gtm    +    1 coordinator    +    (1,2,3) datanode
>>
>> Loading the database 100000 inserts and I received the time:       3m1s
>> and    2m57s     (2 of 6 cores work mainly) -    100% of the data at
>> each datanode (checked)
>> Then I do simple selects on that data and I received the time:
>> 28m29s  and  28m25s    (2 of 6 cores work mainly)
>>
>> I refresh the database (drop and create database)
>> Then I add second datanode to the cluster, do the same test(inserts,
>> selects) and I received:
>>
>> 2m59s    and  2m57s  (inserts)    (2 of 6 cores work mainly) - 50% of
>> the data at each datanode (checked)
>> 15m2s  and  13m41s  (selects)    (2 of 6 cores work mainly)
>>
>> I refresh the database (drop and create database again)
>> Then I add third datanode and I received:
>>
>> 3m1s    and    2m58  (inserts)    (2 of 6 cores work mainly)    - 33% of
>> the data at each datanode (checked)
>> 9m35s  and  10m5s (selects)    (2 of 6 cores work mainly)
>>
>>
>> Is this results ok ? Why did not increase write performance(inserts)
>> after adding a datanode? Why did not benefit more from other cores ?
>>
>>
>> The results are reproducible (as you can see)
>>
>
>  How many clients/threads/processes did you have in the test? Just one?
> It seems like it based on that not many cores were being used.
>
>  Was the test being run on the same server as the database?
>
>  Also, all of the "nodes" are sharing the same storage device, not
> dedicated for each?
>
>  For a better test, you should use multiple clients and ideally multiple
> servers, and preferably  dedicated storage.
>
>
>
>>
>>
>> Regards!
>>
>> BW
>>
>>
>>
>> ------------------------------------------------------------------------------
>> LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
>> 1,500+ hours of tutorials including VisualStudio 2012, Windows 8,
>> SharePoint
>> 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack
>> includes
>> Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>
>
> Regards,
>
>  --
> Mason Sharp
>
> StormDB - http://www.stormdb.com
> The Database Cloud
> Postgres-XC Support and Services
>
>
>
>
> ------------------------------------------------------------------------------
> LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99!
> 1,500+ hours of tutorials including VisualStudio 2012, Windows 8,
> SharePoint
> 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack
> includes
> Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13.
> http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>


-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company