postgres-xc-developers

Re: [Postgres-xc-developers] A question on DBT1 performance

[xiong w. <wan...@gm...>]

Hi Mason,

I tested it by 5 PCs.

The enviroment as follows:

2 PCs,  one datanode and one coordinator together on each of them,
GTM is on another PC,
2 Loaders are on other 2 PCs.
Network 1G.


 I tested Postgres-XC in two methods as follows:

Method 1.

loader -------- coordinator & datanode
                                                     \
                                                         GTM
                                                     /
loader -------- coordinator & datanode


Method 2.
loader-------- coordinator & datanode
          \      /                                    \
            \  /                                        \
             /\                                             GTM

           /   \                                         /
         /       \                                     /
loader -------- coordinator & datanode


The DBT1 test results in these two methods are very different. Method 1 is
much better than Method 2. I don't know why.

If I test Postgres-XC in Method 1, the DBT1 performance is close to what
the document declares. If I test it in Method 2, the result is much worse
than what the document writes. Could you tell me why the two methods have so
much effect on the DBT1 performace.

Thanks,

Regards,
Benny

>How much worse?

>How many physical servers are in each configuration? How is each server
>configured in each, with how many data nodes? What kind of network?
>Gigabit?

>Or was everything on one system? With virtual machines or without and
>just using different ports?

>Are there errors in the log file (connection limits hit)?

>Regards,

>Mason

Re: [Postgres-xc-developers] A question on DBT1 performance

[Koichi S. <ko...@in...>]

Hi, Xiong;

Could you tell me CPU and I/O usage you can measure by sar?   I'm afraid 
load balance is not good in Method 2.   How many backend did you use in 
each coordinator?   Did you have any warning that connection overflew in 
data nodes?

Also, how long warm-up did you have?

I'll let you know our configuration (sorry please let me have a bit).

Regards;
---
Koichi Suzuki

(2010年11月24日 14:40), xiong wang wrote:
> Hi Mason,
> I tested it by 5 PCs.
> The enviroment as follows:
> 2 PCs,  one datanode and one coordinator together on each of them,
> GTM is on another PC,
> 2 Loaders are on other 2 PCs.
> Network 1G.
>   I tested Postgres-XC in two methods as follows:
> Method 1.
> loader -------- coordinator & datanode
>                                                       \
>                                                           GTM
>                                                       /
> loader -------- coordinator & datanode
> Method 2.
> loader-------- coordinator & datanode
>            \      /                                    \
>              \  /                                        \
>               /\                                             GTM
>             /   \                                         /
>           /       \                                     /
> loader -------- coordinator & datanode
> The DBT1 test results in these two methods are very different. Method 1
> is much better than Method 2. I don't know why.
> If I test Postgres-XC in Method 1, the DBT1 performance is close to what
> the document declares. If I test it in Method 2, the result is much
> worse than what the document writes. Could you tell me why the two
> methods have so much effect on the DBT1 performace.
> Thanks,
> Regards,
> Benny
>  >How much worse?
>  >How many physical servers are in each configuration? How is each server
>  >configured in each, with how many data nodes? What kind of network?
>  >Gigabit?
>  >Or was everything on one system? With virtual machines or without and
>  >just using different ports?
>  >Are there errors in the log file (connection limits hit)?
>  >Regards,
>  >Mason
>
>
>
> ------------------------------------------------------------------------------
> Increase Visibility of Your 3D Game App&  Earn a Chance To Win $500!
> Tap into the largest installed PC base&  get more eyes on your game by
> optimizing for Intel(R) Graphics Technology. Get started today with the
> Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
> http://p.sf.net/sfu/intelisp-dev2dev
>
>
>
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers

Re: [Postgres-xc-developers] A question on DBT1 performance

[Michael P. <mic...@gm...>]

Hi,

I have additional questions:
What are the parameters you used for the measurement? How many emulated
users? What is the ramp-up time?
Note: ramp-up time can be decided with the parameter called emulated
users/min.
What is the average response time of DBT-1 transactions?
What is the think time, the time a DBT-1 backend waits between receiving a
response and sending its next request?

Based on those parameters, how much output did you get, in term of
transactions per second?
Instead of output, could you give us a rate of how much it became worse?

-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

Re: [Postgres-xc-developers] A question on DBT1 performance

[xiong w. <wan...@gm...>]

Dears,

I am really appreciated your response.

Here is the basic information about my test.

 My configure on DBT1 as follows:

[appServer]
#dbconnection-connection from 1 dbdriver to 1 backend, with 4 loaders, each
of the 5 coords receives 40 connections
10
#transaction_queue_size
1500
#transaction_array_size
1500

[dbdriver]
#items
1000
#customers
28800
#eu
1500
#eu/min
1000
#mean think_time
0.1
#run_duration in seconds
1200
#access mode access with access_direct or access_appServer
access_appServer
#access clean of order table with access_clean if cleanup, by default
disactivated if let empty
access_clean

The results by sar as follows:

 Method1:

*coordinator/datanode 1:*

        CPU     %user     %nice   %system   %iowait    %steal     %idle

Average:   all     58.90      0.00     23.61      0.81      0.00     16.69**

*coordinator/datanode 2:*

        CPU     %user     %nice   %system   %iowait    %steal     %idle

Average:   all     64.19      0.00     28.67      0.55      0.00      6.58



Method2:

*coordinator/datanode 1:*

        CPU     %user     %nice   %system   %iowait    %steal     %idle

Average:   all     59.35      0.00     25.39      0.47      0.00     14.79

*coordinator/datanode 2:*

        CPU     %user     %nice   %system   %iowait    %steal     %idle

Average:   all     57.71      0.00     24.56      0.55      0.00     17.18





The average results by DBT1 as follows:



Method1:

*loader 1:*

1356.6 bogotransactions per second**

*loader 2:*

1843.7 bogotransactions per second



Method2:

*loader 1:*

757.4 bogotransactions per second**

*loader 2:*

779.3 bogotransactions per second





It's obvious that there is a big difference between Method1 and Method2. I
am courious about why.



Thanks again.





Regards,



Benny




2010/11/24 Koichi Suzuki <ko...@in...>

> Hi, Xiong;
>
> Could you tell me CPU and I/O usage you can measure by sar?   I'm afraid
> load balance is not good in Method 2.   How many backend did you use in each
> coordinator?   Did you have any warning that connection overflew in data
> nodes?

Also, how long warm-up did you have?
>
> I'll let you know our configuration (sorry please let me have a bit).
>
> Regards;
> ---
> Koichi Suzuki
>
>
> (2010年11月24日 14:40), xiong wang wrote:
>
>>  Hi Mason,
>> I tested it by 5 PCs.
>> The enviroment as follows:
>> 2 PCs,  one datanode and one coordinator together on each of them,
>> GTM is on another PC,
>> 2 Loaders are on other 2 PCs.
>> Network 1G.
>>  I tested Postgres-XC in two methods as follows:
>> Method 1.
>> loader -------- coordinator & datanode
>>                                                      \
>>                                                          GTM
>>                                                      /
>> loader -------- coordinator & datanode
>> Method 2.
>> loader-------- coordinator & datanode
>>           \      /                                    \
>>             \  /                                        \
>>              /\                                             GTM
>>            /   \                                         /
>>          /       \                                     /
>> loader -------- coordinator & datanode
>> The DBT1 test results in these two methods are very different. Method 1
>> is much better than Method 2. I don't know why.
>> If I test Postgres-XC in Method 1, the DBT1 performance is close to what
>> the document declares. If I test it in Method 2, the result is much
>> worse than what the document writes. Could you tell me why the two
>> methods have so much effect on the DBT1 performace.
>> Thanks,
>> Regards,
>> Benny
>>  >How much worse?
>>  >How many physical servers are in each configuration? How is each server
>>  >configured in each, with how many data nodes? What kind of network?
>>  >Gigabit?
>>  >Or was everything on one system? With virtual machines or without and
>>  >just using different ports?
>>  >Are there errors in the log file (connection limits hit)?
>>  >Regards,
>>  >Mason
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Increase Visibility of Your 3D Game App&  Earn a Chance To Win $500!
>> Tap into the largest installed PC base&  get more eyes on your game by
>> optimizing for Intel(R) Graphics Technology. Get started today with the
>> Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
>> http://p.sf.net/sfu/intelisp-dev2dev
>>
>>
>>
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>
>

Re: [Postgres-xc-developers] A question on DBT1 performance

[Koichi S. <ko...@in...>]

Hi,

I compared your result with ours and I found your environment consumes
more system CPU (in our case, with 90% workload, system CPU is around 18
to 19%) and also your result idle % (14% and 17%) is a bit higher than
your case.   In our case, user CPU is around 70%, system around 19% and
the idle around was 10% (we put a workload so that 90% of the server CPU
resource is used).  We can reduce the idle to a few % by charging much
more workload.

Idle time shows you've not put a transaction workload to fully use
coordinator/datanode CPUs.

Please take a look at loader's situation (number of transactions per
sec, etc).  I suppose there could be another thing to put DBT-1
transactions equally to all the coordinators.

Kind Regards;
-----------
Koichi Suzuki

(2010年11月25日 14:25), xiong wang wrote:
> Dears,
> I am really appreciated your response.
> 
> Here is the basic information about my test.
> My configure on DBT1 as follows:
> [appServer]
> #dbconnection-connection from 1 dbdriver to 1 backend, with 4 loaders, 
> each of the 5 coords receives 40 connections
> 10
> #transaction_queue_size
> 1500
> #transaction_array_size
> 1500
> 
> [dbdriver]
> #items
> 1000
> #customers
> 28800
> #eu
> 1500
> #eu/min
> 1000
> #mean think_time
> 0.1
> #run_duration in seconds
> 1200
> #access mode access with access_direct or access_appServer
> access_appServer
> #access clean of order table with access_clean if cleanup, by default 
> disactivated if let empty
> access_clean
> The results by sar as follows:
> Method1:
> 
> *coordinator/datanode 1:*
> 
> CPU %user %nice %system %iowait %steal %idle
> 
> Average: all 58.90 0.00 23.61 0.81 0.00 16.69**
> 
> *coordinator/datanode 2:*
> 
> CPU %user %nice %system %iowait %steal %idle
> 
> Average: all 64.19 0.00 28.67 0.55 0.00 6.58
> 
> Method2:
> 
> *coordinator/datanode 1:*
> 
> CPU %user %nice %system %iowait %steal %idle
> 
> Average: all 59.35 0.00 25.39 0.47 0.00 14.79
> 
> *coordinator/datanode 2:*
> 
> CPU %user %nice %system %iowait %steal %idle
> 
> Average: all 57.71 0.00 24.56 0.55 0.00 17.18
> 
> The average results by DBT1 as follows:
> 
> Method1:
> 
> *loader 1:*
> 
> 1356.6 bogotransactions per second**
> 
> *loader 2:*
> 
> 1843.7 bogotransactions per second
> 
> Method2:
> 
> *loader 1:*
> 
> 757.4 bogotransactions per second**
> 
> *loader 2:*
> 
> 779.3 bogotransactions per second
> 
> It's obvious that there is a big difference between Method1 and Method2. 
> I am courious about why.
> 
> Thanks again.
> 
> Regards,
> 
> Benny
> 
> 
> 2010/11/24 Koichi Suzuki <ko...@in... 
> <mailto:ko...@in...>>
> 
>     Hi, Xiong;
> 
>     Could you tell me CPU and I/O usage you can measure by sar? I'm
>     afraid load balance is not good in Method 2. How many backend did
>     you use in each coordinator? Did you have any warning that
>     connection overflew in data nodes? 
> 
>     Also, how long warm-up did you have?
> 
>     I'll let you know our configuration (sorry please let me have a bit).
> 
>     Regards;
>     ---
>     Koichi Suzuki
> 
> 
>     (2010年11月24日 14:40), xiong wang wrote:
> 
>         Hi Mason,
>         I tested it by 5 PCs.
>         The enviroment as follows:
>         2 PCs, one datanode and one coordinator together on each of them,
>         GTM is on another PC,
>         2 Loaders are on other 2 PCs.
>         Network 1G.
>         I tested Postgres-XC in two methods as follows:
>         Method 1.
>         loader -------- coordinator & datanode
>         \
>         GTM
>         /
>         loader -------- coordinator & datanode
>         Method 2.
>         loader-------- coordinator & datanode
>         \ / \
>         \ / \
>         /\ GTM
>         / \ /
>         / \ /
>         loader -------- coordinator & datanode
>         The DBT1 test results in these two methods are very different.
>         Method 1
>         is much better than Method 2. I don't know why.
>         If I test Postgres-XC in Method 1, the DBT1 performance is close
>         to what
>         the document declares. If I test it in Method 2, the result is much
>         worse than what the document writes. Could you tell me why the two
>         methods have so much effect on the DBT1 performace.
>         Thanks,
>         Regards,
>         Benny
>          >How much worse?
>          >How many physical servers are in each configuration? How is
>         each server
>          >configured in each, with how many data nodes? What kind of
>         network?
>          >Gigabit?
>          >Or was everything on one system? With virtual machines or
>         without and
>          >just using different ports?
>          >Are there errors in the log file (connection limits hit)?
>          >Regards,
>          >Mason
> 
> 
> 
>         ------------------------------------------------------------------------------
>         Increase Visibility of Your 3D Game App& Earn a Chance To Win $500!
>         Tap into the largest installed PC base& get more eyes on your
>         game by
>         optimizing for Intel(R) Graphics Technology. Get started today
>         with the
>         Intel(R) Software Partner Program. Five $500 cash prizes are up
>         for grabs.
>         http://p.sf.net/sfu/intelisp-dev2dev
> 
> 
> 
>         _______________________________________________
>         Postgres-xc-developers mailing list
>         Pos...@li...
>         <mailto:Pos...@li...>
>         https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> 
> 
> 

Re: [Postgres-xc-developers] A question on DBT1 performance

[Michael P. <mic...@gm...>]

I have a comment about some of the parameters you are using with DBT-1.
#eu
1500
#eu/min
1000

The current algorithm of DBT-1 just recognizes multiple numbers when
calculating ramp-up value.
For example, if you set eu at 1500, you have to set eu/min at 500, 750 or
1500 to have a ramp up time of 1, 2 or 3 minutes.
If you don't do that ramp up is set I set at 0, what might give you wrong
results.

Then, you seem still have a lot of idle CPU.
You should increase eu by 15%~20% to saturate your cpu.

I would recommend to set eu at 2000~2200 for a eu/min at 500~550.

I can also see in your results that method 1 is using in total 2 connections
between the loaders and coordinators.
Method 2 is using 4 connections.
I think you did so but... Did you include for method 2 the results of
loader1-coordinator1 + loader1-coordinator2 for loader1 results?
Same for loader2?

-- 
Michael Paquier
http://michaelpq.users.sourceforge.net

Re: [Postgres-xc-developers] A question on DBT1 performance

[xiong w. <wan...@gm...>]

Hi, Michael

Thank you for your advice.


2010/11/25 Michael Paquier <mic...@gm...>

> I have a comment about some of the parameters you are using with DBT-1.
>
> #eu
> 1500
> #eu/min
> 1000
>
> The current algorithm of DBT-1 just recognizes multiple numbers when
> calculating ramp-up value.
> For example, if you set eu at 1500, you have to set eu/min at 500, 750 or
> 1500 to have a ramp up time of 1, 2 or 3 minutes.
> If you don't do that ramp up is set I set at 0, what might give you wrong
> results.
>
> Then, you seem still have a lot of idle CPU.
> You should increase eu by 15%~20% to saturate your cpu.
>
> I would recommend to set eu at 2000~2200 for a eu/min at 500~550.
>
I will test it again later.


>
> I can also see in your results that method 1 is using in total 2
> connections between the loaders and coordinators.
> Method 2 is using 4 connections.
> I think you did so but... Did you include for method 2 the results of
> loader1-coordinator1 + loader1-coordinator2 for loader1 results?
> Same for loader2?
>
As I know one loader can get one DBT1 result. As you told, a coordinator
should get one DBT1 result beacuse you said I should use
loader1-coordinator1 + loader1-coordinator2 as the result of method2. I don'
t know how can I get DBT1 results according to coordinator.

Thanks again.

Regards,
Benny

>
> --
> Michael Paquier
> http://michaelpq.users.sourceforge.net
>
>

Re: [Postgres-xc-developers] A question on DBT1 performance

[Michael P. <mic...@gm...>]

>
>
>
>>
>> I can also see in your results that method 1 is using in total 2
>> connections between the loaders and coordinators.
>> Method 2 is using 4 connections.
>> I think you did so but... Did you include for method 2 the results of
>> loader1-coordinator1 + loader1-coordinator2 for loader1 results?
>> Same for loader2?
>>
> As I know one loader can get one DBT1 result. As you told, a coordinator
> should get one DBT1 result beacuse you said I should use
> loader1-coordinator1 + loader1-coordinator2 as the result of method2. I don'
> t know how can I get DBT1 results according to coordinator.
>
>
If you use one DBT-1 folder for two instances to coordinators from a loader,
both instances try to write at the same time in the same mix.log, what could
result in a loss of data.
Why not separating the instances by launching each loaderX-coordinatorX
instance in a different DBT-1 folder?
In the case of method 2, create 2 DBT-1 folders on each loader server, for a
total of 4 folders.
You will have 4 mix.log file for the output of:
- loader1/coordinator1
- loader1/coordinator2
- loader2/coordinator1
- loader2/coordinator2

If you do that, you have just to combine mix.log files of
loader2/coordinator1 and loader2/coordinator2 to get the output of a loader
2 for instance.
It may be also interesting to get the output from Coordinator 1 and 2, by
combining for instance log files of loader1/coordinatorX and
loader2/coordinatorX.

-- 
Michael Paquier
http://michaelpq.users.sourceforge.net