Menu
▾
▴
postgres-xc-general
|
From: Tatsuo I. <is...@po...> - 2012-11-16 05:00:21
|
> (ccing XC general ML) > > On Fri, Nov 16, 2012 at 12:40 PM, Tatsuo Ishii <is...@po...> wrote: > >> We have up to 8 servers, where each server has data >> node/coorrdinator/GTM proxy. We have GTM on a seprate server. Then we >> execute multiple pgbench -k -S against the servers(each pgbench >> connect to different server). >> > OK, this should not happen... > How many max connections are you using per Coordinator 100. > and how many > Coordinators do you have? Up to 4. The number of coordinators are same as data nodes. So when we test 4 data nodes(suppose they are d1 to d4), we are doing something like this: on host c1: pgbench -c 100 -S -h d1 on host c2: pgbench -c 100 -S -h d2 on host c3: pgbench -c 100 -S -h d3 on host c4: pgbench -c 100 -S -h d4 -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese: http://www.sraoss.co.jp |
|
From: Michael P. <mic...@gm...> - 2012-11-16 05:04:26
|
On Fri, Nov 16, 2012 at 1:59 PM, Tatsuo Ishii <is...@po...> wrote: > > (ccing XC general ML) > > > > On Fri, Nov 16, 2012 at 12:40 PM, Tatsuo Ishii <is...@po...> > wrote: > > > >> We have up to 8 servers, where each server has data > >> node/coorrdinator/GTM proxy. We have GTM on a seprate server. Then we > >> execute multiple pgbench -k -S against the servers(each pgbench > >> connect to different server). > >> > > OK, this should not happen... > > How many max connections are you using per Coordinator > > 100. > > > and how many > > Coordinators do you have? > > Up to 4. > The number of coordinators are same as data nodes. So when we test 4 > data nodes(suppose they are d1 to d4), we are doing something like > this: > > on host c1: pgbench -c 100 -S -h d1 > on host c2: pgbench -c 100 -S -h d2 > on host c3: pgbench -c 100 -S -h d3 > on host c4: pgbench -c 100 -S -h d4 > That looks OK. I was wondering if the maximum active transactions possible on GTM (1024) was reached, leading to your error. The warning "Cannot connect to GTM..." "Xid is invalid." only happens when a node tries to get a connection from GTM but it cannot, making me think that GTM is not online or you have some firewall/network problems between the node and GTM. -- Michael Paquier http://michael.otacoo.com |
|
From: Michael P. <mic...@gm...> - 2012-11-16 06:08:02
|
On Fri, Nov 16, 2012 at 2:12 PM, Tatsuo Ishii <is...@po...> wrote: > >> >> We have up to 8 servers, where each server has data > >> >> node/coorrdinator/GTM proxy. We have GTM on a seprate server. Then we > >> >> execute multiple pgbench -k -S against the servers(each pgbench > >> >> connect to different server). > >> >> > >> > OK, this should not happen... > >> > How many max connections are you using per Coordinator > >> > >> 100. > >> > >> > and how many > >> > Coordinators do you have? > >> > >> Up to 4. > >> The number of coordinators are same as data nodes. So when we test 4 > >> data nodes(suppose they are d1 to d4), we are doing something like > >> this: > >> > >> on host c1: pgbench -c 100 -S -h d1 > >> on host c2: pgbench -c 100 -S -h d2 > >> on host c3: pgbench -c 100 -S -h d3 > >> on host c4: pgbench -c 100 -S -h d4 > >> > > That looks OK. I was wondering if the maximum active transactions > possible > > on GTM (1024) was reached, leading to your error. > > The warning "Cannot connect to GTM..." "Xid is invalid." only happens > when > > a node tries to get a connection from GTM but it cannot, making me think > > that GTM is not online or you have some firewall/network problems between > > the node and GTM. > > The error seems not depending on number of node. We did the similar > way as 4 nodes above against 8 nodes configuration. Sometimes we saw > the error, sometimes not. Also we saw the error when the number of > nodes less than 4, for example 2 or 1. > I believe here in a configuration mistake. I work here with clusters up to 10 servers and do not see that if connection to GTM is made correctly even with a huge load. Please double-check gtm_host and gtm_port in each node's postgresql.conf, then check what is the port open on GTM. -- Michael Paquier http://michael.otacoo.com |
|
From: Tatsuo I. <is...@po...> - 2012-11-16 06:32:17
|
> On Fri, Nov 16, 2012 at 2:12 PM, Tatsuo Ishii <is...@po...> wrote: > >> >> >> We have up to 8 servers, where each server has data >> >> >> node/coorrdinator/GTM proxy. We have GTM on a seprate server. Then we >> >> >> execute multiple pgbench -k -S against the servers(each pgbench >> >> >> connect to different server). >> >> >> >> >> > OK, this should not happen... >> >> > How many max connections are you using per Coordinator >> >> >> >> 100. >> >> >> >> > and how many >> >> > Coordinators do you have? >> >> >> >> Up to 4. >> >> The number of coordinators are same as data nodes. So when we test 4 >> >> data nodes(suppose they are d1 to d4), we are doing something like >> >> this: >> >> >> >> on host c1: pgbench -c 100 -S -h d1 >> >> on host c2: pgbench -c 100 -S -h d2 >> >> on host c3: pgbench -c 100 -S -h d3 >> >> on host c4: pgbench -c 100 -S -h d4 >> >> >> > That looks OK. I was wondering if the maximum active transactions >> possible >> > on GTM (1024) was reached, leading to your error. >> > The warning "Cannot connect to GTM..." "Xid is invalid." only happens >> when >> > a node tries to get a connection from GTM but it cannot, making me think >> > that GTM is not online or you have some firewall/network problems between >> > the node and GTM. >> >> The error seems not depending on number of node. We did the similar >> way as 4 nodes above against 8 nodes configuration. Sometimes we saw >> the error, sometimes not. Also we saw the error when the number of >> nodes less than 4, for example 2 or 1. >> > I believe here in a configuration mistake. I work here with clusters up to > 10 servers and do not see that if connection to GTM is made correctly even > with a huge load. > Please double-check gtm_host and gtm_port in each node's postgresql.conf, > then check what is the port open on GTM. Each server has data node, coordinator, and GTM proxy. Every postgresql.conf has gtm_host = 'localhost' and gtm_port = 6666. We double checked all postgresql.conf has same value and also GTM proxy is up and running. -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese: http://www.sraoss.co.jp |
|
From: Michael P. <mic...@gm...> - 2012-11-16 07:23:34
|
On Fri, Nov 16, 2012 at 3:31 PM, Tatsuo Ishii <is...@po...> wrote: > Each server has data node, coordinator, and GTM proxy. Every > postgresql.conf has gtm_host = 'localhost' and gtm_port = 6666. We > double checked all postgresql.conf has same value and also GTM proxy > is up and running. > Do you have error messages in gtm.log, located in gtm data folder, related to thread cleanup or transaction IDs that cannot be taken? -- Michael Paquier http://michael.otacoo.com |
|
From: Tatsuo I. <is...@po...> - 2012-11-16 05:13:12
|
>> >> We have up to 8 servers, where each server has data >> >> node/coorrdinator/GTM proxy. We have GTM on a seprate server. Then we >> >> execute multiple pgbench -k -S against the servers(each pgbench >> >> connect to different server). >> >> >> > OK, this should not happen... >> > How many max connections are you using per Coordinator >> >> 100. >> >> > and how many >> > Coordinators do you have? >> >> Up to 4. >> The number of coordinators are same as data nodes. So when we test 4 >> data nodes(suppose they are d1 to d4), we are doing something like >> this: >> >> on host c1: pgbench -c 100 -S -h d1 >> on host c2: pgbench -c 100 -S -h d2 >> on host c3: pgbench -c 100 -S -h d3 >> on host c4: pgbench -c 100 -S -h d4 >> > That looks OK. I was wondering if the maximum active transactions possible > on GTM (1024) was reached, leading to your error. > The warning "Cannot connect to GTM..." "Xid is invalid." only happens when > a node tries to get a connection from GTM but it cannot, making me think > that GTM is not online or you have some firewall/network problems between > the node and GTM. The error seems not depending on number of node. We did the similar way as 4 nodes above against 8 nodes configuration. Sometimes we saw the error, sometimes not. Also we saw the error when the number of nodes less than 4, for example 2 or 1. -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese: http://www.sraoss.co.jp |
Thanks for helping keep SourceForge clean.
X