Re: [Postgres-xc-bugs] Error about pgxc_pool_reload()

[ZhangJulian <jul...@ou...>]

The error has nothing to do with ALTER TABLE DELETE NODE, which is only to make current sessions connecting to other coordinators. 

The error is about "pgxc_ctl remove datanode master datanode4", anyway, we can work around it by restaring the whole cluster once the cluster status is changed. 

From: ko...@in...
To: jul...@ou...
CC: pos...@li...
Subject: Re: [Postgres-xc-bugs] Error about pgxc_pool_reload()
Date: Mon, 17 Nov 2014 05:44:15 +0000






I’m thinking that pgxc_pool_reload() may not be a good solution.   To be honest, ALTER TABLE DELETE NODE has nothing to do with the pooler.   It just drops specified node from the node list of pgxc_class.   I suspect there may be another cause for the problem.



Furthermore, pgxc_pool_reload() enforces to release all the existing pooler connections to all the node, which may affect existing transactions.



I’d suggest to be a big careful about this handling.



Regards;

---
Koichi Suzuki





2014/11/14 18:35、ZhangJulian <jul...@ou...> のメル：



Thanks for the quich reply!

Furthermore, do you think the pgxc_ctl tool should issue the "select pgxc_pool_reload()" to all the coordinators after it realize the cluster nodes has been changed?




Thanks

Julian



From: ko...@in...

To: jul...@ou...

CC: pos...@li...

Subject: Re: [Postgres-xc-bugs] Error about pgxc_pool_reload()

Date: Fri, 14 Nov 2014 09:25:12 +0000



Hmm.   pgxc_pool_reload() just reads all the pgxc_node catalog, which is usually very small and I think we can ignore its overhead.
Please let me think about the idea.   May respond early next week.



Thank you very much;

---
Koichi Suzuki





2014/11/14 18:10、ZhangJulian <jul...@ou...> のメル：



======Scenario 1: To recreate it, create a 4C4D environment and connect to coord1:

create table t1(id int) distribute by hash(id);

insert into t1 (select generate_series(1, 1000));

ALTER TABLE t1 DELETE NODE (datanode4);

\! pgxc_ctl remove datanode master datanode4

select pgxc_pool_reload();                                 ==>psql to other coordinators, run select pgxc_pool_reload();



create table t2(id int) distribute by hash(id);

ERROR:  cache lookup failed for node 16390

======Scenario 2: To work around it, 

create table t1(id int) distribute by hash(id);

insert into t1 (select generate_series(1, 1000));

ALTER TABLE t1 DELETE NODE (datanode4);    ==>psql to other coordinators before "pgxc_ctl remove datanode"

\! pgxc_ctl remove datanode master datanode4

select pgxc_pool_reload();                                 ==>run select pgxc_pool_reload() on the other connected coordinators



create table t2(id int) distribute by hash(id);

Success!!

======After the debug,

In the function,  

Datum

pgxc_pool_reload(PG_FUNCTION_ARGS)

{

    ......

    /* No need to reload, node information is consistent */

    if (PoolManagerCheckConnectionInfo())

    {

        /* Release the lock on pooler */

        PoolManagerLock(false);

        PG_RETURN_BOOL(true);

    }

    ......

    /* Signal other sessions to reconnect to pooler */

    ReloadConnInfoOnBackends();

    ......

}

In Scenario 1, the new connected sessions could get the updated cluster stat values, for example, "NumDataNodes". Then PoolManagerCheckConnectionInfo() will return true, and ReloadConnInfoOnBackends has no chance to be executed.


That is why Scenario 2 could work around it, since the old sessions know the cluster has been changed and could signal other backends (the backend on coord2-4 for the session of coord1) by ReloadConnInfoOnBackends().




======The possible fix.

Erase the short path of PoolManagerCheckConnectionInfo() check, do the whole process once user run "select pgxc_pool_reload().



Please advise!



Thanks 

Julian







------------------------------------------------------------------------------

Comprehensive Server Monitoring with Site24x7.

Monitor 10 servers for $9/Month.

Get alerted through email, SMS, voice calls or mobile push notifications.

Take corrective actions from your mobile device.

http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk_______________________________________________

Postgres-xc-bugs mailing list

Pos...@li...

https://lists.sourceforge.net/lists/listinfo/postgres-xc-bugs