postgres-xc-general — General info and messages

Re: [Postgres-xc-general] shared_queues and shared_queue_size

[Steven C. <ste...@gm...>]

Thank you for reminding.

Regards,
Steven

2018-05-28 14:04 GMT+08:00 Steven Chang <ste...@gm...>:

> Hello,
>
>       Can anyone give me an advice on how to estimate shared_queues and
> shared_queue_size ??
>
>
> Regards,
> Steven
>

Re: [Postgres-xc-general] shared_queues and shared_queue_size

[鈴木 幸市 <ko...@in...>]

I suggest to go to postgres-xl ML, which succeeded all the XC features with MPP capability.

---
Koichi Suzuki
ko...@in...<mailto:ko...@in...>

2018/05/28 15:04、Steven Chang <ste...@gm...<mailto:ste...@gm...>> のメール：

Hello,

      Can anyone give me an advice on how to estimate shared_queues and shared_queue_size ??


Regards,
Steven
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org<http://slashdot.org>! http://sdm.link/slashdot_______________________________________________
Postgres-xc-general mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

[Postgres-xc-general] shared_queues and shared_queue_size

[Steven C. <ste...@gm...>]

Hello,

      Can anyone give me an advice on how to estimate shared_queues and
shared_queue_size ??


Regards,
Steven

[Postgres-xc-general] Can't find "pgxc_ctl unregister" and "pgxc_remove_gtm" and "pgxc_update_gtm" command

[Steven C. <ste...@gm...>]

Hello,

    I can't find unregister command in pgxc_ctl shell,
    and the comments in pgxc_ctl.conf file mention two commands
pgxc_remove_gtm and pgxc_update_gtm.
    I can't find them either.
    Would you mind telling me where they are and how to use them.
    Thank you.

Best Regards,
Steven

Re: [Postgres-xc-general] Still Active?

[Jov <am...@am...>]

I think so.
the latest commit happened 2 hours ago:
https://git.postgresql.org/gitweb/?p=postgres-xl.git;a=summary

2017-06-20 11:01 GMT+08:00 Zachary Winnerman <zac...@wo...>:

> Hello,
>
> I was researching Postgres distributed options and came across this
> project.
>
> Is this project still active?
>
> Thank you,
> Zachary Winnerman
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

Re: [Postgres-xc-general] Still Active?

[鈴木 幸市 <ko...@in...>]

Unfortunately, the project is not active these days.    Please try PGXL, which is Postgres-XC 1.1 folk, covering PG9.5 and later.
---
Koichi Suzuki
ko...@in...

> 2017/06/20 12:01、Zachary Winnerman <zac...@wo...> のメール：
> 
> Hello,
> 
> I was researching Postgres distributed options and came across this project.
> 
> Is this project still active?
> 
> Thank you,
> Zachary Winnerman
> 
> 
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> 

[Postgres-xc-general] Still Active?

[Zachary W. <zac...@wo...>]

Hello,

I was researching Postgres distributed options and came across this project.

Is this project still active?

Thank you,
Zachary Winnerman

Re: [Postgres-xc-general] what happens to pgxc and transition to xl

[Sandeep G. <gup...@gm...>]

Hi Pavan,

 Sounds great.  I am interested to take a second look at XL.  Will try to
redo the comparison
in light of the performance improvements to XL.


Thanks
Sandeep




On Thu, Jan 21, 2016 at 3:39 AM, Pavan Deolasee <pav...@gm...>
wrote:

> Hi Sandeep,
>
> We're working to consolidate XC and XL efforts, but it may take some time.
> In the meanwhile, both XC and XL are being enhanced independently, often
> leveraging from each other's work.
>
> We've recently fixed several performance related issues in XL, especially
> for OLTP-like workload and I'm quite hopeful that the gap between XC and XL
> for such workloads must have drastically reduced, if not crossed already.
> For OLAP complex queries, XL should generally do better than XC.
>
> Do you mind testing the latest XL 9.5 code base in your environment and
> let us know if the performance is satisfactory? Our goal is to keep
> improving performance for both OLAP as well as OLTP workloads. So if there
> are gaps, those must be filled. The latest code is available here and I
> would suggest using XL9_5_STABLE branch for the tests.
> git://git.postgresql.org/git/postgres-xl.git
>
> Thanks,
> Pavan
>
> On Thu, Jan 21, 2016 at 6:47 AM, Sandeep Gupta <gup...@gm...>
> wrote:
>
>> Hi,
>>
>> In the past we  had used postgres-xc to successfully
>> scale our workload which mostly consisted of simple
>> filter-join-groupby expression.
>> All this required some careful data distribution strategies but on
>> average we were
>> able to extract good performance.
>>
>>  We tested our workloads with XL and the performance, at least for our
>> workload, was
>>  not at par. In this regard, I was wondering what is the future
>> direction of XC.
>>  If it is shelved, then shouldn't there be a way to match the
>> performance in XL given
>>  that XL has more generic framework compared to XC.
>>
>>
>>
>>  Best,
>>  Sandeep
>>
>>
>> ------------------------------------------------------------------------------
>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>
>
>
> --
>  Pavan Deolasee                   http://www.2ndQuadrant.com/
>  PostgreSQL Development, 24x7 Support, Training & Services
>

Re: [Postgres-xc-general] what happens to pgxc and transition to xl

[Pavan D. <pav...@gm...>]

Hi Sandeep,

We're working to consolidate XC and XL efforts, but it may take some time.
In the meanwhile, both XC and XL are being enhanced independently, often
leveraging from each other's work.

We've recently fixed several performance related issues in XL, especially
for OLTP-like workload and I'm quite hopeful that the gap between XC and XL
for such workloads must have drastically reduced, if not crossed already.
For OLAP complex queries, XL should generally do better than XC.

Do you mind testing the latest XL 9.5 code base in your environment and let
us know if the performance is satisfactory? Our goal is to keep improving
performance for both OLAP as well as OLTP workloads. So if there are gaps,
those must be filled. The latest code is available here and I would suggest
using XL9_5_STABLE branch for the tests.
git://git.postgresql.org/git/postgres-xl.git

Thanks,
Pavan

On Thu, Jan 21, 2016 at 6:47 AM, Sandeep Gupta <gup...@gm...>
wrote:

> Hi,
>
> In the past we  had used postgres-xc to successfully
> scale our workload which mostly consisted of simple
> filter-join-groupby expression.
> All this required some careful data distribution strategies but on
> average we were
> able to extract good performance.
>
>  We tested our workloads with XL and the performance, at least for our
> workload, was
>  not at par. In this regard, I was wondering what is the future
> direction of XC.
>  If it is shelved, then shouldn't there be a way to match the
> performance in XL given
>  that XL has more generic framework compared to XC.
>
>
>
>  Best,
>  Sandeep
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>



-- 
 Pavan Deolasee                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services

[Postgres-xc-general] what happens to pgxc and transition to xl

[Sandeep G. <gup...@gm...>]

Hi,

In the past we  had used postgres-xc to successfully
scale our workload which mostly consisted of simple
filter-join-groupby expression.
All this required some careful data distribution strategies but on
average we were
able to extract good performance.

 We tested our workloads with XL and the performance, at least for our
workload, was
 not at par. In this regard, I was wondering what is the future
direction of XC.
 If it is shelved, then shouldn't there be a way to match the
performance in XL given
 that XL has more generic framework compared to XC.



 Best,
 Sandeep

Re: [Postgres-xc-general] segfault while adding delete datanode

[muayad y <mua...@ya...>]

Hi,postgres-xc-1.2.1

dbg:
 Program received signal SIGSEGV, Segmentation fault.0x00000000004b3fd4 in heap_update (relation=0x2af74668f6c8, otid=0x91ae61b4, newtup=0xffffffff91ae62f8, cid=0, crosscheck=0x0, wait=1 '\001',     hufd=0x7fffaff71640, lockmode=0x7fffaff7163c) at heapam.c:30123012 newtup->t_tableOid = RelationGetRelid(relation);
(gdb) info args followed by print for each address relation = 0x2af74668f6c8= > $44 = 47241526572744otid = 0x91ae61b4  => p otid => $45 = (ItemPointer) 0x91ae61b4 => p 0x91ae61b4 =>$46 = 2444124596newtup = 0xffffffff91ae62f8 p=>  $47 = 18446744071858709240cid = 0crosscheck = 0x0wait = 1 '\001'hufd = 0x7fffaff71640 => $48 = 140736145593920lockmode = 0x7fffaff7163c =>  $49 = 140736145593916

(gdb) info localsresult = HeapTupleMayBeUpdatedxid = 67601hot_attrs = 0x91ae6518key_attrs = 0x1e4f3c8lp = 0x2af73e072fb0oldtup = {t_len = 72, t_self = {ip_blkid = {bi_hi = 0, bi_lo = 0}, ip_posid = 7}, t_tableOid = 9001, t_xc_node_id = 108, t_data = 0x2af73e074d70}heaptup = 0x7fffaff715f0page = 0x2af73e072f80 ""block = 0mxact_status = MultiXactStatusForKeySharebuffer = 78newbuf = 52vmbuffer = 0vmbuffer_new = 0need_toast = 0 '\000'already_marked = 0 '\000'newtupsize = 29939120pagefree = 8have_tuple_lock = 0 '\000'iscombo = 0 '\000'satisfies_hot = 0 '\000'satisfies_key = 0 '\000'
use_hot_update = 0 '\000'key_intact = 0 '\000'all_visible_cleared = 0 '\000'all_visible_cleared_new = 0 '\000'checked_lockers = -27 '\345'locker_remains = 83 'S'xmax_new_tuple = 0xmax_old_tuple = 0infomask_old_tuple = 0infomask2_old_tuple = 10999infomask_new_tuple = 18024infomask2_new_tuple = 63992__func__ = "heap_update"
ThanksMU
 


     On Thursday, September 10, 2015 9:53 PM, 鈴木 幸市 <ko...@in...> wrote:
   

 Maybe some internal variable value got bigger than the limit.  Could you analyze the core to find where segfault happened (i.e. the variable value which caused the segfault)?   Also, could you let me know the version of XC you used.
Thank you very much;
---Koichi SuzukiNTT DATA Intellilink Cor...@in...

2015/09/11 9:12、muayad y <mua...@ya...> のメール：
Add/delete node works fine when table has ~ 9million rows, but segfault in heap_update when table gets bigger,  any idea help ? thanks in advance 

Core was generated by `postgres: pgxl test [local] ALTER TABLE                                 '.
Program terminated with signal 11, Segmentation fault.
#0  0x00000000004949f0 in heap_update ()
(gdb) bt
#0  0x00000000004949f0 in heap_update ()
#1  0x0000000000495f12 in simple_heap_update ()
#2  0x00000000004fd3c0 in PgxcClassAlter ()
#3  0x00000000005763c8 in ATExecCmd ()
#4  0x0000000000578d56 in ATController ()
#5  0x000000000068ab1d in ProcessUtilitySlow.isra.8 ()
#6  0x0000000000688dc8 in standard_ProcessUtility ()
#7  0x0000000000685a07 in PortalRunUtility ()
#8  0x000000000068668d in PortalRunMulti ()
#9  0x0000000000687272 in PortalRun ()
#10 0x0000000000684970 in PostgresMain ()
#11 0x000000000046d1f9 in ServerLoop ()
#12 0x00000000006429d6 in PostmasterMain ()
#13 0x000000000046da8d in main ()
(gdb) 


table info:
CREATE TABLE sbtest (id int NOT NULL, k int NOT NULL DEFAULT '0', c char(120) NOT NULL DEFAULT '', pad char(60) NOT NULL DEFAULT '', PRIMARY KEY (id)) DISTRIBUTE BY hash(id);

user limits:
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3101645
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 131072
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 3101645
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Thanks 
MU
------------------------------------------------------------------------------
_______________________________________________
Postgres-xc-general mailing list
Pos...@li...
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general




  

Re: [Postgres-xc-general] segfault while adding delete datanode

[鈴木 幸市 <ko...@in...>]

Maybe some internal variable value got bigger than the limit.  Could you analyze the core to find where segfault happened (i.e. the variable value which caused the segfault)?   Also, could you let me know the version of XC you used.

Thank you very much;
---
Koichi Suzuki
NTT DATA Intellilink Corp.
ko...@in...<mailto:ko...@in...>.jp

2015/09/11 9:12、muayad y <mua...@ya...<mailto:mua...@ya...>> のメール：


Add/delete node works fine when table has ~ 9million rows, but segfault in heap_update when table gets bigger,  any idea help ? thanks in advance

Core was generated by `postgres: pgxl test [local] ALTER TABLE                                 '.
Program terminated with signal 11, Segmentation fault.
#0  0x00000000004949f0 in heap_update ()
(gdb) bt
#0  0x00000000004949f0 in heap_update ()
#1  0x0000000000495f12 in simple_heap_update ()
#2  0x00000000004fd3c0 in PgxcClassAlter ()
#3  0x00000000005763c8 in ATExecCmd ()
#4  0x0000000000578d56 in ATController ()
#5  0x000000000068ab1d in ProcessUtilitySlow.isra.8 ()
#6  0x0000000000688dc8 in standard_ProcessUtility ()
#7  0x0000000000685a07 in PortalRunUtility ()
#8  0x000000000068668d in PortalRunMulti ()
#9  0x0000000000687272 in PortalRun ()
#10 0x0000000000684970 in PostgresMain ()
#11 0x000000000046d1f9 in ServerLoop ()
#12 0x00000000006429d6 in PostmasterMain ()
#13 0x000000000046da8d in main ()
(gdb)


table info:
CREATE TABLE sbtest (id int NOT NULL, k int NOT NULL DEFAULT '0', c char(120) NOT NULL DEFAULT '', pad char(60) NOT NULL DEFAULT '', PRIMARY KEY (id)) DISTRIBUTE BY hash(id);

user limits:
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3101645
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 131072
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 3101645
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Thanks
MU

------------------------------------------------------------------------------
_______________________________________________
Postgres-xc-general mailing list
Pos...@li...<mailto:Pos...@li...>
https://lists.sourceforge.net/lists/listinfo/postgres-xc-general

[Postgres-xc-general] segfault while adding delete datanode

[muayad y <mua...@ya...>]

Add/delete node works fine when table has ~ 9million rows, but segfault in heap_update when table gets bigger,  any idea help ? thanks in advance 

Core was generated by `postgres: pgxl test [local] ALTER TABLE                                 '.
Program terminated with signal 11, Segmentation fault.
#0  0x00000000004949f0 in heap_update ()
(gdb) bt
#0  0x00000000004949f0 in heap_update ()
#1  0x0000000000495f12 in simple_heap_update ()
#2  0x00000000004fd3c0 in PgxcClassAlter ()
#3  0x00000000005763c8 in ATExecCmd ()
#4  0x0000000000578d56 in ATController ()
#5  0x000000000068ab1d in ProcessUtilitySlow.isra.8 ()
#6  0x0000000000688dc8 in standard_ProcessUtility ()
#7  0x0000000000685a07 in PortalRunUtility ()
#8  0x000000000068668d in PortalRunMulti ()
#9  0x0000000000687272 in PortalRun ()
#10 0x0000000000684970 in PostgresMain ()
#11 0x000000000046d1f9 in ServerLoop ()
#12 0x00000000006429d6 in PostmasterMain ()
#13 0x000000000046da8d in main ()
(gdb) 


table info:
CREATE TABLE sbtest (id int NOT NULL, k int NOT NULL DEFAULT '0', c char(120) NOT NULL DEFAULT '', pad char(60) NOT NULL DEFAULT '', PRIMARY KEY (id)) DISTRIBUTE BY hash(id);

user limits:
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 3101645
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 131072
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 3101645
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Thanks 
MU

Re: [Postgres-xc-general] Corosync Resource configuration for Postgres-XC

[Koichi S. <koi...@gm...>]

Thank you very much for taking a look at my presentation material.

I don't have an experience in DRBD (I have several friends working with
this though) and to my knowledge, you are the first guy trying to use XC
with DRBD.

Most of the challenges are building coordinator and datanode slaves with
streaming (log-shipping) replication and switch to the slave when a master
crashes.   I have worked with them on general configuration and design of
their resource agent for XC.

Because coordinator and datanode resource management is almost identical to
vanilla PostgreSQL, you can start with PostgreSQL's resource agent.   The
issue in XC is that you have to take care of many, not only one, resources,
I mean, your Pacemaker has to monitor gtm, gtm_proxies, coordinators and
datanodes, configured among more than one physical/virtual servers.

They determined it is far more than current Pacemaker can do.   Instead,
what they did the following:

1. Build slaves for each pair of coordinator/datanode.

For example, if you build datanode1 and datanode2 at node1 and node2, build
datanode1 slave at node2 and datanode2 slave at node1.   Pacemaker takes
care of resources of this pair of the node.

So if you build coordinator1 and coordinator2 at node1 and node2 as well,
you can build their slaves at node2 and node1 respectively and add them to
the above Pacemaker resource.

2. Configure separate Pacemaker to different pair of node as shown in 1.

This could work, except for gtm.

So far, Pacemaker people around me have not found how to monitor and
control gtm.   When gtm fails, Pacemaker may be able to switch gtm to its
slave.   However, when gtm is switched to the slave, all the coordinators
and datanodes should change its gtm destination.   Yes, restarting
gtm_proxy can do this work.  Issue is how to write resource manage to do
this.   VIP may work.

Sorry that I can only such scattered information.

Thank you  again;



---
Koichi Suzuki
https://www.linkedin.com/in/koichidbms


2015-08-26 18:18 GMT+09:00 ssoorruu <sso...@gm...>:

> Hi Suzuki-san,
>
> Thank you for responding to my email.
> At first, your question was sound tough for me (because I'm new to
> DRBD, Postgres and Corosync/Pacemaker/Heartbeat).
> After I re-read the tutorial and the DRBD references, I'm sure that
> the I have configured the DRBD to run Fully-synchronous (called
> protocol C in DRBD term) in fixed rate (1Gbps).
>
> I've read your presentation slide about: High Availability in
> Postgres-XC, The symmetric PostgreSQL cluster
> In that presentation you were using Pacemaker/Hearbeat with Postgres-XC.
>
>
> Would you share to me the guide of how to configure the
> Pacemaker/Heartbeat with Postgres-XC?
> Or maybe you have publish it, can I have the link to your tutorial
> about Postgres-XC-Pacemaker/Heartbeat?
>
>
> Big thanks,
> FattahRozzaq
> --
> On 26/08/2015, Koichi Suzuki <koi...@gm...> wrote:
> > Hello;
> >
> > I just know general thing about DRBD, Corosync and Pacemaker so I'd like
> to
> > post my comment with a question of DRBD basics.
> >
> > First of all, I have one important questions on DRBD basics:
> >
> > Does DRBD send all the update to slaves in synchronous manner, or is
> there
> > any chance that DRBD slave fails to receive some of the updates at the
> > master?
> >
> > If you cannot configure DRBD update backup synchronously, we may have
> some
> > consistent issue among the data stored in different datanodes.
> >
> > The background is:
> >
> > Postgres-XC assumes consistency of data stored in all the coordinators
> and
> > datanodes.    If some coordinator/datanode fails and is switched to the
> > slave, the slave must not lose any commits done to the master.
> >
> > This is the same in the case of log-shipping replication and this is why
> I
> > advise to use synchronized replication.
> >
> > If you can keep slave in sync with the master, then what you should do is
> > to start Postgres-XC component at the slave.
> >
> > Thank you very much;
> >
> > ---
> > Koichi Suzuki
> > https://www.linkedin.com/in/koichidbms
> >
> >
> > 2015-08-26 16:12 GMT+09:00 ssoorruu <sso...@gm...>:
> >
> >> Dear fellow PGXC enthusiasts,
> >>
> >> Good day and I hope you're all doing well, especially with Postgres-XC
> :)
> >>
> >> I have job to deploy Postgres-XC cluster replication using DRBD, and
> >> cluster resource manager using Corosync-Pacemaker.
> >> I've followed the tutorial from this link:
> >>
> >>
> https://www.howtoforge.com/how-to-set-up-an-active-passive-postgresql-cluster-with-pacemaker-corosync-and-drbd-centos-5.5
> >>
> >> But I cannot understand how to configure a Corosync resource for
> >> Postgres-XC.
> >>
> >> Does anyone can give me tutorial on how to do Postgres-XC with
> >> Corosync-Pacemaker?
> >>
> >> Very appreciate and thank you in advance.
> >>
> >>
> >> Regards,
> >> FattahRozzaq
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> _______________________________________________
> >> Postgres-xc-general mailing list
> >> Pos...@li...
> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >>
> >
>

Re: [Postgres-xc-general] Corosync Resource configuration for Postgres-XC

[ssoorruu <sso...@gm...>]

Hi Suzuki-san,

Thank you for responding to my email.
At first, your question was sound tough for me (because I'm new to
DRBD, Postgres and Corosync/Pacemaker/Heartbeat).
After I re-read the tutorial and the DRBD references, I'm sure that
the I have configured the DRBD to run Fully-synchronous (called
protocol C in DRBD term) in fixed rate (1Gbps).

I've read your presentation slide about: High Availability in
Postgres-XC, The symmetric PostgreSQL cluster
In that presentation you were using Pacemaker/Hearbeat with Postgres-XC.


Would you share to me the guide of how to configure the
Pacemaker/Heartbeat with Postgres-XC?
Or maybe you have publish it, can I have the link to your tutorial
about Postgres-XC-Pacemaker/Heartbeat?


Big thanks,
FattahRozzaq
--
On 26/08/2015, Koichi Suzuki <koi...@gm...> wrote:
> Hello;
>
> I just know general thing about DRBD, Corosync and Pacemaker so I'd like to
> post my comment with a question of DRBD basics.
>
> First of all, I have one important questions on DRBD basics:
>
> Does DRBD send all the update to slaves in synchronous manner, or is there
> any chance that DRBD slave fails to receive some of the updates at the
> master?
>
> If you cannot configure DRBD update backup synchronously, we may have some
> consistent issue among the data stored in different datanodes.
>
> The background is:
>
> Postgres-XC assumes consistency of data stored in all the coordinators and
> datanodes.    If some coordinator/datanode fails and is switched to the
> slave, the slave must not lose any commits done to the master.
>
> This is the same in the case of log-shipping replication and this is why I
> advise to use synchronized replication.
>
> If you can keep slave in sync with the master, then what you should do is
> to start Postgres-XC component at the slave.
>
> Thank you very much;
>
> ---
> Koichi Suzuki
> https://www.linkedin.com/in/koichidbms
>
>
> 2015-08-26 16:12 GMT+09:00 ssoorruu <sso...@gm...>:
>
>> Dear fellow PGXC enthusiasts,
>>
>> Good day and I hope you're all doing well, especially with Postgres-XC :)
>>
>> I have job to deploy Postgres-XC cluster replication using DRBD, and
>> cluster resource manager using Corosync-Pacemaker.
>> I've followed the tutorial from this link:
>>
>> https://www.howtoforge.com/how-to-set-up-an-active-passive-postgresql-cluster-with-pacemaker-corosync-and-drbd-centos-5.5
>>
>> But I cannot understand how to configure a Corosync resource for
>> Postgres-XC.
>>
>> Does anyone can give me tutorial on how to do Postgres-XC with
>> Corosync-Pacemaker?
>>
>> Very appreciate and thank you in advance.
>>
>>
>> Regards,
>> FattahRozzaq
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>

Re: [Postgres-xc-general] Corosync Resource configuration for Postgres-XC

[Koichi S. <koi...@gm...>]

Hello;

I just know general thing about DRBD, Corosync and Pacemaker so I'd like to
post my comment with a question of DRBD basics.

First of all, I have one important questions on DRBD basics:

Does DRBD send all the update to slaves in synchronous manner, or is there
any chance that DRBD slave fails to receive some of the updates at the
master?

If you cannot configure DRBD update backup synchronously, we may have some
consistent issue among the data stored in different datanodes.

The background is:

Postgres-XC assumes consistency of data stored in all the coordinators and
datanodes.    If some coordinator/datanode fails and is switched to the
slave, the slave must not lose any commits done to the master.

This is the same in the case of log-shipping replication and this is why I
advise to use synchronized replication.

If you can keep slave in sync with the master, then what you should do is
to start Postgres-XC component at the slave.

Thank you very much;

---
Koichi Suzuki
https://www.linkedin.com/in/koichidbms


2015-08-26 16:12 GMT+09:00 ssoorruu <sso...@gm...>:

> Dear fellow PGXC enthusiasts,
>
> Good day and I hope you're all doing well, especially with Postgres-XC :)
>
> I have job to deploy Postgres-XC cluster replication using DRBD, and
> cluster resource manager using Corosync-Pacemaker.
> I've followed the tutorial from this link:
>
> https://www.howtoforge.com/how-to-set-up-an-active-passive-postgresql-cluster-with-pacemaker-corosync-and-drbd-centos-5.5
>
> But I cannot understand how to configure a Corosync resource for
> Postgres-XC.
>
> Does anyone can give me tutorial on how to do Postgres-XC with
> Corosync-Pacemaker?
>
> Very appreciate and thank you in advance.
>
>
> Regards,
> FattahRozzaq
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>

[Postgres-xc-general] Corosync Resource configuration for Postgres-XC

[ssoorruu <sso...@gm...>]

Dear fellow PGXC enthusiasts,

Good day and I hope you're all doing well, especially with Postgres-XC :)

I have job to deploy Postgres-XC cluster replication using DRBD, and
cluster resource manager using Corosync-Pacemaker.
I've followed the tutorial from this link:
https://www.howtoforge.com/how-to-set-up-an-active-passive-postgresql-cluster-with-pacemaker-corosync-and-drbd-centos-5.5

But I cannot understand how to configure a Corosync resource for Postgres-XC.

Does anyone can give me tutorial on how to do Postgres-XC with
Corosync-Pacemaker?

Very appreciate and thank you in advance.


Regards,
FattahRozzaq

[Postgres-xc-general] Performance Documentation for Postgres XL

[Silva, R. <rs...@ja...>]

Fellow XL users & developers,

Is there documentation out there on the internets regarding datanode count and performance for an XL environment?

We currently have 10 datanodes and now contemplating the performance impact if we were to add 5 more.

thank you...
R!chard Silva  ETL Architect
o 972 443 7052
e rs...@ja...<applewebdata://DB38DD67-D9E1-4CF8-8E0C-08B3C400B24A/rs...@ja...> |  www.javelin.mg<applewebdata://C6724941-5434-4FFF-A530-0F83A527BB10/www.javelin.mg>





Access Manager: This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. Dissemination, distribution or copying of this e-mail or the information herein by anyone other than the intended recipient, or an employee or agent responsible for delivering the message to the intended recipient, is prohibited. If you have received this e-mail in error, please immediately notify us by calling the Help Desk at (855) 237-8324.

Re: [Postgres-xc-general] Random failure when creating database

[鈴木 幸市 <ko...@in...>]

Yes, this has already been reported and fix is available at github repo, https://github.com/postgres-x2/postgres-x2

Please try REL1_2_STABLE or REL1_1_STABLE branch.   They are candidate for the next minor release.

Regards;
---
Koichi Suzuki
NTT DATA Intellilink Corp.
ko...@in...

> 2015/07/20 17:43、Artur Graniszewski <aa...@bo...> のメール：
> 
> Hi
> 
> I found a strange issue when creating the database in Postgres XC:
> 
> postgres=# create database bench2;
> WARNING:  unexpected EOF on datanode connection
> WARNING:  unexpected EOF on datanode connection
> ERROR:  sorry, too many clients already
> 
> This happens randomly so I can't easily reproduce this issue but sometimes
> after that error, I'm unable to access one of the datanodes because of "too
> many clients already" message (even though I'm the only user using these
> instances). This may be somehow connected to 
> 
> 
> This issue appears on the following version of XC:
> psql (PGXC 1.1, based on PG 9.2.4)
> 
> I use two datanodes&coordinators on separate machines and additional one
> with GTM only
> 
> Cheers,
> Artur
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> 

[Postgres-xc-general] Random failure when creating database

[Artur G. <aa...@bo...>]

Hi

I found a strange issue when creating the database in Postgres XC:

postgres=# create database bench2;
WARNING:  unexpected EOF on datanode connection
WARNING:  unexpected EOF on datanode connection
ERROR:  sorry, too many clients already

This happens randomly so I can't easily reproduce this issue but sometimes
after that error, I'm unable to access one of the datanodes because of "too
many clients already" message (even though I'm the only user using these
instances). This may be somehow connected to 


This issue appears on the following version of XC:
psql (PGXC 1.1, based on PG 9.2.4)

I use two datanodes&coordinators on separate machines and additional one
with GTM only

Cheers,
Artur

Re: [Postgres-xc-general] Uncommited transaction? Weird issues with DDL and missing database

[Koichi S. <koi...@gm...>]

To be honest, yes and unfortunately, it is common to XL.  Now XC has new
developer team and is planning to make next release this August.   This fix
is included in the plan too.   This fix can go to XL too.

Regards;



---
Koichi Suzuki
https://www.linkedin.com/in/koichidbms


2015-07-13 16:28 GMT+09:00 Artur Graniszewski <aa...@bo...>:

> Hi,
>
> does it mean that CREATE TABLE command may fail the same way? Is there any
> possibility to reduce the risk of such situation by for example setting
> preferred or primary node in Postgre XC configuration? (ALTER NODE
> command).
>
> Also, how often are the new versions (with bugfixes) of XC released? I
> know that fixing such bug could take a while, but I wonder if this fix
> won't be hindered by a release plan too..
>
> On Fri, 10 Jul 2015 13:13:30 +0900, Koichi Suzuki <koi...@gm...>
> wrote:
> > In PGXC, DDL operation is not serialized, I mean, the order of the node
> > where each peace of DDL executed is not defined.   This may cause some
> race
> > conditions and could cause local errors, resulting in inconsistent
> status.
> >
> > Recetly, I found CREATE TABLE in parallel schedule of regression test
> cause
> > similar problems.   I think this is common to XC and XL.
> >
> > There are a couple of ways to fix this:
> >
> > 1. Use coordinator primary node to go first, as done in replicated
> tables,
> > 2. Serialize DDL execution, possibly by using advisory lock internally.
> >
> > Anyway, this is a bug and we need a fix.   Solution 1 is bit more
> > complicated than 2.
> >
> > Best;
> >
> > ---
> > Koichi Suzuki
> > https://www.linkedin.com/in/koichidbms
> >
> >
> > 2015-07-09 23:22 GMT+09:00 Artur Graniszewski <aa...@bo...>:
> >
> >> Hi,
> >>
> >> I just finished the multi-master configuration of Postgres XC on my
> test
> >> environment (three virtual machines):
> >>
> >> * VM1 - with GTM only
> >> * VM2, VM2 - two masters with one data-node and one coordinator on each
> >> machine
> >>
> >> I tried to create some databases on each master to check if everything
> >> works ok, but I found a strange issue with one database that had been
> >> created only on one of the masters (while other databases I created and
> >> deleted afterwards were ok):
> >>
> >> postgres=# \l
> >>                                   List of databases
> >>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
> >> privileges
> >>
> >>
>
> -----------+----------+----------+-------------+-------------+-----------------------
> >>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> >>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> >> =c/postgres          +
> >>            |          |          |             |             |
> >> postgres=CTc/postgres
> >>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> >> =c/postgres          +
> >>            |          |          |             |             |
> >> postgres=CTc/postgres
> >>  test2     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> >>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> >>
> >> vs
> >>
> >> postgres=# \l
> >>                                   List of databases
> >>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
> >> privileges
> >>
> >>
>
> -----------+----------+----------+-------------+-------------+-----------------------
> >>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> >>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> >> =c/postgres          +
> >>            |          |          |             |             |
> >> postgres=CTc/postgres
> >>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> >> =c/postgres          +
> >>            |          |          |             |             |
> >> postgres=CTc/postgres
> >>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> >>
> >> Unfortunately, when I try to perform any DDL command on a server that
> >> sees
> >> "test2" database I receive the following errors:
> >>
> >> postgres=# drop database test2;
> >> ERROR:  database "test2" does not exist
> >> postgres=# create database test2;
> >> ERROR:  database "test2" already exists
> >>
> >> Also, when I try to remove another database called "whatnot", drop
> >> operation never finishes. It looks like a transaction that hangs some
> DDL
> >> operations:
> >>
> >> postgres=# select * from pgxc_prepared_xacts;
> >>  pgxc_prepared_xact
> >> --------------------
> >>  T74589
> >> (1 row)
> >>
> >> postgres=# select * from pg_prepared_xacts;
> >>  transaction |  gid   |           prepared            |  owner   |
> >> database
> >>
> >>
>
> -------------+--------+-------------------------------+----------+----------
> >>        73716 | T73716 | 2015-07-09 12:58:25.823385+02 | postgres |
> >>        postgres
> >> (1 row)
> >>
> >> postgres=#
> >>
> >> When I try to rollback the transaction, Postgres says that given GID
> does
> >> not exist.
> >>
> >> Is it a bug in Postgres XC? Is there a way to synchronize the masters?
> >> (eg
> >> delete "test2" on one master and "whatnot" on both of them). Please
> keep
> >> in
> >> mind that I created/dropped more databases without any issues, so I'm
> >> unsure what went wrong in case of these two specific databases.
> >>
> >> Cheers,
> >>
> >> Artur
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
>
> ------------------------------------------------------------------------------
> >> Don't Limit Your Business. Reach for the Cloud.
> >> GigeNET's Cloud Solutions provide you with the tools and support that
> >> you need to offload your IT needs and focus on growing your business.
> >> Configured For All Businesses. Start Your Cloud Today.
> >> https://www.gigenetcloud.com/
> >> _______________________________________________
> >> Postgres-xc-general mailing list
> >> Pos...@li...
> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >>
> >>
>

Re: [Postgres-xc-general] Uncommited transaction? Weird issues with DDL and missing database

[Artur G. <aa...@bo...>]

Hi,

does it mean that CREATE TABLE command may fail the same way? Is there any
possibility to reduce the risk of such situation by for example setting
preferred or primary node in Postgre XC configuration? (ALTER NODE
command).

Also, how often are the new versions (with bugfixes) of XC released? I
know that fixing such bug could take a while, but I wonder if this fix
won't be hindered by a release plan too..

On Fri, 10 Jul 2015 13:13:30 +0900, Koichi Suzuki <koi...@gm...>
wrote:
> In PGXC, DDL operation is not serialized, I mean, the order of the node
> where each peace of DDL executed is not defined.   This may cause some
race
> conditions and could cause local errors, resulting in inconsistent
status.
> 
> Recetly, I found CREATE TABLE in parallel schedule of regression test
cause
> similar problems.   I think this is common to XC and XL.
> 
> There are a couple of ways to fix this:
> 
> 1. Use coordinator primary node to go first, as done in replicated
tables,
> 2. Serialize DDL execution, possibly by using advisory lock internally.
> 
> Anyway, this is a bug and we need a fix.   Solution 1 is bit more
> complicated than 2.
> 
> Best;
> 
> ---
> Koichi Suzuki
> https://www.linkedin.com/in/koichidbms
> 
> 
> 2015-07-09 23:22 GMT+09:00 Artur Graniszewski <aa...@bo...>:
> 
>> Hi,
>>
>> I just finished the multi-master configuration of Postgres XC on my
test
>> environment (three virtual machines):
>>
>> * VM1 - with GTM only
>> * VM2, VM2 - two masters with one data-node and one coordinator on each
>> machine
>>
>> I tried to create some databases on each master to check if everything
>> works ok, but I found a strange issue with one database that had been
>> created only on one of the masters (while other databases I created and
>> deleted afterwards were ok):
>>
>> postgres=# \l
>>                                   List of databases
>>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
>> privileges
>>
>>
-----------+----------+----------+-------------+-------------+-----------------------
>>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres          +
>>            |          |          |             |             |
>> postgres=CTc/postgres
>>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres          +
>>            |          |          |             |             |
>> postgres=CTc/postgres
>>  test2     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>
>> vs
>>
>> postgres=# \l
>>                                   List of databases
>>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
>> privileges
>>
>>
-----------+----------+----------+-------------+-------------+-----------------------
>>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres          +
>>            |          |          |             |             |
>> postgres=CTc/postgres
>>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres          +
>>            |          |          |             |             |
>> postgres=CTc/postgres
>>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>
>> Unfortunately, when I try to perform any DDL command on a server that
>> sees
>> "test2" database I receive the following errors:
>>
>> postgres=# drop database test2;
>> ERROR:  database "test2" does not exist
>> postgres=# create database test2;
>> ERROR:  database "test2" already exists
>>
>> Also, when I try to remove another database called "whatnot", drop
>> operation never finishes. It looks like a transaction that hangs some
DDL
>> operations:
>>
>> postgres=# select * from pgxc_prepared_xacts;
>>  pgxc_prepared_xact
>> --------------------
>>  T74589
>> (1 row)
>>
>> postgres=# select * from pg_prepared_xacts;
>>  transaction |  gid   |           prepared            |  owner   |
>> database
>>
>>
-------------+--------+-------------------------------+----------+----------
>>        73716 | T73716 | 2015-07-09 12:58:25.823385+02 | postgres |
>>        postgres
>> (1 row)
>>
>> postgres=#
>>
>> When I try to rollback the transaction, Postgres says that given GID
does
>> not exist.
>>
>> Is it a bug in Postgres XC? Is there a way to synchronize the masters?
>> (eg
>> delete "test2" on one master and "whatnot" on both of them). Please
keep
>> in
>> mind that I created/dropped more databases without any issues, so I'm
>> unsure what went wrong in case of these two specific databases.
>>
>> Cheers,
>>
>> Artur
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
------------------------------------------------------------------------------
>> Don't Limit Your Business. Reach for the Cloud.
>> GigeNET's Cloud Solutions provide you with the tools and support that
>> you need to offload your IT needs and focus on growing your business.
>> Configured For All Businesses. Start Your Cloud Today.
>> https://www.gigenetcloud.com/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>

Re: [Postgres-xc-general] Uncommited transaction? Weird issues with DDL and missing database

[Koichi S. <koi...@gm...>]

Yes, I agree.   I began to write a code for this.

In creating/dropping nodes, transaction level lock is used.   Because some
DDL don’t run inside the session block, I think we should use session
level, although we need some error handling when the transaction aborts.

Thank you;

---
Koichi Suzuki
https://www.linkedin.com/in/koichidbms


2015-07-10 14:11 GMT+09:00 Ashutosh Bapat <ash...@en...>:

> I vaguely remember, Abbas had used advisory lock machinary for creating
> and dropping nodes. May be we have to use the same mechanism for all DDLs.
>
> On Fri, Jul 10, 2015 at 9:43 AM, Koichi Suzuki <koi...@gm...>
> wrote:
>
>> In PGXC, DDL operation is not serialized, I mean, the order of the node
>> where each peace of DDL executed is not defined.   This may cause some race
>> conditions and could cause local errors, resulting in inconsistent status.
>>
>> Recetly, I found CREATE TABLE in parallel schedule of regression test
>> cause similar problems.   I think this is common to XC and XL.
>>
>> There are a couple of ways to fix this:
>>
>> 1. Use coordinator primary node to go first, as done in replicated tables,
>> 2. Serialize DDL execution, possibly by using advisory lock internally.
>>
>> Anyway, this is a bug and we need a fix.   Solution 1 is bit more
>> complicated than 2.
>>
>> Best;
>>
>> ---
>> Koichi Suzuki
>> https://www.linkedin.com/in/koichidbms
>>
>>
>> 2015-07-09 23:22 GMT+09:00 Artur Graniszewski <aa...@bo...>:
>>
>>> Hi,
>>>
>>> I just finished the multi-master configuration of Postgres XC on my test
>>> environment (three virtual machines):
>>>
>>> * VM1 - with GTM only
>>> * VM2, VM2 - two masters with one data-node and one coordinator on each
>>> machine
>>>
>>> I tried to create some databases on each master to check if everything
>>> works ok, but I found a strange issue with one database that had been
>>> created only on one of the masters (while other databases I created and
>>> deleted afterwards were ok):
>>>
>>> postgres=# \l
>>>                                   List of databases
>>>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
>>> privileges
>>>
>>> -----------+----------+----------+-------------+-------------+-----------------------
>>>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>> =c/postgres          +
>>>            |          |          |             |             |
>>> postgres=CTc/postgres
>>>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>> =c/postgres          +
>>>            |          |          |             |             |
>>> postgres=CTc/postgres
>>>  test2     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>>
>>> vs
>>>
>>> postgres=# \l
>>>                                   List of databases
>>>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
>>> privileges
>>>
>>> -----------+----------+----------+-------------+-------------+-----------------------
>>>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>> =c/postgres          +
>>>            |          |          |             |             |
>>> postgres=CTc/postgres
>>>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>> =c/postgres          +
>>>            |          |          |             |             |
>>> postgres=CTc/postgres
>>>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>>
>>> Unfortunately, when I try to perform any DDL command on a server that
>>> sees "test2" database I receive the following errors:
>>>
>>> postgres=# drop database test2;
>>> ERROR:  database "test2" does not exist
>>> postgres=# create database test2;
>>> ERROR:  database "test2" already exists
>>>
>>> Also, when I try to remove another database called "whatnot", drop
>>> operation never finishes. It looks like a transaction that hangs some DDL
>>> operations:
>>>
>>> postgres=# select * from pgxc_prepared_xacts;
>>>  pgxc_prepared_xact
>>> --------------------
>>>  T74589
>>> (1 row)
>>>
>>> postgres=# select * from pg_prepared_xacts;
>>>  transaction |  gid   |           prepared            |  owner   |
>>> database
>>>
>>> -------------+--------+-------------------------------+----------+----------
>>>        73716 | T73716 | 2015-07-09 12:58:25.823385+02 | postgres |
>>> postgres
>>> (1 row)
>>>
>>> postgres=#
>>>
>>> When I try to rollback the transaction, Postgres says that given GID
>>> does not exist.
>>>
>>> Is it a bug in Postgres XC? Is there a way to synchronize the masters?
>>> (eg delete "test2" on one master and "whatnot" on both of them). Please
>>> keep in mind that I created/dropped more databases without any issues, so
>>> I'm unsure what went wrong in case of these two specific databases.
>>>
>>> Cheers,
>>>
>>> Artur
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Don't Limit Your Business. Reach for the Cloud.
>>> GigeNET's Cloud Solutions provide you with the tools and support that
>>> you need to offload your IT needs and focus on growing your business.
>>> Configured For All Businesses. Start Your Cloud Today.
>>> https://www.gigenetcloud.com/
>>> _______________________________________________
>>> Postgres-xc-general mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Don't Limit Your Business. Reach for the Cloud.
>> GigeNET's Cloud Solutions provide you with the tools and support that
>> you need to offload your IT needs and focus on growing your business.
>> Configured For All Businesses. Start Your Cloud Today.
>> https://www.gigenetcloud.com/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>
>
> --
> Best Wishes,
> Ashutosh Bapat
> EnterpriseDB Corporation
> The Postgres Database Company
>

Re: [Postgres-xc-general] Uncommited transaction? Weird issues with DDL and missing database

[Ashutosh B. <ash...@en...>]

I vaguely remember, Abbas had used advisory lock machinary for creating and
dropping nodes. May be we have to use the same mechanism for all DDLs.

On Fri, Jul 10, 2015 at 9:43 AM, Koichi Suzuki <koi...@gm...>
wrote:

> In PGXC, DDL operation is not serialized, I mean, the order of the node
> where each peace of DDL executed is not defined.   This may cause some race
> conditions and could cause local errors, resulting in inconsistent status.
>
> Recetly, I found CREATE TABLE in parallel schedule of regression test
> cause similar problems.   I think this is common to XC and XL.
>
> There are a couple of ways to fix this:
>
> 1. Use coordinator primary node to go first, as done in replicated tables,
> 2. Serialize DDL execution, possibly by using advisory lock internally.
>
> Anyway, this is a bug and we need a fix.   Solution 1 is bit more
> complicated than 2.
>
> Best;
>
> ---
> Koichi Suzuki
> https://www.linkedin.com/in/koichidbms
>
>
> 2015-07-09 23:22 GMT+09:00 Artur Graniszewski <aa...@bo...>:
>
>> Hi,
>>
>> I just finished the multi-master configuration of Postgres XC on my test
>> environment (three virtual machines):
>>
>> * VM1 - with GTM only
>> * VM2, VM2 - two masters with one data-node and one coordinator on each
>> machine
>>
>> I tried to create some databases on each master to check if everything
>> works ok, but I found a strange issue with one database that had been
>> created only on one of the masters (while other databases I created and
>> deleted afterwards were ok):
>>
>> postgres=# \l
>>                                   List of databases
>>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
>> privileges
>>
>> -----------+----------+----------+-------------+-------------+-----------------------
>>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres          +
>>            |          |          |             |             |
>> postgres=CTc/postgres
>>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres          +
>>            |          |          |             |             |
>> postgres=CTc/postgres
>>  test2     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>
>> vs
>>
>> postgres=# \l
>>                                   List of databases
>>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
>> privileges
>>
>> -----------+----------+----------+-------------+-------------+-----------------------
>>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres          +
>>            |          |          |             |             |
>> postgres=CTc/postgres
>>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>> =c/postgres          +
>>            |          |          |             |             |
>> postgres=CTc/postgres
>>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>>
>> Unfortunately, when I try to perform any DDL command on a server that
>> sees "test2" database I receive the following errors:
>>
>> postgres=# drop database test2;
>> ERROR:  database "test2" does not exist
>> postgres=# create database test2;
>> ERROR:  database "test2" already exists
>>
>> Also, when I try to remove another database called "whatnot", drop
>> operation never finishes. It looks like a transaction that hangs some DDL
>> operations:
>>
>> postgres=# select * from pgxc_prepared_xacts;
>>  pgxc_prepared_xact
>> --------------------
>>  T74589
>> (1 row)
>>
>> postgres=# select * from pg_prepared_xacts;
>>  transaction |  gid   |           prepared            |  owner   |
>> database
>>
>> -------------+--------+-------------------------------+----------+----------
>>        73716 | T73716 | 2015-07-09 12:58:25.823385+02 | postgres |
>> postgres
>> (1 row)
>>
>> postgres=#
>>
>> When I try to rollback the transaction, Postgres says that given GID does
>> not exist.
>>
>> Is it a bug in Postgres XC? Is there a way to synchronize the masters?
>> (eg delete "test2" on one master and "whatnot" on both of them). Please
>> keep in mind that I created/dropped more databases without any issues, so
>> I'm unsure what went wrong in case of these two specific databases.
>>
>> Cheers,
>>
>> Artur
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Don't Limit Your Business. Reach for the Cloud.
>> GigeNET's Cloud Solutions provide you with the tools and support that
>> you need to offload your IT needs and focus on growing your business.
>> Configured For All Businesses. Start Your Cloud Today.
>> https://www.gigenetcloud.com/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>>
>>
>
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>


-- 
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company

Re: [Postgres-xc-general] Uncommited transaction? Weird issues with DDL and missing database

[Koichi S. <koi...@gm...>]

In PGXC, DDL operation is not serialized, I mean, the order of the node
where each peace of DDL executed is not defined.   This may cause some race
conditions and could cause local errors, resulting in inconsistent status.

Recetly, I found CREATE TABLE in parallel schedule of regression test cause
similar problems.   I think this is common to XC and XL.

There are a couple of ways to fix this:

1. Use coordinator primary node to go first, as done in replicated tables,
2. Serialize DDL execution, possibly by using advisory lock internally.

Anyway, this is a bug and we need a fix.   Solution 1 is bit more
complicated than 2.

Best;

---
Koichi Suzuki
https://www.linkedin.com/in/koichidbms


2015-07-09 23:22 GMT+09:00 Artur Graniszewski <aa...@bo...>:

> Hi,
>
> I just finished the multi-master configuration of Postgres XC on my test
> environment (three virtual machines):
>
> * VM1 - with GTM only
> * VM2, VM2 - two masters with one data-node and one coordinator on each
> machine
>
> I tried to create some databases on each master to check if everything
> works ok, but I found a strange issue with one database that had been
> created only on one of the masters (while other databases I created and
> deleted afterwards were ok):
>
> postgres=# \l
>                                   List of databases
>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
> privileges
>
> -----------+----------+----------+-------------+-------------+-----------------------
>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> =c/postgres          +
>            |          |          |             |             |
> postgres=CTc/postgres
>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> =c/postgres          +
>            |          |          |             |             |
> postgres=CTc/postgres
>  test2     | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>
> vs
>
> postgres=# \l
>                                   List of databases
>    Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access
> privileges
>
> -----------+----------+----------+-------------+-------------+-----------------------
>  postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>  template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> =c/postgres          +
>            |          |          |             |             |
> postgres=CTc/postgres
>  template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
> =c/postgres          +
>            |          |          |             |             |
> postgres=CTc/postgres
>  whatnot   | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
>
> Unfortunately, when I try to perform any DDL command on a server that sees
> "test2" database I receive the following errors:
>
> postgres=# drop database test2;
> ERROR:  database "test2" does not exist
> postgres=# create database test2;
> ERROR:  database "test2" already exists
>
> Also, when I try to remove another database called "whatnot", drop
> operation never finishes. It looks like a transaction that hangs some DDL
> operations:
>
> postgres=# select * from pgxc_prepared_xacts;
>  pgxc_prepared_xact
> --------------------
>  T74589
> (1 row)
>
> postgres=# select * from pg_prepared_xacts;
>  transaction |  gid   |           prepared            |  owner   |
> database
>
> -------------+--------+-------------------------------+----------+----------
>        73716 | T73716 | 2015-07-09 12:58:25.823385+02 | postgres | postgres
> (1 row)
>
> postgres=#
>
> When I try to rollback the transaction, Postgres says that given GID does
> not exist.
>
> Is it a bug in Postgres XC? Is there a way to synchronize the masters? (eg
> delete "test2" on one master and "whatnot" on both of them). Please keep in
> mind that I created/dropped more databases without any issues, so I'm
> unsure what went wrong in case of these two specific databases.
>
> Cheers,
>
> Artur
>
>
>
>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Don't Limit Your Business. Reach for the Cloud.
> GigeNET's Cloud Solutions provide you with the tools and support that
> you need to offload your IT needs and focus on growing your business.
> Configured For All Businesses. Start Your Cloud Today.
> https://www.gigenetcloud.com/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>