Menu
▾
▴
postgres-xc-developers — Postgres-XC hackers and developers
You can subscribe to this list here.
| 2010 |
Jan
|
Feb
|
Mar
|
Apr
(10) |
May
(17) |
Jun
(3) |
Jul
|
Aug
|
Sep
(8) |
Oct
(18) |
Nov
(51) |
Dec
(74) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2011 |
Jan
(47) |
Feb
(44) |
Mar
(44) |
Apr
(102) |
May
(35) |
Jun
(25) |
Jul
(56) |
Aug
(69) |
Sep
(32) |
Oct
(37) |
Nov
(31) |
Dec
(16) |
| 2012 |
Jan
(34) |
Feb
(127) |
Mar
(218) |
Apr
(252) |
May
(80) |
Jun
(137) |
Jul
(205) |
Aug
(159) |
Sep
(35) |
Oct
(50) |
Nov
(82) |
Dec
(52) |
| 2013 |
Jan
(107) |
Feb
(159) |
Mar
(118) |
Apr
(163) |
May
(151) |
Jun
(89) |
Jul
(106) |
Aug
(177) |
Sep
(49) |
Oct
(63) |
Nov
(46) |
Dec
(7) |
| 2014 |
Jan
(65) |
Feb
(128) |
Mar
(40) |
Apr
(11) |
May
(4) |
Jun
(8) |
Jul
(16) |
Aug
(11) |
Sep
(4) |
Oct
(1) |
Nov
(5) |
Dec
(16) |
| 2015 |
Jan
(5) |
Feb
|
Mar
(2) |
Apr
(5) |
May
(4) |
Jun
(12) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(4) |
| 2019 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: 鈴木 幸市 <ko...@in...> - 2014-02-14 04:20:03
|
Not really although there will be no feature addition in 1.1. I will keep maintaining 1.1 and 1.0 for a while. As I wrote in the note, 1.2 needs at least one improvement of update/delete in replicated tables. It will be good to begin 1.2 work after GA unless you’d like to practice on it. Regards; --- Koichi Suzuki 2014/02/14 13:12、David E. Wheeler <da...@ju...> のメール： > On Feb 13, 2014, at 5:30 PM, Abbas Butt <abb...@en...> wrote: > >> Would it be possible for you to base the release on 1.2 instead? >> We have just released 1.2 beta and would soon be going GA. >> I have tested that the case works fine on 1.2. > > I can start working on a 1.2 build soon, yeah. So is 1.1 end-of-lifed? No more work to be done on it? > > Thanks, > > David > > > ------------------------------------------------------------------------------ > Android apps run on BlackBerry 10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > |
|
From: David E. W. <da...@ju...> - 2014-02-14 04:12:54
|
On Feb 13, 2014, at 5:30 PM, Abbas Butt <abb...@en...> wrote: > Would it be possible for you to base the release on 1.2 instead? > We have just released 1.2 beta and would soon be going GA. > I have tested that the case works fine on 1.2. I can start working on a 1.2 build soon, yeah. So is 1.1 end-of-lifed? No more work to be done on it? Thanks, David |
|
From: Koichi S. <koi...@gm...> - 2014-02-14 02:07:21
|
I misunderstand the implication. Anyway additional wait is separate from your suggestion. Disconnecting the connection as you suggested will bring another problem such as TEMPORARY object in the subsequent queries. We do not support TEMPORARY object but I believe we should be consistent on this for future releases. Thoughts? --- Koichi Suzuki 2014-02-14 2:30 GMT+09:00 Andrei Martsinchyk <and...@gm...>: > Hello, > > Postgres establishes separate connection to deliver Cancel command to the > target session. > On a heavily loaded node it may take fairly long. Longer sleep would help > out, but it means longer recovery after an error. > Better solution is to remove canceled connection from the pool and therefore > do not use it to handle subsequent queries. > > > > 2014-02-13 11:10 GMT+02:00 Koichi Suzuki <koi...@gm...>: >> >> I think it hits the point. I tested this patch several times and it >> seems to work fine. The delay time (at present 10ms) is short enough >> and it is applied only when we need to cancel a statement. >> >> We should check this into all the master and STABLE branches improving >> magic number with some meaningful name. >> >> Any thoughts? >> --- >> Koichi Suzuki >> >> >> 2014-01-24 18:25 GMT+09:00 Masataka Saito <pg...@gm...>: >> > Hello, >> > >> > As I've been exasperated by random failures, I'm willing to whip the >> > cause >> > of the issue. >> > >> > This issue is related to cancel of the failed query. >> > When a datanode reports an error of a query, a coordinator sends a >> > cancel >> > request to non-idle nodes, waits the node to get ready and requests >> > nodes to >> > rollback the transaction. >> > >> > Where's the problem? Consider the next case. >> > 1. Datanode A (PID 1) reports an error to coordinator A. ([1] 'E' >> > message) >> > 2. Coordinator A receives [1] and reports an error to a frontend. ([2] >> > 'E' >> > message) >> > 3. Coordinator A starts aborting process and it thinks datanode A (PID >> > 1) is >> > not idle. >> > 4. Coordinator A sends a cancel request about PID 1 to datanode A (PID >> > 2). >> > ([3] cancel message) >> > 5. Datanode A (PID 1) reports ready to coordinator A. ([4] 'Z' message) >> > 6. Coordinator A receives [4] and sends "ROLLBACK TRANSACTION" >> > immediately. >> > ([5] 'Q' message) >> > 7. Datanode A (PID 1) receives [5] and starts processing the query. >> > 8. Datanode A (PID 2) receives [3]. >> > 9. Datanode A (PID 2) notify PID 1 of [3]. >> > 10. Datanode A (PID 1) cancel processing [5] and reports an error to >> > Coordinator A. ([6] 'E' message) >> > 11. Coordinator A receives [6] and reports an error to a frontend. ([7] >> > 'E' >> > message) >> > >> > [7] makes unexpected output and a test fails. >> > >> > Saying an extreme thing, it could occur that the next query of [5] is >> > cancelled by [3]. >> > >> > As far as I know, there's no way to know when to the cancel request get >> > to >> > be processed, I think we can't not wait an experimental duration after >> > cancelling like the attached patch. >> > >> > Does anyone have another cool idea to solve this issue? >> > >> > Regards. >> > >> > >> > ------------------------------------------------------------------------------ >> > CenturyLink Cloud: The Leader in Enterprise Cloud Services. >> > Learn Why More Businesses Are Choosing CenturyLink Cloud For >> > Critical Workloads, Development Environments & Everything In Between. >> > Get a Quote or Start a Free Trial Today. >> > >> > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk >> > _______________________________________________ >> > Postgres-xc-developers mailing list >> > Pos...@li... >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> > >> >> >> ------------------------------------------------------------------------------ >> Android apps run on BlackBerry 10 >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> Get your Android app in front of a whole new audience. Start now. >> >> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk >> >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > -- > Andrei Martsinchyk > > StormDB - http://www.stormdb.com > The Database Cloud > |
|
From: Koichi S. <koi...@gm...> - 2014-02-14 02:04:30
|
In this case, we can use these conditions as WHERE clause in usual UPDATE/DELETE statements, which we cannot somehow now. --- Koichi Suzuki 2014-02-14 10:46 GMT+09:00 Abbas Butt <abb...@en...>: > > > > On Fri, Feb 14, 2014 at 6:09 AM, Koichi Suzuki <koi...@gm...> > wrote: >> >> I don't think it works because there're no guarantee each cursor >> against different datanode returns rows in the same order. > > > Why would a query with an ORDER BY clause on all the columns not return the > rows in the same order on each datanode? > >> >> >> Regards; >> --- >> Koichi Suzuki >> >> >> 2014-02-13 15:50 GMT+09:00 Ashutosh Bapat >> <ash...@en...>: >> > WCO on datanodes should be working fine, so that shouldn't be a problem. >> > >> > >> > On Thu, Feb 13, 2014 at 12:13 PM, 鈴木 幸市 <ko...@in...> >> > wrote: >> >> >> >> Are you sure that this does not come up with any bad side effects to >> >> support WCO in 1.3? >> >> --- >> >> Koichi Suzuki >> >> >> >> 2014/02/13 15:24、Ashutosh Bapat <ash...@en...> のメール： >> >> >> >> One more solution would be to use cursors for replicated tables. The >> >> idea >> >> is to open cursors on all the copies of the table and append the query >> >> with >> >> an ORDER BY clause on all the columns. Thus we are sure that the >> >> current of >> >> each of these cursors point to same row on all the copies. While >> >> fetching a >> >> row from a replicated table, we fetch from all the cursors and choose >> >> only >> >> one row for the data processing. While updating or deleting we send >> >> UPDATE >> >> or DELETE with WHERE CURRENT OF. The down side of this approach is >> >> that, if >> >> there are coordinator quals, we will end up locking more rows than >> >> necessary, increasing the probability of the deadlock but at least >> >> there >> >> won't be a necessary restriction of having primary or unique key and we >> >> won't break backward compatibility. >> >> >> >> If there two identical rows, we might mix the update from different >> >> nodes, >> >> but then who knew which of them were corresponded across the nodes to >> >> start >> >> with. >> >> >> >> >> >> On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki <koi...@gm...> >> >> wrote: >> >>> >> >>> Hi, >> >>> >> >>> I tested the patch and found that primary key is mandatory. We need >> >>> to modify regression test considerably to give each replicated table >> >>> primary keys. >> >>> >> >>> I think this patch helps but I'm not afraid this is good, especially >> >>> when we try to take XC features back to PG. >> >>> >> >>> Did you post another patch to use all column values if primary key is >> >>> not available? >> >>> >> >>> I think better way is as follows: >> >>> >> >>> 1) If primary key is defined, use it, >> >>> 2) If not, create a primary key as system column, the size should be >> >>> 64bit. >> >>> 3) If primary key is added to a replicated table, remove system >> >>> primary >> >>> key. >> >>> >> >>> The value of primary key can be obtained as follows: >> >>> >> >>> 1) add new column to pgxc_class catalog to represent maximum value of >> >>> the system primary key, >> >>> 2) when first "insert" is done to the primary node, system primary key >> >>> value is taken from 1) and 1) is updated. The value is returned to >> >>> the coordinator to be propagated to other nodes. >> >>> 3) when subsequent "insert" is being done, system primary key value is >> >>> added to the column value. In this case, each datanode updates 1) >> >>> column value if it is larger than the current maximum value. >> >>> >> >>> 3) is important to change primary node to another. This is needed to >> >>> carry over the primary node to another. >> >>> >> >>> ALTER TABLE should take care of them. >> >>> >> >>> Other issues are: >> >>> >> >>> 4) pg_dump/pg_dumpall should not include this system column value, >> >>> 5) cluster may need to handle this too to repack system primary key >> >>> value (not now but at least in 1.3 or later). >> >>> >> >>> Regards; >> >>> --- >> >>> Koichi Suzuki >> >>> >> >>> >> >>> 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>: >> >>> > Please see attached patch that tries to address the issue of XC >> >>> > using >> >>> > CTID >> >>> > for replicated updates and deletes when it is evaluated at a >> >>> > coordinator >> >>> > instead of being pushed down. >> >>> > >> >>> > The problem here is that CTID could be referring to a different >> >>> > tuple >> >>> > altogether on a different data node, which is what happened for one >> >>> > of >> >>> > our >> >>> > Postgres-XC support customers, leading to data issues. >> >>> > >> >>> > Instead, the patch looks for a primary key or unique index (with the >> >>> > primary >> >>> > key preferred) and uses those values instead of CTID. >> >>> > >> >>> > The patch could be improved further. Extra parameters are set even >> >>> > if >> >>> > not >> >>> > used in the execution of the prepared statement sent down to the >> >>> > data >> >>> > nodes. >> >>> > >> >>> > Regards, >> >>> > >> >>> > >> >>> > -- >> >>> > Mason Sharp >> >>> > >> >>> > TransLattice - http://www.translattice.com >> >>> > Distributed and Clustered Database Solutions >> >>> > >> >>> > >> >>> > >> >>> > ------------------------------------------------------------------------------ >> >>> > November Webinars for C, C++, Fortran Developers >> >>> > Accelerate application performance with scalable programming models. >> >>> > Explore >> >>> > techniques for threading, error checking, porting, and tuning. Get >> >>> > the >> >>> > most >> >>> > from the latest Intel processors and coprocessors. See abstracts and >> >>> > register >> >>> > >> >>> > >> >>> > http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk >> >>> > _______________________________________________ >> >>> > Postgres-xc-developers mailing list >> >>> > Pos...@li... >> >>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >>> > >> >>> >> >>> >> >>> >> >>> ------------------------------------------------------------------------------ >> >>> Android apps run on BlackBerry 10 >> >>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> >>> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> >>> Get your Android app in front of a whole new audience. Start now. >> >>> >> >>> >> >>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk >> >>> _______________________________________________ >> >>> Postgres-xc-developers mailing list >> >>> Pos...@li... >> >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> >> >> >> >> >> >> >> >> -- >> >> Best Wishes, >> >> Ashutosh Bapat >> >> EnterpriseDB Corporation >> >> The Postgres Database Company >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> Android apps run on BlackBerry 10 >> >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> >> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> >> Get your Android app in front of a whole new audience. Start now. >> >> >> >> >> >> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ >> >> Postgres-xc-developers mailing list >> >> Pos...@li... >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> >> >> >> > >> > >> > >> > -- >> > Best Wishes, >> > Ashutosh Bapat >> > EnterpriseDB Corporation >> > The Postgres Database Company >> >> >> ------------------------------------------------------------------------------ >> Android apps run on BlackBerry 10 >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> Get your Android app in front of a whole new audience. Start now. >> >> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > -- > -- > Abbas > Architect > > Ph: 92.334.5100153 > Skype ID: gabbasb > www.enterprisedb.com > > Follow us on Twitter > @EnterpriseDB > > Visit EnterpriseDB for tutorials, webinars, whitepapers and more |
|
From: Abbas B. <abb...@en...> - 2014-02-14 01:55:15
|
On Thu, Feb 13, 2014 at 11:24 AM, Ashutosh Bapat < ash...@en...> wrote: > One more solution would be to use cursors for replicated tables. The idea > is to open cursors on all the copies of the table and append the query with > an ORDER BY clause on all the columns. Thus we are sure that the current of > each of these cursors point to same row on all the copies. While fetching a > row from a replicated table, we fetch from all the cursors and choose only > one row for the data processing. While updating or deleting we send UPDATE > or DELETE with WHERE CURRENT OF. The down side of this approach is that, if > there are coordinator quals, we will end up locking more rows than > necessary, increasing the probability of the deadlock but at least there > won't be a necessary restriction of having primary or unique key and we > won't break backward compatibility. > > If there two identical rows, we might mix the update from different nodes, > but then who knew which of them were corresponded across the nodes to start > with. > Thanks for the suggestion but we currently do not support WCO and we were thinking of fixing this issue before we declare 1.2 beta is generally available. > > > On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki <koi...@gm...>wrote: > >> Hi, >> >> I tested the patch and found that primary key is mandatory. We need >> to modify regression test considerably to give each replicated table >> primary keys. >> >> I think this patch helps but I'm not afraid this is good, especially >> when we try to take XC features back to PG. >> >> Did you post another patch to use all column values if primary key is >> not available? >> >> I think better way is as follows: >> >> 1) If primary key is defined, use it, >> 2) If not, create a primary key as system column, the size should be >> 64bit. >> 3) If primary key is added to a replicated table, remove system primary >> key. >> >> The value of primary key can be obtained as follows: >> >> 1) add new column to pgxc_class catalog to represent maximum value of >> the system primary key, >> 2) when first "insert" is done to the primary node, system primary key >> value is taken from 1) and 1) is updated. The value is returned to >> the coordinator to be propagated to other nodes. >> 3) when subsequent "insert" is being done, system primary key value is >> added to the column value. In this case, each datanode updates 1) >> column value if it is larger than the current maximum value. >> >> 3) is important to change primary node to another. This is needed to >> carry over the primary node to another. >> >> ALTER TABLE should take care of them. >> >> Other issues are: >> >> 4) pg_dump/pg_dumpall should not include this system column value, >> 5) cluster may need to handle this too to repack system primary key >> value (not now but at least in 1.3 or later). >> >> Regards; >> --- >> Koichi Suzuki >> >> >> 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>: >> > Please see attached patch that tries to address the issue of XC using >> CTID >> > for replicated updates and deletes when it is evaluated at a coordinator >> > instead of being pushed down. >> > >> > The problem here is that CTID could be referring to a different tuple >> > altogether on a different data node, which is what happened for one of >> our >> > Postgres-XC support customers, leading to data issues. >> > >> > Instead, the patch looks for a primary key or unique index (with the >> primary >> > key preferred) and uses those values instead of CTID. >> > >> > The patch could be improved further. Extra parameters are set even if >> not >> > used in the execution of the prepared statement sent down to the data >> nodes. >> > >> > Regards, >> > >> > >> > -- >> > Mason Sharp >> > >> > TransLattice - http://www.translattice.com >> > Distributed and Clustered Database Solutions >> > >> > >> ------------------------------------------------------------------------------ >> > November Webinars for C, C++, Fortran Developers >> > Accelerate application performance with scalable programming models. >> Explore >> > techniques for threading, error checking, porting, and tuning. Get the >> most >> > from the latest Intel processors and coprocessors. See abstracts and >> > register >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk >> > _______________________________________________ >> > Postgres-xc-developers mailing list >> > Pos...@li... >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> > >> >> >> ------------------------------------------------------------------------------ >> Android apps run on BlackBerry 10 >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> Get your Android app in front of a whole new audience. Start now. >> >> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> > > > > -- > Best Wishes, > Ashutosh Bapat > EnterpriseDB Corporation > The Postgres Database Company > > > ------------------------------------------------------------------------------ > Android apps run on BlackBerry 10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > -- -- *Abbas* Architect Ph: 92.334.5100153 Skype ID: gabbasb www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/> *Follow us on Twitter* @EnterpriseDB Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community> |
|
From: Abbas B. <abb...@en...> - 2014-02-14 01:51:29
|
On Thu, Feb 13, 2014 at 9:15 AM, Koichi Suzuki <koi...@gm...>wrote:
> Hi,
>
> I tested the patch and found that primary key is mandatory. We need
> to modify regression test considerably to give each replicated table
> primary keys.
>
> I think this patch helps but I'm not afraid this is good, especially
> when we try to take XC features back to PG.
>
> Did you post another patch to use all column values if primary key is
> not available?
>
> I think better way is as follows:
>
> 1) If primary key is defined, use it,
> 2) If not, create a primary key as system column, the size should be 64bit.
> 3) If primary key is added to a replicated table, remove system primary
> key.
>
> The value of primary key can be obtained as follows:
>
> 1) add new column to pgxc_class catalog to represent maximum value of
> the system primary key,
> 2) when first "insert" is done to the primary node, system primary key
> value is taken from 1) and 1) is updated. The value is returned to
> the coordinator to be propagated to other nodes.
> 3) when subsequent "insert" is being done, system primary key value is
> added to the column value. In this case, each datanode updates 1)
> column value if it is larger than the current maximum value.
>
> 3) is important to change primary node to another. This is needed to
> carry over the primary node to another.
>
Otherwise the added primary key could be of type serial, with default
nextval('sequence_type') right?
>
> ALTER TABLE should take care of them.
>
> Other issues are:
>
> 4) pg_dump/pg_dumpall should not include this system column value,
> 5) cluster may need to handle this too to repack system primary key
> value (not now but at least in 1.3 or later).
>
> Regards;
> ---
> Koichi Suzuki
>
>
> 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>:
> > Please see attached patch that tries to address the issue of XC using
> CTID
> > for replicated updates and deletes when it is evaluated at a coordinator
> > instead of being pushed down.
> >
> > The problem here is that CTID could be referring to a different tuple
> > altogether on a different data node, which is what happened for one of
> our
> > Postgres-XC support customers, leading to data issues.
> >
> > Instead, the patch looks for a primary key or unique index (with the
> primary
> > key preferred) and uses those values instead of CTID.
> >
> > The patch could be improved further. Extra parameters are set even if
> not
> > used in the execution of the prepared statement sent down to the data
> nodes.
> >
> > Regards,
> >
> >
> > --
> > Mason Sharp
> >
> > TransLattice - http://www.translattice.com
> > Distributed and Clustered Database Solutions
> >
> >
> ------------------------------------------------------------------------------
> > November Webinars for C, C++, Fortran Developers
> > Accelerate application performance with scalable programming models.
> Explore
> > techniques for threading, error checking, porting, and tuning. Get the
> most
> > from the latest Intel processors and coprocessors. See abstracts and
> > register
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Postgres-xc-developers mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
> >
>
>
> ------------------------------------------------------------------------------
> Android apps run on BlackBerry 10
> Introducing the new BlackBerry 10.2.1 Runtime for Android apps.
> Now with support for Jelly Bean, Bluetooth, Mapview and more.
> Get your Android app in front of a whole new audience. Start now.
>
> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>
--
--
*Abbas*
Architect
Ph: 92.334.5100153
Skype ID: gabbasb
www.enterprisedb.co
<http://www.enterprisedb.com/>m<http://www.enterprisedb.com/>
*Follow us on Twitter*
@EnterpriseDB
Visit EnterpriseDB for tutorials, webinars,
whitepapers<http://www.enterprisedb.com/resources-community>and
more<http://www.enterprisedb.com/resources-community>
|
|
From: Abbas B. <abb...@en...> - 2014-02-14 01:46:37
|
On Fri, Feb 14, 2014 at 6:09 AM, Koichi Suzuki <koi...@gm...>wrote: > I don't think it works because there're no guarantee each cursor > against different datanode returns rows in the same order. > Why would a query with an ORDER BY clause on all the columns not return the rows in the same order on each datanode? > > Regards; > --- > Koichi Suzuki > > > 2014-02-13 15:50 GMT+09:00 Ashutosh Bapat <ash...@en... > >: > > WCO on datanodes should be working fine, so that shouldn't be a problem. > > > > > > On Thu, Feb 13, 2014 at 12:13 PM, 鈴木 幸市 <ko...@in...> > wrote: > >> > >> Are you sure that this does not come up with any bad side effects to > >> support WCO in 1.3? > >> --- > >> Koichi Suzuki > >> > >> 2014/02/13 15:24、Ashutosh Bapat <ash...@en...> のメール： > >> > >> One more solution would be to use cursors for replicated tables. The > idea > >> is to open cursors on all the copies of the table and append the query > with > >> an ORDER BY clause on all the columns. Thus we are sure that the > current of > >> each of these cursors point to same row on all the copies. While > fetching a > >> row from a replicated table, we fetch from all the cursors and choose > only > >> one row for the data processing. While updating or deleting we send > UPDATE > >> or DELETE with WHERE CURRENT OF. The down side of this approach is > that, if > >> there are coordinator quals, we will end up locking more rows than > >> necessary, increasing the probability of the deadlock but at least there > >> won't be a necessary restriction of having primary or unique key and we > >> won't break backward compatibility. > >> > >> If there two identical rows, we might mix the update from different > nodes, > >> but then who knew which of them were corresponded across the nodes to > start > >> with. > >> > >> > >> On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki <koi...@gm...> > >> wrote: > >>> > >>> Hi, > >>> > >>> I tested the patch and found that primary key is mandatory. We need > >>> to modify regression test considerably to give each replicated table > >>> primary keys. > >>> > >>> I think this patch helps but I'm not afraid this is good, especially > >>> when we try to take XC features back to PG. > >>> > >>> Did you post another patch to use all column values if primary key is > >>> not available? > >>> > >>> I think better way is as follows: > >>> > >>> 1) If primary key is defined, use it, > >>> 2) If not, create a primary key as system column, the size should be > >>> 64bit. > >>> 3) If primary key is added to a replicated table, remove system primary > >>> key. > >>> > >>> The value of primary key can be obtained as follows: > >>> > >>> 1) add new column to pgxc_class catalog to represent maximum value of > >>> the system primary key, > >>> 2) when first "insert" is done to the primary node, system primary key > >>> value is taken from 1) and 1) is updated. The value is returned to > >>> the coordinator to be propagated to other nodes. > >>> 3) when subsequent "insert" is being done, system primary key value is > >>> added to the column value. In this case, each datanode updates 1) > >>> column value if it is larger than the current maximum value. > >>> > >>> 3) is important to change primary node to another. This is needed to > >>> carry over the primary node to another. > >>> > >>> ALTER TABLE should take care of them. > >>> > >>> Other issues are: > >>> > >>> 4) pg_dump/pg_dumpall should not include this system column value, > >>> 5) cluster may need to handle this too to repack system primary key > >>> value (not now but at least in 1.3 or later). > >>> > >>> Regards; > >>> --- > >>> Koichi Suzuki > >>> > >>> > >>> 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>: > >>> > Please see attached patch that tries to address the issue of XC using > >>> > CTID > >>> > for replicated updates and deletes when it is evaluated at a > >>> > coordinator > >>> > instead of being pushed down. > >>> > > >>> > The problem here is that CTID could be referring to a different tuple > >>> > altogether on a different data node, which is what happened for one > of > >>> > our > >>> > Postgres-XC support customers, leading to data issues. > >>> > > >>> > Instead, the patch looks for a primary key or unique index (with the > >>> > primary > >>> > key preferred) and uses those values instead of CTID. > >>> > > >>> > The patch could be improved further. Extra parameters are set even > if > >>> > not > >>> > used in the execution of the prepared statement sent down to the data > >>> > nodes. > >>> > > >>> > Regards, > >>> > > >>> > > >>> > -- > >>> > Mason Sharp > >>> > > >>> > TransLattice - http://www.translattice.com > >>> > Distributed and Clustered Database Solutions > >>> > > >>> > > >>> > > ------------------------------------------------------------------------------ > >>> > November Webinars for C, C++, Fortran Developers > >>> > Accelerate application performance with scalable programming models. > >>> > Explore > >>> > techniques for threading, error checking, porting, and tuning. Get > the > >>> > most > >>> > from the latest Intel processors and coprocessors. See abstracts and > >>> > register > >>> > > >>> > > http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk > >>> > _______________________________________________ > >>> > Postgres-xc-developers mailing list > >>> > Pos...@li... > >>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > >>> > > >>> > >>> > >>> > ------------------------------------------------------------------------------ > >>> Android apps run on BlackBerry 10 > >>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > >>> Now with support for Jelly Bean, Bluetooth, Mapview and more. > >>> Get your Android app in front of a whole new audience. Start now. > >>> > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk > >>> _______________________________________________ > >>> Postgres-xc-developers mailing list > >>> Pos...@li... > >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > >> > >> > >> > >> > >> -- > >> Best Wishes, > >> Ashutosh Bapat > >> EnterpriseDB Corporation > >> The Postgres Database Company > >> > >> > ------------------------------------------------------------------------------ > >> Android apps run on BlackBerry 10 > >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > >> Now with support for Jelly Bean, Bluetooth, Mapview and more. > >> Get your Android app in front of a whole new audience. Start now. > >> > >> > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ > >> Postgres-xc-developers mailing list > >> Pos...@li... > >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > >> > >> > > > > > > > > -- > > Best Wishes, > > Ashutosh Bapat > > EnterpriseDB Corporation > > The Postgres Database Company > > > ------------------------------------------------------------------------------ > Android apps run on BlackBerry 10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > -- -- *Abbas* Architect Ph: 92.334.5100153 Skype ID: gabbasb www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/> *Follow us on Twitter* @EnterpriseDB Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community> |
|
From: Abbas B. <abb...@en...> - 2014-02-14 01:31:02
|
Hi, Would it be possible for you to base the release on 1.2 instead? We have just released 1.2 beta and would soon be going GA. I have tested that the case works fine on 1.2. Regards On Wed, Feb 12, 2014 at 10:28 PM, David E. Wheeler <da...@ju...>wrote: > On Feb 5, 2014, at 11:17 PM, Ahsan Hadi <ahs...@en...> > wrote: > > > Abbas, > > > > Can you test this with 1.1? Perhaps we fixed this issue in the master > which means that it will be part of 1.2 beta that is planned for this month. > > Hey All, > > Any word on this? Holding up a release Sqitch till I hear back. > > Thanks, > > David -- -- *Abbas* Architect Ph: 92.334.5100153 Skype ID: gabbasb www.enterprisedb.co <http://www.enterprisedb.com/>m<http://www.enterprisedb.com/> *Follow us on Twitter* @EnterpriseDB Visit EnterpriseDB for tutorials, webinars, whitepapers<http://www.enterprisedb.com/resources-community>and more<http://www.enterprisedb.com/resources-community> |
|
From: Koichi S. <koi...@gm...> - 2014-02-14 01:09:59
|
I don't think it works because there're no guarantee each cursor against different datanode returns rows in the same order. Regards; --- Koichi Suzuki 2014-02-13 15:50 GMT+09:00 Ashutosh Bapat <ash...@en...>: > WCO on datanodes should be working fine, so that shouldn't be a problem. > > > On Thu, Feb 13, 2014 at 12:13 PM, 鈴木 幸市 <ko...@in...> wrote: >> >> Are you sure that this does not come up with any bad side effects to >> support WCO in 1.3? >> --- >> Koichi Suzuki >> >> 2014/02/13 15:24、Ashutosh Bapat <ash...@en...> のメール： >> >> One more solution would be to use cursors for replicated tables. The idea >> is to open cursors on all the copies of the table and append the query with >> an ORDER BY clause on all the columns. Thus we are sure that the current of >> each of these cursors point to same row on all the copies. While fetching a >> row from a replicated table, we fetch from all the cursors and choose only >> one row for the data processing. While updating or deleting we send UPDATE >> or DELETE with WHERE CURRENT OF. The down side of this approach is that, if >> there are coordinator quals, we will end up locking more rows than >> necessary, increasing the probability of the deadlock but at least there >> won't be a necessary restriction of having primary or unique key and we >> won't break backward compatibility. >> >> If there two identical rows, we might mix the update from different nodes, >> but then who knew which of them were corresponded across the nodes to start >> with. >> >> >> On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki <koi...@gm...> >> wrote: >>> >>> Hi, >>> >>> I tested the patch and found that primary key is mandatory. We need >>> to modify regression test considerably to give each replicated table >>> primary keys. >>> >>> I think this patch helps but I'm not afraid this is good, especially >>> when we try to take XC features back to PG. >>> >>> Did you post another patch to use all column values if primary key is >>> not available? >>> >>> I think better way is as follows: >>> >>> 1) If primary key is defined, use it, >>> 2) If not, create a primary key as system column, the size should be >>> 64bit. >>> 3) If primary key is added to a replicated table, remove system primary >>> key. >>> >>> The value of primary key can be obtained as follows: >>> >>> 1) add new column to pgxc_class catalog to represent maximum value of >>> the system primary key, >>> 2) when first "insert" is done to the primary node, system primary key >>> value is taken from 1) and 1) is updated. The value is returned to >>> the coordinator to be propagated to other nodes. >>> 3) when subsequent "insert" is being done, system primary key value is >>> added to the column value. In this case, each datanode updates 1) >>> column value if it is larger than the current maximum value. >>> >>> 3) is important to change primary node to another. This is needed to >>> carry over the primary node to another. >>> >>> ALTER TABLE should take care of them. >>> >>> Other issues are: >>> >>> 4) pg_dump/pg_dumpall should not include this system column value, >>> 5) cluster may need to handle this too to repack system primary key >>> value (not now but at least in 1.3 or later). >>> >>> Regards; >>> --- >>> Koichi Suzuki >>> >>> >>> 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>: >>> > Please see attached patch that tries to address the issue of XC using >>> > CTID >>> > for replicated updates and deletes when it is evaluated at a >>> > coordinator >>> > instead of being pushed down. >>> > >>> > The problem here is that CTID could be referring to a different tuple >>> > altogether on a different data node, which is what happened for one of >>> > our >>> > Postgres-XC support customers, leading to data issues. >>> > >>> > Instead, the patch looks for a primary key or unique index (with the >>> > primary >>> > key preferred) and uses those values instead of CTID. >>> > >>> > The patch could be improved further. Extra parameters are set even if >>> > not >>> > used in the execution of the prepared statement sent down to the data >>> > nodes. >>> > >>> > Regards, >>> > >>> > >>> > -- >>> > Mason Sharp >>> > >>> > TransLattice - http://www.translattice.com >>> > Distributed and Clustered Database Solutions >>> > >>> > >>> > ------------------------------------------------------------------------------ >>> > November Webinars for C, C++, Fortran Developers >>> > Accelerate application performance with scalable programming models. >>> > Explore >>> > techniques for threading, error checking, porting, and tuning. Get the >>> > most >>> > from the latest Intel processors and coprocessors. See abstracts and >>> > register >>> > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk >>> > _______________________________________________ >>> > Postgres-xc-developers mailing list >>> > Pos...@li... >>> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> > >>> >>> >>> ------------------------------------------------------------------------------ >>> Android apps run on BlackBerry 10 >>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >>> Now with support for Jelly Bean, Bluetooth, Mapview and more. >>> Get your Android app in front of a whole new audience. Start now. >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Postgres-xc-developers mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> >> >> >> -- >> Best Wishes, >> Ashutosh Bapat >> EnterpriseDB Corporation >> The Postgres Database Company >> >> ------------------------------------------------------------------------------ >> Android apps run on BlackBerry 10 >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> Get your Android app in front of a whole new audience. Start now. >> >> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> > > > > -- > Best Wishes, > Ashutosh Bapat > EnterpriseDB Corporation > The Postgres Database Company |
|
From: 鈴木 幸市 <ko...@in...> - 2014-02-14 00:48:56
|
This will be good to have. The issue looks something different. Sometimes canceling the statement with separate connection becomes effective after the call returns and subsequent statements are issued. This is why Masataka tried to add some (small) delay between the two. This delay is needed only at statement canceling. Any more comments on this? --- Koichi Suzuki 2014/02/14 2:30、Andrei Martsinchyk <and...@gm...<mailto:and...@gm...>> のメール： Hello, Postgres establishes separate connection to deliver Cancel command to the target session. On a heavily loaded node it may take fairly long. Longer sleep would help out, but it means longer recovery after an error. Better solution is to remove canceled connection from the pool and therefore do not use it to handle subsequent queries. 2014-02-13 11:10 GMT+02:00 Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>>: I think it hits the point. I tested this patch several times and it seems to work fine. The delay time (at present 10ms) is short enough and it is applied only when we need to cancel a statement. We should check this into all the master and STABLE branches improving magic number with some meaningful name. Any thoughts? --- Koichi Suzuki 2014-01-24 18:25 GMT+09:00 Masataka Saito <pg...@gm...<mailto:pg...@gm...>>: > Hello, > > As I've been exasperated by random failures, I'm willing to whip the cause > of the issue. > > This issue is related to cancel of the failed query. > When a datanode reports an error of a query, a coordinator sends a cancel > request to non-idle nodes, waits the node to get ready and requests nodes to > rollback the transaction. > > Where's the problem? Consider the next case. > 1. Datanode A (PID 1) reports an error to coordinator A. ([1] 'E' message) > 2. Coordinator A receives [1] and reports an error to a frontend. ([2] 'E' > message) > 3. Coordinator A starts aborting process and it thinks datanode A (PID 1) is > not idle. > 4. Coordinator A sends a cancel request about PID 1 to datanode A (PID 2). > ([3] cancel message) > 5. Datanode A (PID 1) reports ready to coordinator A. ([4] 'Z' message) > 6. Coordinator A receives [4] and sends "ROLLBACK TRANSACTION" immediately. > ([5] 'Q' message) > 7. Datanode A (PID 1) receives [5] and starts processing the query. > 8. Datanode A (PID 2) receives [3]. > 9. Datanode A (PID 2) notify PID 1 of [3]. > 10. Datanode A (PID 1) cancel processing [5] and reports an error to > Coordinator A. ([6] 'E' message) > 11. Coordinator A receives [6] and reports an error to a frontend. ([7] 'E' > message) > > [7] makes unexpected output and a test fails. > > Saying an extreme thing, it could occur that the next query of [5] is > cancelled by [3]. > > As far as I know, there's no way to know when to the cancel request get to > be processed, I think we can't not wait an experimental duration after > cancelling like the attached patch. > > Does anyone have another cool idea to solve this issue? > > Regards. > > ------------------------------------------------------------------------------ > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li...<mailto:Pos...@li...> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > ------------------------------------------------------------------------------ Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk _______________________________________________ Postgres-xc-developers mailing list Pos...@li...<mailto:Pos...@li...> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers -- Andrei Martsinchyk StormDB - http://www.stormdb.com<http://www.stormdb.com/> The Database Cloud ------------------------------------------------------------------------------ Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ Postgres-xc-developers mailing list Pos...@li... https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers |
|
From: Andrei M. <and...@gm...> - 2014-02-13 17:30:23
|
Hello, Postgres establishes separate connection to deliver Cancel command to the target session. On a heavily loaded node it may take fairly long. Longer sleep would help out, but it means longer recovery after an error. Better solution is to remove canceled connection from the pool and therefore do not use it to handle subsequent queries. 2014-02-13 11:10 GMT+02:00 Koichi Suzuki <koi...@gm...>: > I think it hits the point. I tested this patch several times and it > seems to work fine. The delay time (at present 10ms) is short enough > and it is applied only when we need to cancel a statement. > > We should check this into all the master and STABLE branches improving > magic number with some meaningful name. > > Any thoughts? > --- > Koichi Suzuki > > > 2014-01-24 18:25 GMT+09:00 Masataka Saito <pg...@gm...>: > > Hello, > > > > As I've been exasperated by random failures, I'm willing to whip the > cause > > of the issue. > > > > This issue is related to cancel of the failed query. > > When a datanode reports an error of a query, a coordinator sends a cancel > > request to non-idle nodes, waits the node to get ready and requests > nodes to > > rollback the transaction. > > > > Where's the problem? Consider the next case. > > 1. Datanode A (PID 1) reports an error to coordinator A. ([1] 'E' > message) > > 2. Coordinator A receives [1] and reports an error to a frontend. ([2] > 'E' > > message) > > 3. Coordinator A starts aborting process and it thinks datanode A (PID > 1) is > > not idle. > > 4. Coordinator A sends a cancel request about PID 1 to datanode A (PID > 2). > > ([3] cancel message) > > 5. Datanode A (PID 1) reports ready to coordinator A. ([4] 'Z' message) > > 6. Coordinator A receives [4] and sends "ROLLBACK TRANSACTION" > immediately. > > ([5] 'Q' message) > > 7. Datanode A (PID 1) receives [5] and starts processing the query. > > 8. Datanode A (PID 2) receives [3]. > > 9. Datanode A (PID 2) notify PID 1 of [3]. > > 10. Datanode A (PID 1) cancel processing [5] and reports an error to > > Coordinator A. ([6] 'E' message) > > 11. Coordinator A receives [6] and reports an error to a frontend. ([7] > 'E' > > message) > > > > [7] makes unexpected output and a test fails. > > > > Saying an extreme thing, it could occur that the next query of [5] is > > cancelled by [3]. > > > > As far as I know, there's no way to know when to the cancel request get > to > > be processed, I think we can't not wait an experimental duration after > > cancelling like the attached patch. > > > > Does anyone have another cool idea to solve this issue? > > > > Regards. > > > > > ------------------------------------------------------------------------------ > > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > > Learn Why More Businesses Are Choosing CenturyLink Cloud For > > Critical Workloads, Development Environments & Everything In Between. > > Get a Quote or Start a Free Trial Today. > > > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > > _______________________________________________ > > Postgres-xc-developers mailing list > > Pos...@li... > > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > ------------------------------------------------------------------------------ > Android apps run on BlackBerry 10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > -- Andrei Martsinchyk StormDB - http://www.stormdb.com The Database Cloud |
|
From: Koichi S. <koi...@gm...> - 2014-02-13 09:54:36
|
Actually, revised patch worked and I've already committed it. Thank you. --- Koichi Suzuki 2014-02-13 18:42 GMT+09:00 Tomonari Katsumata <kat...@po...>: > Hi, > > I'm sorry. > master has the same trouble too. > Here is the patch for the master(*). > (*)0b4543953f32ac3d8d3b7ba1983a19f14c8c091e > > regards, > ----------------- > Tomonari Katsumata > > > (2014/02/13 17:59), 鈴木 幸市 wrote: >> I’m afraid the patch is not committed to the master either. Could you >> provide the patch for the master if needed? >> >> Regards; >> --- >> Koichi Suzuki >> >> 2014/02/13 15:25、Tomonari Katsumata <kat...@po...> >> のメール： >> >>> Hi, >>> >>> Congratulation on releasing Postgres-XC 1.2 beta! >>> >>> The problem in this thread has already been fixed at >>> Postgres-XC 1.1 stable, but unfortunately not been fixed >>> at Postgres-XC 1.2 beta. >>> >>> I confirmed the problem was resolved with same fix >>> to Postgres-XC 1.1 stable. >>> see attached patch. This patch is for Postgres-XC 1.2 beta(*). >>> (*)9b6be5b8ba66eb9e083a4a80669e7d41a2cea3dc >>> >>> I hope you'll fix in formal release of Postgres-XC 1.2. >>> >>> regards, >>> ------------------- >>> Tomonari Katsumata >>> >>> (2013/06/04 10:31), Tomonari Katsumata wrote: >>>> Hi, I have a problem with query executing. >>>> >>>> I cant't have any response when I execute a query. >>>> This problem occurs when some conditions are met. >>>> >>>> The conditions are below. >>>> --------------------------------------------------------------------- >>>> 1. Issuing "EXECUTE DIRECT" to Datanode(ofcourse, via Coordinator). >>>> >>>> 2. The Query Executing on Datanode has subquery on its FROM-clause. >>>> >>>> 3. In the subquery, it has a JOIN clause. >>>> >>>> 4. The Join clause is consisted with another subquery. >>>> --------------------------------------------------------------------- >>>> >>>> >>>> Simple example query is below. >>>> --------------------------------------------------------------- >>>> EXECUTE DIRECT ON (data1) $$ >>>> SELECT >>>> count(*) >>>> FROM >>>> (SELECT * FROM pg_locks l LEFT JOIN >>>> (SELECT * FROM pg_stat_activity) s ON l.database = s.datid) a >>>> $$ >>>> --------------------------------------------------------------- >>>> >>>> FYI: >>>> This query works fine with Postgres-XC 1.0.3. >>>> Is this already known bug ? >>>> >>>> >>>> How can I avoid this problem ? >>>> And what kind of info do you need to investigate it ? >>>> >>>> ---------- >>>> NTT Software Corporation >>>> Tomonari Katsumata >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> How ServiceNow helps IT people transform IT departments: >>>> 1. A cloud service to automate IT design, transition and operations >>>> 2. Dashboards that offer high-level views of enterprise services >>>> 3. A single system of record for all IT processes >>>> http://p.sf.net/sfu/servicenow-d2d-j >>>> _______________________________________________ >>>> Postgres-xc-developers mailing list >>>> Pos...@li... >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>>> >>> >>> >>> >>> <revised_fqs_for12beta.patch>------------------------------------------------------------------------------ >>> Android apps run on BlackBerry 10 >>> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >>> Now with support for Jelly Bean, Bluetooth, Mapview and more. >>> Get your Android app in front of a whole new audience. Start now. >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ >>> Postgres-xc-developers mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> >> > > > ------------------------------------------------------------------------------ > Android apps run on BlackBerry 10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > |
|
From: Tomonari K. <kat...@po...> - 2014-02-13 09:43:30
|
Hi, I'm sorry. master has the same trouble too. Here is the patch for the master(*). (*)0b4543953f32ac3d8d3b7ba1983a19f14c8c091e regards, ----------------- Tomonari Katsumata (2014/02/13 17:59), 鈴木 幸市 wrote: > I’m afraid the patch is not committed to the master either. Could you provide the patch for the master if needed? > > Regards; > --- > Koichi Suzuki > > 2014/02/13 15:25、Tomonari Katsumata <kat...@po...> のメール： > >> Hi, >> >> Congratulation on releasing Postgres-XC 1.2 beta! >> >> The problem in this thread has already been fixed at >> Postgres-XC 1.1 stable, but unfortunately not been fixed >> at Postgres-XC 1.2 beta. >> >> I confirmed the problem was resolved with same fix >> to Postgres-XC 1.1 stable. >> see attached patch. This patch is for Postgres-XC 1.2 beta(*). >> (*)9b6be5b8ba66eb9e083a4a80669e7d41a2cea3dc >> >> I hope you'll fix in formal release of Postgres-XC 1.2. >> >> regards, >> ------------------- >> Tomonari Katsumata >> >> (2013/06/04 10:31), Tomonari Katsumata wrote: >>> Hi, I have a problem with query executing. >>> >>> I cant't have any response when I execute a query. >>> This problem occurs when some conditions are met. >>> >>> The conditions are below. >>> --------------------------------------------------------------------- >>> 1. Issuing "EXECUTE DIRECT" to Datanode(ofcourse, via Coordinator). >>> >>> 2. The Query Executing on Datanode has subquery on its FROM-clause. >>> >>> 3. In the subquery, it has a JOIN clause. >>> >>> 4. The Join clause is consisted with another subquery. >>> --------------------------------------------------------------------- >>> >>> >>> Simple example query is below. >>> --------------------------------------------------------------- >>> EXECUTE DIRECT ON (data1) $$ >>> SELECT >>> count(*) >>> FROM >>> (SELECT * FROM pg_locks l LEFT JOIN >>> (SELECT * FROM pg_stat_activity) s ON l.database = s.datid) a >>> $$ >>> --------------------------------------------------------------- >>> >>> FYI: >>> This query works fine with Postgres-XC 1.0.3. >>> Is this already known bug ? >>> >>> >>> How can I avoid this problem ? >>> And what kind of info do you need to investigate it ? >>> >>> ---------- >>> NTT Software Corporation >>> Tomonari Katsumata >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> How ServiceNow helps IT people transform IT departments: >>> 1. A cloud service to automate IT design, transition and operations >>> 2. Dashboards that offer high-level views of enterprise services >>> 3. A single system of record for all IT processes >>> http://p.sf.net/sfu/servicenow-d2d-j >>> _______________________________________________ >>> Postgres-xc-developers mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> >> >> >> <revised_fqs_for12beta.patch>------------------------------------------------------------------------------ >> Android apps run on BlackBerry 10 >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> Get your Android app in front of a whole new audience. Start now. >> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > |
|
From: Koichi S. <koi...@gm...> - 2014-02-13 09:10:28
|
I think it hits the point. I tested this patch several times and it seems to work fine. The delay time (at present 10ms) is short enough and it is applied only when we need to cancel a statement. We should check this into all the master and STABLE branches improving magic number with some meaningful name. Any thoughts? --- Koichi Suzuki 2014-01-24 18:25 GMT+09:00 Masataka Saito <pg...@gm...>: > Hello, > > As I've been exasperated by random failures, I'm willing to whip the cause > of the issue. > > This issue is related to cancel of the failed query. > When a datanode reports an error of a query, a coordinator sends a cancel > request to non-idle nodes, waits the node to get ready and requests nodes to > rollback the transaction. > > Where's the problem? Consider the next case. > 1. Datanode A (PID 1) reports an error to coordinator A. ([1] 'E' message) > 2. Coordinator A receives [1] and reports an error to a frontend. ([2] 'E' > message) > 3. Coordinator A starts aborting process and it thinks datanode A (PID 1) is > not idle. > 4. Coordinator A sends a cancel request about PID 1 to datanode A (PID 2). > ([3] cancel message) > 5. Datanode A (PID 1) reports ready to coordinator A. ([4] 'Z' message) > 6. Coordinator A receives [4] and sends "ROLLBACK TRANSACTION" immediately. > ([5] 'Q' message) > 7. Datanode A (PID 1) receives [5] and starts processing the query. > 8. Datanode A (PID 2) receives [3]. > 9. Datanode A (PID 2) notify PID 1 of [3]. > 10. Datanode A (PID 1) cancel processing [5] and reports an error to > Coordinator A. ([6] 'E' message) > 11. Coordinator A receives [6] and reports an error to a frontend. ([7] 'E' > message) > > [7] makes unexpected output and a test fails. > > Saying an extreme thing, it could occur that the next query of [5] is > cancelled by [3]. > > As far as I know, there's no way to know when to the cancel request get to > be processed, I think we can't not wait an experimental duration after > cancelling like the attached patch. > > Does anyone have another cool idea to solve this issue? > > Regards. > > ------------------------------------------------------------------------------ > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > |
|
From: 鈴木 幸市 <ko...@in...> - 2014-02-13 08:59:13
|
I’m afraid the patch is not committed to the master either. Could you provide the patch for the master if needed? Regards; --- Koichi Suzuki 2014/02/13 15:25、Tomonari Katsumata <kat...@po...> のメール： > Hi, > > Congratulation on releasing Postgres-XC 1.2 beta! > > The problem in this thread has already been fixed at > Postgres-XC 1.1 stable, but unfortunately not been fixed > at Postgres-XC 1.2 beta. > > I confirmed the problem was resolved with same fix > to Postgres-XC 1.1 stable. > see attached patch. This patch is for Postgres-XC 1.2 beta(*). > (*)9b6be5b8ba66eb9e083a4a80669e7d41a2cea3dc > > I hope you'll fix in formal release of Postgres-XC 1.2. > > regards, > ------------------- > Tomonari Katsumata > > (2013/06/04 10:31), Tomonari Katsumata wrote: > > Hi, I have a problem with query executing. > > > > I cant't have any response when I execute a query. > > This problem occurs when some conditions are met. > > > > The conditions are below. > > --------------------------------------------------------------------- > > 1. Issuing "EXECUTE DIRECT" to Datanode(ofcourse, via Coordinator). > > > > 2. The Query Executing on Datanode has subquery on its FROM-clause. > > > > 3. In the subquery, it has a JOIN clause. > > > > 4. The Join clause is consisted with another subquery. > > --------------------------------------------------------------------- > > > > > > Simple example query is below. > > --------------------------------------------------------------- > > EXECUTE DIRECT ON (data1) $$ > > SELECT > > count(*) > > FROM > > (SELECT * FROM pg_locks l LEFT JOIN > > (SELECT * FROM pg_stat_activity) s ON l.database = s.datid) a > > $$ > > --------------------------------------------------------------- > > > > FYI: > > This query works fine with Postgres-XC 1.0.3. > > Is this already known bug ? > > > > > > How can I avoid this problem ? > > And what kind of info do you need to investigate it ? > > > > ---------- > > NTT Software Corporation > > Tomonari Katsumata > > > > > > > > ------------------------------------------------------------------------------ > > How ServiceNow helps IT people transform IT departments: > > 1. A cloud service to automate IT design, transition and operations > > 2. Dashboards that offer high-level views of enterprise services > > 3. A single system of record for all IT processes > > http://p.sf.net/sfu/servicenow-d2d-j > > _______________________________________________ > > Postgres-xc-developers mailing list > > Pos...@li... > > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > <revised_fqs_for12beta.patch>------------------------------------------------------------------------------ > Android apps run on BlackBerry 10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers |
|
From: Tomonari K. <kat...@po...> - 2014-02-13 06:58:12
|
Hi Suzuki-san, I'm looking forward to your commit. Thanks a lot! regards, --------- Tomonari Katsumata (2014/02/13 15:44), 鈴木 幸市 wrote: > Thanks. I was worrying about it. Will be committed soon. > --- > Koichi Suzuki > > 2014/02/13 15:25、Tomonari Katsumata <kat...@po...> のメール： > >> Hi, >> >> Congratulation on releasing Postgres-XC 1.2 beta! >> >> The problem in this thread has already been fixed at >> Postgres-XC 1.1 stable, but unfortunately not been fixed >> at Postgres-XC 1.2 beta. >> >> I confirmed the problem was resolved with same fix >> to Postgres-XC 1.1 stable. >> see attached patch. This patch is for Postgres-XC 1.2 beta(*). >> (*)9b6be5b8ba66eb9e083a4a80669e7d41a2cea3dc >> >> I hope you'll fix in formal release of Postgres-XC 1.2. >> >> regards, >> ------------------- >> Tomonari Katsumata >> >> (2013/06/04 10:31), Tomonari Katsumata wrote: >>> Hi, I have a problem with query executing. >>> >>> I cant't have any response when I execute a query. >>> This problem occurs when some conditions are met. >>> >>> The conditions are below. >>> --------------------------------------------------------------------- >>> 1. Issuing "EXECUTE DIRECT" to Datanode(ofcourse, via Coordinator). >>> >>> 2. The Query Executing on Datanode has subquery on its FROM-clause. >>> >>> 3. In the subquery, it has a JOIN clause. >>> >>> 4. The Join clause is consisted with another subquery. >>> --------------------------------------------------------------------- >>> >>> >>> Simple example query is below. >>> --------------------------------------------------------------- >>> EXECUTE DIRECT ON (data1) $$ >>> SELECT >>> count(*) >>> FROM >>> (SELECT * FROM pg_locks l LEFT JOIN >>> (SELECT * FROM pg_stat_activity) s ON l.database = s.datid) a >>> $$ >>> --------------------------------------------------------------- >>> >>> FYI: >>> This query works fine with Postgres-XC 1.0.3. >>> Is this already known bug ? >>> >>> >>> How can I avoid this problem ? >>> And what kind of info do you need to investigate it ? >>> >>> ---------- >>> NTT Software Corporation >>> Tomonari Katsumata >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> How ServiceNow helps IT people transform IT departments: >>> 1. A cloud service to automate IT design, transition and operations >>> 2. Dashboards that offer high-level views of enterprise services >>> 3. A single system of record for all IT processes >>> http://p.sf.net/sfu/servicenow-d2d-j >>> _______________________________________________ >>> Postgres-xc-developers mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> >> >> >> <revised_fqs_for12beta.patch>------------------------------------------------------------------------------ >> Android apps run on BlackBerry 10 >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> Get your Android app in front of a whole new audience. Start now. >> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > |
|
From: Ashutosh B. <ash...@en...> - 2014-02-13 06:50:28
|
WCO on datanodes should be working fine, so that shouldn't be a problem. On Thu, Feb 13, 2014 at 12:13 PM, 鈴木 幸市 <ko...@in...> wrote: > Are you sure that this does not come up with any bad side effects to > support WCO in 1.3? > --- > Koichi Suzuki > > 2014/02/13 15:24、Ashutosh Bapat <ash...@en...> のメール： > > One more solution would be to use cursors for replicated tables. The > idea is to open cursors on all the copies of the table and append the query > with an ORDER BY clause on all the columns. Thus we are sure that the > current of each of these cursors point to same row on all the copies. While > fetching a row from a replicated table, we fetch from all the cursors and > choose only one row for the data processing. While updating or deleting we > send UPDATE or DELETE with WHERE CURRENT OF. The down side of this approach > is that, if there are coordinator quals, we will end up locking more rows > than necessary, increasing the probability of the deadlock but at least > there won't be a necessary restriction of having primary or unique key and > we won't break backward compatibility. > > If there two identical rows, we might mix the update from different > nodes, but then who knew which of them were corresponded across the nodes > to start with. > > > On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki <koi...@gm...>wrote: > >> Hi, >> >> I tested the patch and found that primary key is mandatory. We need >> to modify regression test considerably to give each replicated table >> primary keys. >> >> I think this patch helps but I'm not afraid this is good, especially >> when we try to take XC features back to PG. >> >> Did you post another patch to use all column values if primary key is >> not available? >> >> I think better way is as follows: >> >> 1) If primary key is defined, use it, >> 2) If not, create a primary key as system column, the size should be >> 64bit. >> 3) If primary key is added to a replicated table, remove system primary >> key. >> >> The value of primary key can be obtained as follows: >> >> 1) add new column to pgxc_class catalog to represent maximum value of >> the system primary key, >> 2) when first "insert" is done to the primary node, system primary key >> value is taken from 1) and 1) is updated. The value is returned to >> the coordinator to be propagated to other nodes. >> 3) when subsequent "insert" is being done, system primary key value is >> added to the column value. In this case, each datanode updates 1) >> column value if it is larger than the current maximum value. >> >> 3) is important to change primary node to another. This is needed to >> carry over the primary node to another. >> >> ALTER TABLE should take care of them. >> >> Other issues are: >> >> 4) pg_dump/pg_dumpall should not include this system column value, >> 5) cluster may need to handle this too to repack system primary key >> value (not now but at least in 1.3 or later). >> >> Regards; >> --- >> Koichi Suzuki >> >> >> 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>: >> > Please see attached patch that tries to address the issue of XC using >> CTID >> > for replicated updates and deletes when it is evaluated at a coordinator >> > instead of being pushed down. >> > >> > The problem here is that CTID could be referring to a different tuple >> > altogether on a different data node, which is what happened for one of >> our >> > Postgres-XC support customers, leading to data issues. >> > >> > Instead, the patch looks for a primary key or unique index (with the >> primary >> > key preferred) and uses those values instead of CTID. >> > >> > The patch could be improved further. Extra parameters are set even if >> not >> > used in the execution of the prepared statement sent down to the data >> nodes. >> > >> > Regards, >> > >> > >> > -- >> > Mason Sharp >> > >> > TransLattice - http://www.translattice.com >> > Distributed and Clustered Database Solutions >> > >> > >> ------------------------------------------------------------------------------ >> > November Webinars for C, C++, Fortran Developers >> > Accelerate application performance with scalable programming models. >> Explore >> > techniques for threading, error checking, porting, and tuning. Get the >> most >> > from the latest Intel processors and coprocessors. See abstracts and >> > register >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk >> > _______________________________________________ >> > Postgres-xc-developers mailing list >> > Pos...@li... >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> > >> >> >> ------------------------------------------------------------------------------ >> Android apps run on BlackBerry 10 >> Introducing the new BlackBerry 10.2.1 Runtime for Android apps. >> Now with support for Jelly Bean, Bluetooth, Mapview and more. >> Get your Android app in front of a whole new audience. Start now. >> >> http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> > > > > -- > Best Wishes, > Ashutosh Bapat > EnterpriseDB Corporation > The Postgres Database Company > > ------------------------------------------------------------------------------ > Android apps run on BlackBerry 10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company |
|
From: 鈴木 幸市 <ko...@in...> - 2014-02-13 06:45:07
|
Thanks. I was worrying about it. Will be committed soon. --- Koichi Suzuki 2014/02/13 15:25、Tomonari Katsumata <kat...@po...> のメール： > Hi, > > Congratulation on releasing Postgres-XC 1.2 beta! > > The problem in this thread has already been fixed at > Postgres-XC 1.1 stable, but unfortunately not been fixed > at Postgres-XC 1.2 beta. > > I confirmed the problem was resolved with same fix > to Postgres-XC 1.1 stable. > see attached patch. This patch is for Postgres-XC 1.2 beta(*). > (*)9b6be5b8ba66eb9e083a4a80669e7d41a2cea3dc > > I hope you'll fix in formal release of Postgres-XC 1.2. > > regards, > ------------------- > Tomonari Katsumata > > (2013/06/04 10:31), Tomonari Katsumata wrote: > > Hi, I have a problem with query executing. > > > > I cant't have any response when I execute a query. > > This problem occurs when some conditions are met. > > > > The conditions are below. > > --------------------------------------------------------------------- > > 1. Issuing "EXECUTE DIRECT" to Datanode(ofcourse, via Coordinator). > > > > 2. The Query Executing on Datanode has subquery on its FROM-clause. > > > > 3. In the subquery, it has a JOIN clause. > > > > 4. The Join clause is consisted with another subquery. > > --------------------------------------------------------------------- > > > > > > Simple example query is below. > > --------------------------------------------------------------- > > EXECUTE DIRECT ON (data1) $$ > > SELECT > > count(*) > > FROM > > (SELECT * FROM pg_locks l LEFT JOIN > > (SELECT * FROM pg_stat_activity) s ON l.database = s.datid) a > > $$ > > --------------------------------------------------------------- > > > > FYI: > > This query works fine with Postgres-XC 1.0.3. > > Is this already known bug ? > > > > > > How can I avoid this problem ? > > And what kind of info do you need to investigate it ? > > > > ---------- > > NTT Software Corporation > > Tomonari Katsumata > > > > > > > > ------------------------------------------------------------------------------ > > How ServiceNow helps IT people transform IT departments: > > 1. A cloud service to automate IT design, transition and operations > > 2. Dashboards that offer high-level views of enterprise services > > 3. A single system of record for all IT processes > > http://p.sf.net/sfu/servicenow-d2d-j > > _______________________________________________ > > Postgres-xc-developers mailing list > > Pos...@li... > > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > <revised_fqs_for12beta.patch>------------------------------------------------------------------------------ > Android apps run on BlackBerry 10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers |
|
From: 鈴木 幸市 <ko...@in...> - 2014-02-13 06:44:08
|
Are you sure that this does not come up with any bad side effects to support WCO in 1.3? --- Koichi Suzuki 2014/02/13 15:24、Ashutosh Bapat <ash...@en...<mailto:ash...@en...>> のメール： One more solution would be to use cursors for replicated tables. The idea is to open cursors on all the copies of the table and append the query with an ORDER BY clause on all the columns. Thus we are sure that the current of each of these cursors point to same row on all the copies. While fetching a row from a replicated table, we fetch from all the cursors and choose only one row for the data processing. While updating or deleting we send UPDATE or DELETE with WHERE CURRENT OF. The down side of this approach is that, if there are coordinator quals, we will end up locking more rows than necessary, increasing the probability of the deadlock but at least there won't be a necessary restriction of having primary or unique key and we won't break backward compatibility. If there two identical rows, we might mix the update from different nodes, but then who knew which of them were corresponded across the nodes to start with. On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki <koi...@gm...<mailto:koi...@gm...>> wrote: Hi, I tested the patch and found that primary key is mandatory. We need to modify regression test considerably to give each replicated table primary keys. I think this patch helps but I'm not afraid this is good, especially when we try to take XC features back to PG. Did you post another patch to use all column values if primary key is not available? I think better way is as follows: 1) If primary key is defined, use it, 2) If not, create a primary key as system column, the size should be 64bit. 3) If primary key is added to a replicated table, remove system primary key. The value of primary key can be obtained as follows: 1) add new column to pgxc_class catalog to represent maximum value of the system primary key, 2) when first "insert" is done to the primary node, system primary key value is taken from 1) and 1) is updated. The value is returned to the coordinator to be propagated to other nodes. 3) when subsequent "insert" is being done, system primary key value is added to the column value. In this case, each datanode updates 1) column value if it is larger than the current maximum value. 3) is important to change primary node to another. This is needed to carry over the primary node to another. ALTER TABLE should take care of them. Other issues are: 4) pg_dump/pg_dumpall should not include this system column value, 5) cluster may need to handle this too to repack system primary key value (not now but at least in 1.3 or later). Regards; --- Koichi Suzuki 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...<mailto:ms...@tr...>>: > Please see attached patch that tries to address the issue of XC using CTID > for replicated updates and deletes when it is evaluated at a coordinator > instead of being pushed down. > > The problem here is that CTID could be referring to a different tuple > altogether on a different data node, which is what happened for one of our > Postgres-XC support customers, leading to data issues. > > Instead, the patch looks for a primary key or unique index (with the primary > key preferred) and uses those values instead of CTID. > > The patch could be improved further. Extra parameters are set even if not > used in the execution of the prepared statement sent down to the data nodes. > > Regards, > > > -- > Mason Sharp > > TransLattice - http://www.translattice.com<http://www.translattice.com/> > Distributed and Clustered Database Solutions > > ------------------------------------------------------------------------------ > November Webinars for C, C++, Fortran Developers > Accelerate application performance with scalable programming models. Explore > techniques for threading, error checking, porting, and tuning. Get the most > from the latest Intel processors and coprocessors. See abstracts and > register > http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li...<mailto:Pos...@li...> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > ------------------------------------------------------------------------------ Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk _______________________________________________ Postgres-xc-developers mailing list Pos...@li...<mailto:Pos...@li...> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company ------------------------------------------------------------------------------ Android apps run on BlackBerry 10 Introducing the new BlackBerry 10.2.1 Runtime for Android apps. Now with support for Jelly Bean, Bluetooth, Mapview and more. Get your Android app in front of a whole new audience. Start now. http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk_______________________________________________ Postgres-xc-developers mailing list Pos...@li... https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers |
|
From: Tomonari K. <kat...@po...> - 2014-02-13 06:40:57
|
Hi, Congratulation on releasing Postgres-XC 1.2 beta! The problem in this thread has already been fixed at Postgres-XC 1.1 stable, but unfortunately not been fixed at Postgres-XC 1.2 beta. I confirmed the problem was resolved with same fix to Postgres-XC 1.1 stable. see attached patch. This patch is for Postgres-XC 1.2 beta(*). (*)9b6be5b8ba66eb9e083a4a80669e7d41a2cea3dc I hope you'll fix in formal release of Postgres-XC 1.2. regards, ------------------- Tomonari Katsumata (2013/06/04 10:31), Tomonari Katsumata wrote: > Hi, I have a problem with query executing. > > I cant't have any response when I execute a query. > This problem occurs when some conditions are met. > > The conditions are below. > --------------------------------------------------------------------- > 1. Issuing "EXECUTE DIRECT" to Datanode(ofcourse, via Coordinator). > > 2. The Query Executing on Datanode has subquery on its FROM-clause. > > 3. In the subquery, it has a JOIN clause. > > 4. The Join clause is consisted with another subquery. > --------------------------------------------------------------------- > > > Simple example query is below. > --------------------------------------------------------------- > EXECUTE DIRECT ON (data1) $$ > SELECT > count(*) > FROM > (SELECT * FROM pg_locks l LEFT JOIN > (SELECT * FROM pg_stat_activity) s ON l.database = s.datid) a > $$ > --------------------------------------------------------------- > > FYI: > This query works fine with Postgres-XC 1.0.3. > Is this already known bug ? > > > How can I avoid this problem ? > And what kind of info do you need to investigate it ? > > ---------- > NTT Software Corporation > Tomonari Katsumata > > > > ------------------------------------------------------------------------------ > How ServiceNow helps IT people transform IT departments: > 1. A cloud service to automate IT design, transition and operations > 2. Dashboards that offer high-level views of enterprise services > 3. A single system of record for all IT processes > http://p.sf.net/sfu/servicenow-d2d-j > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > |
|
From: Ashutosh B. <ash...@en...> - 2014-02-13 06:24:26
|
One more solution would be to use cursors for replicated tables. The idea is to open cursors on all the copies of the table and append the query with an ORDER BY clause on all the columns. Thus we are sure that the current of each of these cursors point to same row on all the copies. While fetching a row from a replicated table, we fetch from all the cursors and choose only one row for the data processing. While updating or deleting we send UPDATE or DELETE with WHERE CURRENT OF. The down side of this approach is that, if there are coordinator quals, we will end up locking more rows than necessary, increasing the probability of the deadlock but at least there won't be a necessary restriction of having primary or unique key and we won't break backward compatibility. If there two identical rows, we might mix the update from different nodes, but then who knew which of them were corresponded across the nodes to start with. On Thu, Feb 13, 2014 at 9:45 AM, Koichi Suzuki <koi...@gm...>wrote: > Hi, > > I tested the patch and found that primary key is mandatory. We need > to modify regression test considerably to give each replicated table > primary keys. > > I think this patch helps but I'm not afraid this is good, especially > when we try to take XC features back to PG. > > Did you post another patch to use all column values if primary key is > not available? > > I think better way is as follows: > > 1) If primary key is defined, use it, > 2) If not, create a primary key as system column, the size should be 64bit. > 3) If primary key is added to a replicated table, remove system primary > key. > > The value of primary key can be obtained as follows: > > 1) add new column to pgxc_class catalog to represent maximum value of > the system primary key, > 2) when first "insert" is done to the primary node, system primary key > value is taken from 1) and 1) is updated. The value is returned to > the coordinator to be propagated to other nodes. > 3) when subsequent "insert" is being done, system primary key value is > added to the column value. In this case, each datanode updates 1) > column value if it is larger than the current maximum value. > > 3) is important to change primary node to another. This is needed to > carry over the primary node to another. > > ALTER TABLE should take care of them. > > Other issues are: > > 4) pg_dump/pg_dumpall should not include this system column value, > 5) cluster may need to handle this too to repack system primary key > value (not now but at least in 1.3 or later). > > Regards; > --- > Koichi Suzuki > > > 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>: > > Please see attached patch that tries to address the issue of XC using > CTID > > for replicated updates and deletes when it is evaluated at a coordinator > > instead of being pushed down. > > > > The problem here is that CTID could be referring to a different tuple > > altogether on a different data node, which is what happened for one of > our > > Postgres-XC support customers, leading to data issues. > > > > Instead, the patch looks for a primary key or unique index (with the > primary > > key preferred) and uses those values instead of CTID. > > > > The patch could be improved further. Extra parameters are set even if > not > > used in the execution of the prepared statement sent down to the data > nodes. > > > > Regards, > > > > > > -- > > Mason Sharp > > > > TransLattice - http://www.translattice.com > > Distributed and Clustered Database Solutions > > > > > ------------------------------------------------------------------------------ > > November Webinars for C, C++, Fortran Developers > > Accelerate application performance with scalable programming models. > Explore > > techniques for threading, error checking, porting, and tuning. Get the > most > > from the latest Intel processors and coprocessors. See abstracts and > > register > > > http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk > > _______________________________________________ > > Postgres-xc-developers mailing list > > Pos...@li... > > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > ------------------------------------------------------------------------------ > Android apps run on BlackBerry 10 > Introducing the new BlackBerry 10.2.1 Runtime for Android apps. > Now with support for Jelly Bean, Bluetooth, Mapview and more. > Get your Android app in front of a whole new audience. Start now. > > http://pubads.g.doubleclick.net/gampad/clk?id=124407151&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company |
|
From: Koichi S. <koi...@gm...> - 2014-02-13 04:15:52
|
Hi, I tested the patch and found that primary key is mandatory. We need to modify regression test considerably to give each replicated table primary keys. I think this patch helps but I'm not afraid this is good, especially when we try to take XC features back to PG. Did you post another patch to use all column values if primary key is not available? I think better way is as follows: 1) If primary key is defined, use it, 2) If not, create a primary key as system column, the size should be 64bit. 3) If primary key is added to a replicated table, remove system primary key. The value of primary key can be obtained as follows: 1) add new column to pgxc_class catalog to represent maximum value of the system primary key, 2) when first "insert" is done to the primary node, system primary key value is taken from 1) and 1) is updated. The value is returned to the coordinator to be propagated to other nodes. 3) when subsequent "insert" is being done, system primary key value is added to the column value. In this case, each datanode updates 1) column value if it is larger than the current maximum value. 3) is important to change primary node to another. This is needed to carry over the primary node to another. ALTER TABLE should take care of them. Other issues are: 4) pg_dump/pg_dumpall should not include this system column value, 5) cluster may need to handle this too to repack system primary key value (not now but at least in 1.3 or later). Regards; --- Koichi Suzuki 2013-11-02 9:26 GMT+09:00 Mason Sharp <ms...@tr...>: > Please see attached patch that tries to address the issue of XC using CTID > for replicated updates and deletes when it is evaluated at a coordinator > instead of being pushed down. > > The problem here is that CTID could be referring to a different tuple > altogether on a different data node, which is what happened for one of our > Postgres-XC support customers, leading to data issues. > > Instead, the patch looks for a primary key or unique index (with the primary > key preferred) and uses those values instead of CTID. > > The patch could be improved further. Extra parameters are set even if not > used in the execution of the prepared statement sent down to the data nodes. > > Regards, > > > -- > Mason Sharp > > TransLattice - http://www.translattice.com > Distributed and Clustered Database Solutions > > ------------------------------------------------------------------------------ > November Webinars for C, C++, Fortran Developers > Accelerate application performance with scalable programming models. Explore > techniques for threading, error checking, porting, and tuning. Get the most > from the latest Intel processors and coprocessors. See abstracts and > register > http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > |
|
From: 鈴木 幸市 <ko...@in...> - 2014-02-13 00:55:31
|
Also, to remove serious danger associated with the current TRIGGER, I think we should apply Mason’s patch at least to 1.1. We should release 1.1.1 prior to 1.2.0. Thoughts? --- Koichi Suzuki 2014/02/13 1:18、Mason Sharp <ms...@tr...<mailto:ms...@tr...>> のメール： I’m afraid it takes long effort to fix all the influences of this change. How do you think about this? As I noted, approach C has good point. The issue is how long it takes. With approach B, we can easily change this handling to approach C. I’d like to have you opinion on this. It seems unnecessary if the table already has a primary key or unique index. Anyway, approach C is the approach that I originally took with GridSQL/Stado, adding something called xrowid, but we later disabled it by default. Was there any other reason of disabling it other than code simplicity and maintainability? Yes. We could have saved time and worked on other things. In hindsight I would have saved the trouble and not implemented it to keep the code simpler and easier to maintain, and just left it up to the user to use a key. To summarize, I would go with B. What is your stance on the fact that going with option B makes us backward in-compatible? Right now the situation is not good. Better to make it backward incompatible than to have data issues. I also don't believe the user base is too big as of yet to cause too many people headaches. Also, other RDBMSs require primary keys for replication. IIRC even Slony requires one. -- Mason Sharp TransLattice - http://www.translattice.com<http://www.translattice.com/> Distributed and Clustered Database Solutions |
|
From: David E. W. <da...@ju...> - 2014-02-12 17:28:19
|
On Feb 5, 2014, at 11:17 PM, Ahsan Hadi <ahs...@en...> wrote: > Abbas, > > Can you test this with 1.1? Perhaps we fixed this issue in the master which means that it will be part of 1.2 beta that is planned for this month. Hey All, Any word on this? Holding up a release Sqitch till I hear back. Thanks, David |
|
From: Mason S. <ms...@tr...> - 2014-02-12 16:19:01
|
> >>> I'm afraid it takes long effort to fix all the influences of this >>> change. How do you think about this? As I noted, approach C has good >>> point. The issue is how long it takes. With approach B, we can easily >>> change this handling to approach C. I'd like to have you opinion on this. >>> >> >> It seems unnecessary if the table already has a primary key or unique >> index. Anyway, approach C is the approach that I originally took with >> GridSQL/Stado, adding something called xrowid, but we later disabled it by >> default. >> > > Was there any other reason of disabling it other than code simplicity and > maintainability? > Yes. We could have saved time and worked on other things. > > >> In hindsight I would have saved the trouble and not implemented it to >> keep the code simpler and easier to maintain, and just left it up to the >> user to use a key. >> >> To summarize, I would go with B. >> > > What is your stance on the fact that going with option B makes us backward > in-compatible? > Right now the situation is not good. Better to make it backward incompatible than to have data issues. I also don't believe the user base is too big as of yet to cause too many people headaches. Also, other RDBMSs require primary keys for replication. IIRC even Slony requires one. -- Mason Sharp TransLattice - http://www.translattice.com Distributed and Clustered Database Solutions |
261 messages has been excluded from this view by a project administrator.
