You can subscribe to this list here.
2010 |
Jan
|
Feb
|
Mar
|
Apr
(10) |
May
(17) |
Jun
(3) |
Jul
|
Aug
|
Sep
(8) |
Oct
(18) |
Nov
(51) |
Dec
(74) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2011 |
Jan
(47) |
Feb
(44) |
Mar
(44) |
Apr
(102) |
May
(35) |
Jun
(25) |
Jul
(56) |
Aug
(69) |
Sep
(32) |
Oct
(37) |
Nov
(31) |
Dec
(16) |
2012 |
Jan
(34) |
Feb
(127) |
Mar
(218) |
Apr
(252) |
May
(80) |
Jun
(137) |
Jul
(205) |
Aug
(159) |
Sep
(35) |
Oct
(50) |
Nov
(82) |
Dec
(52) |
2013 |
Jan
(107) |
Feb
(159) |
Mar
(118) |
Apr
(163) |
May
(151) |
Jun
(89) |
Jul
(106) |
Aug
(177) |
Sep
(49) |
Oct
(63) |
Nov
(46) |
Dec
(7) |
2014 |
Jan
(65) |
Feb
(128) |
Mar
(40) |
Apr
(11) |
May
(4) |
Jun
(8) |
Jul
(16) |
Aug
(11) |
Sep
(4) |
Oct
(1) |
Nov
(5) |
Dec
(16) |
2015 |
Jan
(5) |
Feb
|
Mar
(2) |
Apr
(5) |
May
(4) |
Jun
(12) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(4) |
2019 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: ZhangJulian <jul...@ou...> - 2014-01-25 00:28:20
|
Hi All, I am thinking 3 choices as below: 1. Pacemaker and Corosync. I have little experience on Linux HA, so one week passed, I even can not install them successfully, including Pacemaker/Corosync/crmsh/resouce agent. There are some website mentioned Pacemaker/corosync can help PGXC to build a HA infrastructure, but I can not find a comprehensive guide to do it. There are much more commponents in PGXC than PG, I think I should learn how to build it based on PG first. 2. Zookeeper It seems that Zookeeper has the ability to build a HA solution for PGXC, which have the similar function with Pacemaker, but I have to develop the heartbeat function for Zookeeper to start/stop/monitor/failover PGXC. And I do not know if my understand is right. 3. PGXC support HA internally. Because the table of pgxc_nodes in coordinator already have some information about the cluster, it can be enhanced to save the Master/Slave relations, it is replicated between all coordinators, then it can used as a CRM(Cluster Resource Management, as Pacemaker) compoment. And the coordinator will connect to datanode/gtm/other coordinator in its regular work, so the heartbeat function exists natually. Even when the database is in the spare time, the coordinator can send a simple query as "select 1+1" to datanodes as the heartbeat ticks. What need to do is that, the coordinator will start a new process when starting, the new process will act as a heartbeat /resouce_agent to monitor the cluster status, and restart/failover once one commponent fails. As my initial understanding, Choice 3 is better than Choice 2 which is better than Choice 1. But for the development effort, the order is reversed, Choice 1 is easy achieved based on current existing codes. I am very appreciated that you can share your advice with me. Thanks Julian |
From: David E. W. <da...@ju...> - 2014-01-24 22:17:35
|
On Jan 24, 2014, at 12:24 AM, Michael Meskes <me...@po...> wrote: > We haven't completely solved the coexistance part yet, but mainly due to time > constraints. What we do so far is only build those pieces of XC that are > different and rely on the PG packages for the rest. Really? You actually have RPMs that depend on Postgres, and add the XC functionality in a binary distribution? > As for the docs, I remember the clean target not removing the generated SGML > files. However, we do have the specific manpages in place and I do not remember > them needing any special treatment. But then I may have simply forgotton about > that one. I just didn't realize they were in doc-xc instead of doc. >> I need to maintain corresponding PG docs for work and for merge >> process, it's convenient to have XC docs in a different directory. >> Do you think original PG docs should not be a part of XC release? > > I think they have to be part of it. After all people might just install XC > without PG alongside it. But the XC docs include everything from the pg docs, no? Best, David |
From: Masataka S. <pg...@gm...> - 2014-01-24 09:26:00
|
Hello, As I've been exasperated by random failures, I'm willing to whip the cause of the issue. This issue is related to cancel of the failed query. When a datanode reports an error of a query, a coordinator sends a cancel request to non-idle nodes, waits the node to get ready and requests nodes to rollback the transaction. Where's the problem? Consider the next case. 1. Datanode A (PID 1) reports an error to coordinator A. ([1] 'E' message) 2. Coordinator A receives [1] and reports an error to a frontend. ([2] 'E' message) 3. Coordinator A starts aborting process and it thinks datanode A (PID 1) is not idle. 4. Coordinator A sends a cancel request about PID 1 to datanode A (PID 2). ([3] cancel message) 5. Datanode A (PID 1) reports ready to coordinator A. ([4] 'Z' message) 6. Coordinator A receives [4] and sends "ROLLBACK TRANSACTION" immediately. ([5] 'Q' message) 7. Datanode A (PID 1) receives [5] and starts processing the query. 8. Datanode A (PID 2) receives [3]. 9. Datanode A (PID 2) notify PID 1 of [3]. 10. Datanode A (PID 1) cancel processing [5] and reports an error to Coordinator A. ([6] 'E' message) 11. Coordinator A receives [6] and reports an error to a frontend. ([7] 'E' message) [7] makes unexpected output and a test fails. Saying an extreme thing, it could occur that the next query of [5] is cancelled by [3]. As far as I know, there's no way to know when to the cancel request get to be processed, I think we can't not wait an experimental duration after cancelling like the attached patch. Does anyone have another cool idea to solve this issue? Regards. |
From: Michael M. <me...@po...> - 2014-01-24 08:24:19
|
On Thu, Jan 23, 2014 at 11:16:54AM +0900, Koichi Suzuki wrote: > We should consider the target installation directory especially when > XC coexists with PG. Michael Meskes had similar problem in XC > packaging for Debian and I hope he has some experience and ideas on > this. We haven't completely solved the coexistance part yet, but mainly due to time constraints. What we do so far is only build those pieces of XC that are different and rely on the PG packages for the rest. As for the docs, I remember the clean target not removing the generated SGML files. However, we do have the specific manpages in place and I do not remember them needing any special treatment. But then I may have simply forgotton about that one. > I need to maintain corresponding PG docs for work and for merge > process, it's convenient to have XC docs in a different directory. > Do you think original PG docs should not be a part of XC release? I think they have to be part of it. After all people might just install XC without PG alongside it. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: michael.meskes at gmail dot com VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL |
From: Koichi S. <koi...@gm...> - 2014-01-24 01:48:02
|
Hi, 2014/1/24 David E. Wheeler <da...@ju...>: > On Jan 22, 2014, at 6:16 PM, Koichi Suzuki <koi...@gm...> wrote: > >> Thank you very much for the effort and sorry for the late response. >> I agree on your points. Yes, as PG, tarball directory should be >> postgres-xc-1.1 or so. Please let me change this in the following >> releases. > > I also suggest you make it 1.1.0 in the future, so that the format is the same if you later release 1.1.1. > >> As per documents, release tarballs include html and man. Which is in >> doc-xc/src/sgml, not in doc. They all comes with XC-specific stuff >> as well as differences as you found in XC pages. The location of >> docs is not described clearly in the notes, which need to improve. > > Ahhhh! There they are! I will fix the spec file today. > >> We should consider the target installation directory especially when >> XC coexists with PG. Michael Meskes had similar problem in XC >> packaging for Debian and I hope he has some experience and ideas on >> this. > > I’ve been chatting with Jeff Frost and Devrim GÜNDÜZ about it, too. Devrim seems to think that there should be a separate XC yum repo, perhaps under the postgres-xc.com domain. Are you suggesting to have separate site as yum (and could be deb) repo for XC? It will be nice if somebody can help this and to move current XC pages (now in sourceforge wiki) to a separate place. Now current active members are fully booked in development, test and fixes. I appreciate for any of such help. > >> I need to maintain corresponding PG docs for work and for merge >> process, it's convenient to have XC docs in a different directory. >> Do you think original PG docs should not be a part of XC release? > > No, if the XC docs are a superset of the PG docs, then you don't need to build and ship the PG docs. Might make sense to not ship their SGML at all. Maybe `rm -rf doc && mv doc-xc doc`. Okay. As you have noticed, XC doc comes with the note which is the same as PG and which is different in XC. In this sense, XC doc is a superset of PG docs. > >> On the other hand I'm worrying is Mason Sharp has already published >> his rpms for linux 64bit and 32bit at his stormDB page. I don't >> know how long he can continue this work and it will be nice if you >> visit his page and see how we can work together. > > Ah-ha! I was not aware of this. Seems like quite a bit of overlap. > > http://yum.stormdb.com/repos/Postgres-XC/1.1.0/centos64/ > >> Please understand I highly appreciate for efforts of XC packaging, >> which is a great help for XC deployment. >> >> Mason, any more inputs? > > We should certainly collaborate to create a “community” repo, I think. Thoughts? I believe it is a good idea to collect each effort. In this case, yes you could collaborate with Mason. Could you launch this discussion in "general" or "developers" mailing list? > > BTW, I also created a separate gtm RPM. > > https://github.com/theory/postgres-xc-rpm/commit/f94354a0365f15fa6850d24024bb5a59d2b202d8 > > Best, > > David > Thank you; --- Koichi Suzuki |
From: Koichi S. <koi...@gm...> - 2014-01-24 01:39:42
|
Sorry for taking long. We're now working hard to merge and port PostgreSQL 9.3 features and fixes into XC. It is taking longer than I first expected. The first effort will be available as early as next week and then I can include your improvement to our master (I began this effort last October). Good things on this work is that major new PostgreSQL feature will be available with XC as well, including event triggers, updatable views, LATERAL and material views. This tool longer because considerable changes in the planner internal affected existing XC planner as well. Your patch will be included just after these are in XC's master branch. I appreciate for your patience. Regards; --- Koichi Suzuki 2014/1/24 Matt Warner <MW...@xi...>: > It's been several months since I submitted the changes that bring XC into line with the 9.3 release of Postgres, including bringing configure.in up to date. > > I'm interested in having this work integrated. What can I do to assist? > > -----Original Message----- > From: Michael Paquier [mailto:mic...@gm...] > Sent: Thursday, October 24, 2013 3:59 PM > To: Matt Warner > Cc: Postgres-XC Developers; Suzuki; 鈴木 幸市 (ko...@in...) > Subject: Re: [Postgres-xc-developers] Minor Fixes > > On Fri, Oct 25, 2013 at 1:14 AM, Matt Warner <MW...@xi...> wrote: >> Good morning. I didn't see any responses to this. >> >> I know everyone is busy. Were the changes acceptable? > Sorry, I'm quite busy the last couple of weeks, so I got no time to > look at your patches. > -- > Michael > > ------------------------------------------------------------------------------ > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers |
From: David E. W. <da...@ju...> - 2014-01-24 00:45:38
|
On Jan 23, 2014, at 9:28 AM, David E. Wheeler <da...@ju...> wrote: > Ahhhh! There they are! I will fix the spec file today. Done. https://github.com/theory/postgres-xc-rpm/commit/5a7b7cb3e735e8b8faf99ad6081e72cf7f790dde Best, David |
From: Matt W. <MW...@xi...> - 2014-01-23 17:53:36
|
It's been several months since I submitted the changes that bring XC into line with the 9.3 release of Postgres, including bringing configure.in up to date. I'm interested in having this work integrated. What can I do to assist? -----Original Message----- From: Michael Paquier [mailto:mic...@gm...] Sent: Thursday, October 24, 2013 3:59 PM To: Matt Warner Cc: Postgres-XC Developers; Suzuki; 鈴木 幸市 (ko...@in...) Subject: Re: [Postgres-xc-developers] Minor Fixes On Fri, Oct 25, 2013 at 1:14 AM, Matt Warner <MW...@xi...> wrote: > Good morning. I didn't see any responses to this. > > I know everyone is busy. Were the changes acceptable? Sorry, I'm quite busy the last couple of weeks, so I got no time to look at your patches. -- Michael |
From: David E. W. <da...@ju...> - 2014-01-23 17:28:59
|
On Jan 22, 2014, at 6:16 PM, Koichi Suzuki <koi...@gm...> wrote: > Thank you very much for the effort and sorry for the late response. > I agree on your points. Yes, as PG, tarball directory should be > postgres-xc-1.1 or so. Please let me change this in the following > releases. I also suggest you make it 1.1.0 in the future, so that the format is the same if you later release 1.1.1. > As per documents, release tarballs include html and man. Which is in > doc-xc/src/sgml, not in doc. They all comes with XC-specific stuff > as well as differences as you found in XC pages. The location of > docs is not described clearly in the notes, which need to improve. Ahhhh! There they are! I will fix the spec file today. > We should consider the target installation directory especially when > XC coexists with PG. Michael Meskes had similar problem in XC > packaging for Debian and I hope he has some experience and ideas on > this. I’ve been chatting with Jeff Frost and Devrim GÜNDÜZ about it, too. Devrim seems to think that there should be a separate XC yum repo, perhaps under the postgres-xc.com domain. > I need to maintain corresponding PG docs for work and for merge > process, it's convenient to have XC docs in a different directory. > Do you think original PG docs should not be a part of XC release? No, if the XC docs are a superset of the PG docs, then you don't need to build and ship the PG docs. Might make sense to not ship their SGML at all. Maybe `rm -rf doc && mv doc-xc doc`. > On the other hand I'm worrying is Mason Sharp has already published > his rpms for linux 64bit and 32bit at his stormDB page. I don't > know how long he can continue this work and it will be nice if you > visit his page and see how we can work together. Ah-ha! I was not aware of this. Seems like quite a bit of overlap. http://yum.stormdb.com/repos/Postgres-XC/1.1.0/centos64/ > Please understand I highly appreciate for efforts of XC packaging, > which is a great help for XC deployment. > > Mason, any more inputs? We should certainly collaborate to create a “community” repo, I think. Thoughts? BTW, I also created a separate gtm RPM. https://github.com/theory/postgres-xc-rpm/commit/f94354a0365f15fa6850d24024bb5a59d2b202d8 Best, David |
From: Koichi S. <koi...@gm...> - 2014-01-23 02:17:02
|
Hello David; Thank you very much for the effort and sorry for the late response. I agree on your points. Yes, as PG, tarball directory should be postgres-xc-1.1 or so. Please let me change this in the following releases. As per documents, release tarballs include html and man. Which is in doc-xc/src/sgml, not in doc. They all comes with XC-specific stuff as well as differences as you found in XC pages. The location of docs is not described clearly in the notes, which need to improve. We should consider the target installation directory especially when XC coexists with PG. Michael Meskes had similar problem in XC packaging for Debian and I hope he has some experience and ideas on this. I need to maintain corresponding PG docs for work and for merge process, it's convenient to have XC docs in a different directory. Do you think original PG docs should not be a part of XC release? On the other hand I'm worrying is Mason Sharp has already published his rpms for linux 64bit and 32bit at his stormDB page. I don't know how long he can continue this work and it will be nice if you visit his page and see how we can work together. Please understand I highly appreciate for efforts of XC packaging, which is a great help for XC deployment. Mason, any more inputs? Regards; --- Koichi Suzuki 2014/1/22 David E. Wheeler <da...@ju...>: > On Jan 21, 2014, at 1:30 PM, David E. Wheeler <da...@ju...> wrote: > >> * Have you considered building the docs before shipping? Most folks don’t have jade installed, so I think the Postgres core guys build the docs before taring up the release. For the RPM I'm just building them, so no biggie there, but I doubt I’d ever build them on my Mac. > > Oh, BTW, this does not appear to generate the man pages for the XC-specific stuff. For example, there are no gtm.1, gtm_proxy.1, initgtm.1, or gtm_ctl.1 files in share/man/man1/. Also, should there be man pages in share/man/man3/ for XC-speific commands? > > Best, > > David > > > ------------------------------------------------------------------------------ > CenturyLink Cloud: The Leader in Enterprise Cloud Services. > Learn Why More Businesses Are Choosing CenturyLink Cloud For > Critical Workloads, Development Environments & Everything In Between. > Get a Quote or Start a Free Trial Today. > http://pubads.g.doubleclick.net/gampad/clk?id=119420431&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers |
From: David E. W. <da...@ju...> - 2014-01-22 00:02:23
|
On Jan 21, 2014, at 1:30 PM, David E. Wheeler <da...@ju...> wrote: > * Have you considered building the docs before shipping? Most folks don’t have jade installed, so I think the Postgres core guys build the docs before taring up the release. For the RPM I'm just building them, so no biggie there, but I doubt I’d ever build them on my Mac. Oh, BTW, this does not appear to generate the man pages for the XC-specific stuff. For example, there are no gtm.1, gtm_proxy.1, initgtm.1, or gtm_ctl.1 files in share/man/man1/. Also, should there be man pages in share/man/man3/ for XC-speific commands? Best, David |
From: David E. W. <da...@ju...> - 2014-01-21 21:47:09
|
Postgres-XCers, I forked Devrim’s RPM spec file for Postgres 9.3 on EL-6 and started on building RPMs for XC. You can see my progress here: https://github.com/theory/postgres-xc-rpm It successfully builds these RPMs: * postgresxc11-docs-1.1.0 * postgresxc11-contrib-1.1.0 * postgresxc11-plperl-1.1.0 * postgresxc11-server-1.1.0 * postgresxc11-pltcl-1.1.0 * postgresxc11-debuginfo-1.1.0 * postgresxc11-devel-1.1.0 * postgresxc11-test-1.1.0 * postgresxc11-1.1.0 * postgresxc11-libs-1.1.0 * postgresxc11-plpython-1.1.0 Not bad, right? However, I have a few questions and suggestions about things I’ve found: * I suggest using the version in the decompressed directory prefix. The tarball is called “pgxc-v1.1.tar.gz”, but decompresses into a directory named “postgres-xc”. I think these should be the same. In other words, the the directory should be named “pgxc-v1.1”. Or, if you wanted things to be closer to what Postgres and other projects do, maybe call the tarball “postgres-xc-1.1.tar.gz” and the directory “postgres-xc-1.1”. Or maybe “postgresxc-1.1.tar.gz” and the directory “postgresxc-1.1”. Thoughts? * I’m using “postgresxc” for the RPM name. This keeps it more or less in line with the Postgres core, which uses “postgresql”. Make sense to you? Or is the hyphen strongly preferred? * I’m building with the directory path /usr/pgxc-1.1 instead of /usr/pgsql-1.1 (or /usr/pgsql-9.2). I think it would be reasonable for the default to use “pgxc” instead of “pgsql”, again, to keep the two things distinct. Thoughts? * In that vein, it would be useful to allow “pgxc” in the directory path. I refer to the “NOTE” just below the docs for --htmldir here: http://postgres-xc.sourceforge.net/docs/1_1/install-procedure.html It says: > First, the string "/postgresql" is automatically appended to datadir, sysconfdir, and docdir, unless the fully expanded directory name already contains the string "postgres" or "pgsql" I suggest appending “or pgxc” to that list. Here’s a patch for that: https://github.com/theory/postgres-xc-rpm/blob/master/SOURCES/rpm-pgxc.patch (Though that patch also replaces “/postgresql” with “/pgxc”, which you might not want. Maybe it should be “/postgrexc”?) * Have you considered building the docs before shipping? Most folks don’t have jade installed, so I think the Postgres core guys build the docs before taring up the release. For the RPM I'm just building them, so no biggie there, but I doubt I’d ever build them on my Mac. * Finally, does the pgxc community have an interest in hosting a Yum repository the way pgrpms does? Or should I just try to get it into pgrpms and see what happens? It might be more complicated than they want, since they build all the other stuff around each version of Postgres. So you can install postgres92 or postgres93, and if you want PostGIS, you install the corresponding postgis92 or postgis93. Not sure how that would be handled for XC. I mean, it would be postgis11, but eventually there might be overlap in the version, right? Will be starting an experiment with XC over the next few weeks, so I expect to have more questions, though probably fewer about packaging from here on in. :-) Thanks, David |
From: Masataka S. <pg...@gm...> - 2014-01-17 07:47:21
|
I would take option #1 if possible, but I think estimating impact of the fix and testing requires intimate knowledge of SQL and the optimizer: How the planner optimize a plan tree and what SQL could have two entries in a target list with the same value, How ressortgroup, resno, plan's target list and remote query's target list and base tlist is used by the optimizer and executor. It is still hard for me. My concern with 2nd option is how's the performance compared to the shipped case and how much case are there that SQLs is not optimized. I made some aggregation queries to test how's the query optimized with the attached patch, and I found that just returning false in pgxc_locate_grouping_columns() is not enough to fix this problem. For example, the next query generates a borked plan. postgres=# CREATE TABLE tbl AS SELECT * FROM (SELECT generate_series(1,3) as i) a, (SELECT generate_series(1,3) AS j) b; INSERT 0 9 postgres=# EXPLAIN VERBOSE SELECT i AS a, i AS b, avg(j) AS c FROM tblGROUP BY a ORDER BY b; QUERY PLAN --------------------------------------------------------------------------------------------------- GroupAggregate (cost=49.83..57.45 rows=10 width=8) Output: tbl.i, tbl.i, avg(tbl.i) -> Sort (cost=49.83..52.33 rows=1000 width=8) Output: tbl.i, tbl.i Sort Key: tbl.i -> Data Node Scan on tbl "_REMOTE_TABLE_QUERY_" (cost=0.00..0.00 rows=1000 width=8) Output: tbl.i, tbl.i, (avg(tbl.j)) Node/s: datanode1, datanode2 Remote query: SELECT i, pg_catalog.int8_avg(avg(j)) FROM ONLY public.tbl WHERE true (9 rows) Of course, it fails. postgres=# SELECT i AS a, i AS b, avg(j) AS c FROM tbl GROUP BY a ORDER BY b; ERROR: column "tbl.i" must appear in the GROUP BY clause or be used in an aggregate function In any case, this problem needs more study. Regards. On Wed, Jan 15, 2014 at 7:18 PM, Ashutosh Bapat < ash...@en...> wrote: > > > > On Wed, Jan 15, 2014 at 1:55 PM, Masataka Saito <pg...@gm...> wrote: > >> Hi Ashutosh, >> >> Specified groupColIdx doesn't found in tlist. >> >> It seems that groupColIdx is correct and tlist is wrong. >> Because the value of ressortgroupref in the target entry(base_tlist) >> they want to locate was zero. >> >> The mechanism why it comes to be zero is that when >> pgxc_rqplan_adjust_tlist updates base_tlist using >> pgxc_build_shippable_tlist, a ressortgroupref in a target entry is reset >> if a plan's target list has two equal Vars. >> >> > You have spotted the problem correctly. While writing this code, I thought > that having two entries in target list with same expression would be > problem when it comes to setting references (set_plan_ref), and hence chose > to disallow that. The assumption there was, if locate_grouping_columns() > doesn't find the column, it would return an invalid value, and thus we can > choose not to optimize that case. But locate_grouping_columns() is actually > throwing error, which breaks the assumption. > > So, there are two possibilities > 1. bite the bullet and let there be two entries in the targetlist with > same expressions but different ressortgroupref and see if that approach has > any other problems - we will have to test a bunch of scenarios to make sure > that there is no problem there. > 2. Modify pgxc_locate_grouping_columns() to return NULL when no expression > is found with matching ressortgroupref. But that might need some magic with > PG_TRY() and CATCH or code duplication. > > >> It occurs when SELECT target has two aliases to the same column and each >> are referred by GROUP BY clause and ORDER BY clause. I will show a >> sample query at next. >> >> postgres-# CREATE TABLE tbl(i INT); >> postgres-# SELECT i AS a, i AS b FROM tbl GROUP BY a ORDER BY b; >> >> It needs more analysis, but I'm thinking how about using >> locate_grouping_columns with plan's target list and base_tlist instead >> of pgxc_locate_grouping_columns. >> >> Regards. >> >> >> On Thu, Nov 21, 2013 at 1:40 PM, Ashutosh Bapat < >> ash...@en...> wrote: >> >>> Hi Masataka, >>> To start debugging, you might want to fire this query with debugger >>> breaking on errstart() or errfinish() or some error reporting hook where >>> you can break. The stack stress should give you a lead as to where the >>> error is coming from. It's high chance that it's coming from >>> pgxc_locate_grouping_columns(). But anyway, start looking around the >>> callers of get_sortgroupref_tle() and check their arguments. Best luck. >>> >>> >>> On Thu, Nov 21, 2013 at 6:51 AM, Masataka Saito <pg...@gm...>wrote: >>> >>>> Thanks for your information. >>>> It seems very helpful. >>>> >>>> cx=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id = >>>> t2.id GROUP BY b; >>>> ERROR: XX000: ORDER/GROUP BY expression not found in targetlist >>>> LOCATION: get_sortgroupref_tle, tlist.c:251 >>>> >>>> On Thu, Nov 21, 2013 at 9:53 AM, 鈴木 幸市 <ko...@in...> >>>> wrote: >>>> > Yes, you can do it with SET command as well just for a target >>>> statement. >>>> > >>>> > Regards; >>>> > --- >>>> > Koichi Suzuki >>>> > >>>> > 2013/11/21 7:46、Michael Paquier <mic...@gm...> のメール: >>>> > >>>> >> On Thu, Nov 21, 2013 at 12:32 AM, Masataka Saito <pg...@gm...> >>>> wrote: >>>> >>> XC can't build a plan. >>>> >>> >>>> >>> db=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id= >>>> >>> t2.id GROUP BY b; >>>> >>> ERROR: ORDER/GROUP BY expression not found in targetlist >>>> >> This is a higher-level bug, planner bug just by looking at this code >>>> >> path src/backend/optimizer/util/tlist.c... >>>> >> >>>> >> Note: Setting up VERBOSITY to verbose in .psqlrc helps grabbing more >>>> >> details about the errors that occurred in server like the file name + >>>> >> name of this elog/ereport ERROR: >>>> >> \set VERBOSITY verbose >>>> >> For dev purposes it is a pretty useful default ;) >>>> >> >>>> >> Regards, >>>> >> -- >>>> >> Michael >>>> >> >>>> >> >>>> ------------------------------------------------------------------------------ >>>> >> Shape the Mobile Experience: Free Subscription >>>> >> Software experts and developers: Be at the forefront of tech >>>> innovation. >>>> >> Intel(R) Software Adrenaline delivers strategic insight and >>>> game-changing >>>> >> conversations that shape the rapidly evolving mobile landscape. Sign >>>> up now. >>>> >> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk >>>> >> _______________________________________________ >>>> >> Postgres-xc-developers mailing list >>>> >> Pos...@li... >>>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>>> >> >>>> > >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Shape the Mobile Experience: Free Subscription >>>> Software experts and developers: Be at the forefront of tech innovation. >>>> Intel(R) Software Adrenaline delivers strategic insight and >>>> game-changing >>>> conversations that shape the rapidly evolving mobile landscape. Sign up >>>> now. >>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> Postgres-xc-developers mailing list >>>> Pos...@li... >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>>> >>> >>> >>> >>> -- >>> Best Wishes, >>> Ashutosh Bapat >>> EnterpriseDB Corporation >>> The Postgres Database Company >>> >> >> > > > -- > Best Wishes, > Ashutosh Bapat > EnterpriseDB Corporation > The Postgres Database Company > |
From: Ashutosh B. <ash...@en...> - 2014-01-15 10:19:03
|
On Wed, Jan 15, 2014 at 1:55 PM, Masataka Saito <pg...@gm...> wrote: > Hi Ashutosh, > > Specified groupColIdx doesn't found in tlist. > > It seems that groupColIdx is correct and tlist is wrong. > Because the value of ressortgroupref in the target entry(base_tlist) they > want to locate was zero. > > The mechanism why it comes to be zero is that when > pgxc_rqplan_adjust_tlist updates base_tlist using > pgxc_build_shippable_tlist, a ressortgroupref in a target entry is reset > if a plan's target list has two equal Vars. > > You have spotted the problem correctly. While writing this code, I thought that having two entries in target list with same expression would be problem when it comes to setting references (set_plan_ref), and hence chose to disallow that. The assumption there was, if locate_grouping_columns() doesn't find the column, it would return an invalid value, and thus we can choose not to optimize that case. But locate_grouping_columns() is actually throwing error, which breaks the assumption. So, there are two possibilities 1. bite the bullet and let there be two entries in the targetlist with same expressions but different ressortgroupref and see if that approach has any other problems - we will have to test a bunch of scenarios to make sure that there is no problem there. 2. Modify pgxc_locate_grouping_columns() to return NULL when no expression is found with matching ressortgroupref. But that might need some magic with PG_TRY() and CATCH or code duplication. > It occurs when SELECT target has two aliases to the same column and each > are referred by GROUP BY clause and ORDER BY clause. I will show a sample > query at next. > > postgres-# CREATE TABLE tbl(i INT); > postgres-# SELECT i AS a, i AS b FROM tbl GROUP BY a ORDER BY b; > > It needs more analysis, but I'm thinking how about using > locate_grouping_columns with plan's target list and base_tlist instead of > pgxc_locate_grouping_columns. > > Regards. > > > On Thu, Nov 21, 2013 at 1:40 PM, Ashutosh Bapat < > ash...@en...> wrote: > >> Hi Masataka, >> To start debugging, you might want to fire this query with debugger >> breaking on errstart() or errfinish() or some error reporting hook where >> you can break. The stack stress should give you a lead as to where the >> error is coming from. It's high chance that it's coming from >> pgxc_locate_grouping_columns(). But anyway, start looking around the >> callers of get_sortgroupref_tle() and check their arguments. Best luck. >> >> >> On Thu, Nov 21, 2013 at 6:51 AM, Masataka Saito <pg...@gm...> wrote: >> >>> Thanks for your information. >>> It seems very helpful. >>> >>> cx=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id = >>> t2.id GROUP BY b; >>> ERROR: XX000: ORDER/GROUP BY expression not found in targetlist >>> LOCATION: get_sortgroupref_tle, tlist.c:251 >>> >>> On Thu, Nov 21, 2013 at 9:53 AM, 鈴木 幸市 <ko...@in...> wrote: >>> > Yes, you can do it with SET command as well just for a target >>> statement. >>> > >>> > Regards; >>> > --- >>> > Koichi Suzuki >>> > >>> > 2013/11/21 7:46、Michael Paquier <mic...@gm...> のメール: >>> > >>> >> On Thu, Nov 21, 2013 at 12:32 AM, Masataka Saito <pg...@gm...> >>> wrote: >>> >>> XC can't build a plan. >>> >>> >>> >>> db=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id = >>> >>> t2.id GROUP BY b; >>> >>> ERROR: ORDER/GROUP BY expression not found in targetlist >>> >> This is a higher-level bug, planner bug just by looking at this code >>> >> path src/backend/optimizer/util/tlist.c... >>> >> >>> >> Note: Setting up VERBOSITY to verbose in .psqlrc helps grabbing more >>> >> details about the errors that occurred in server like the file name + >>> >> name of this elog/ereport ERROR: >>> >> \set VERBOSITY verbose >>> >> For dev purposes it is a pretty useful default ;) >>> >> >>> >> Regards, >>> >> -- >>> >> Michael >>> >> >>> >> >>> ------------------------------------------------------------------------------ >>> >> Shape the Mobile Experience: Free Subscription >>> >> Software experts and developers: Be at the forefront of tech >>> innovation. >>> >> Intel(R) Software Adrenaline delivers strategic insight and >>> game-changing >>> >> conversations that shape the rapidly evolving mobile landscape. Sign >>> up now. >>> >> >>> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk >>> >> _______________________________________________ >>> >> Postgres-xc-developers mailing list >>> >> Pos...@li... >>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> >> >>> > >>> >>> >>> ------------------------------------------------------------------------------ >>> Shape the Mobile Experience: Free Subscription >>> Software experts and developers: Be at the forefront of tech innovation. >>> Intel(R) Software Adrenaline delivers strategic insight and game-changing >>> conversations that shape the rapidly evolving mobile landscape. Sign up >>> now. >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Postgres-xc-developers mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> >> >> >> >> -- >> Best Wishes, >> Ashutosh Bapat >> EnterpriseDB Corporation >> The Postgres Database Company >> > > -- Best Wishes, Ashutosh Bapat EnterpriseDB Corporation The Postgres Database Company |
From: Masataka S. <pg...@gm...> - 2014-01-15 08:26:10
|
Hi Ashutosh, Specified groupColIdx doesn't found in tlist. It seems that groupColIdx is correct and tlist is wrong. Because the value of ressortgroupref in the target entry(base_tlist) they want to locate was zero. The mechanism why it comes to be zero is that when pgxc_rqplan_adjust_tlist updates base_tlist using pgxc_build_shippable_tlist, a ressortgroupref in a target entry is reset if a plan's target list has two equal Vars. It occurs when SELECT target has two aliases to the same column and each are referred by GROUP BY clause and ORDER BY clause. I will show a sample query at next. postgres-# CREATE TABLE tbl(i INT); postgres-# SELECT i AS a, i AS b FROM tbl GROUP BY a ORDER BY b; It needs more analysis, but I'm thinking how about using locate_grouping_columns with plan's target list and base_tlist instead of pgxc_locate_grouping_columns. Regards. On Thu, Nov 21, 2013 at 1:40 PM, Ashutosh Bapat < ash...@en...> wrote: > Hi Masataka, > To start debugging, you might want to fire this query with debugger > breaking on errstart() or errfinish() or some error reporting hook where > you can break. The stack stress should give you a lead as to where the > error is coming from. It's high chance that it's coming from > pgxc_locate_grouping_columns(). But anyway, start looking around the > callers of get_sortgroupref_tle() and check their arguments. Best luck. > > > On Thu, Nov 21, 2013 at 6:51 AM, Masataka Saito <pg...@gm...> wrote: > >> Thanks for your information. >> It seems very helpful. >> >> cx=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id = >> t2.id GROUP BY b; >> ERROR: XX000: ORDER/GROUP BY expression not found in targetlist >> LOCATION: get_sortgroupref_tle, tlist.c:251 >> >> On Thu, Nov 21, 2013 at 9:53 AM, 鈴木 幸市 <ko...@in...> wrote: >> > Yes, you can do it with SET command as well just for a target statement. >> > >> > Regards; >> > --- >> > Koichi Suzuki >> > >> > 2013/11/21 7:46、Michael Paquier <mic...@gm...> のメール: >> > >> >> On Thu, Nov 21, 2013 at 12:32 AM, Masataka Saito <pg...@gm...> >> wrote: >> >>> XC can't build a plan. >> >>> >> >>> db=# explain verbose SELECT DISTINCT t2.b FROM t1 JOIN t2 ON t1.id = >> >>> t2.id GROUP BY b; >> >>> ERROR: ORDER/GROUP BY expression not found in targetlist >> >> This is a higher-level bug, planner bug just by looking at this code >> >> path src/backend/optimizer/util/tlist.c... >> >> >> >> Note: Setting up VERBOSITY to verbose in .psqlrc helps grabbing more >> >> details about the errors that occurred in server like the file name + >> >> name of this elog/ereport ERROR: >> >> \set VERBOSITY verbose >> >> For dev purposes it is a pretty useful default ;) >> >> >> >> Regards, >> >> -- >> >> Michael >> >> >> >> >> ------------------------------------------------------------------------------ >> >> Shape the Mobile Experience: Free Subscription >> >> Software experts and developers: Be at the forefront of tech >> innovation. >> >> Intel(R) Software Adrenaline delivers strategic insight and >> game-changing >> >> conversations that shape the rapidly evolving mobile landscape. Sign >> up now. >> >> >> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk >> >> _______________________________________________ >> >> Postgres-xc-developers mailing list >> >> Pos...@li... >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> >> > >> >> >> ------------------------------------------------------------------------------ >> Shape the Mobile Experience: Free Subscription >> Software experts and developers: Be at the forefront of tech innovation. >> Intel(R) Software Adrenaline delivers strategic insight and game-changing >> conversations that shape the rapidly evolving mobile landscape. Sign up >> now. >> >> http://pubads.g.doubleclick.net/gampad/clk?id=63431311&iu=/4140/ostg.clktrk >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> > > > > -- > Best Wishes, > Ashutosh Bapat > EnterpriseDB Corporation > The Postgres Database Company > |
From: Koichi S. <koi...@gm...> - 2013-12-30 13:33:44
|
A) GTM standby writes its backup (both usual backup and Barrier backup) at the same timing as GTM master so you don't have to restore these data from the master to the slave at the failover. Please note that usual backup timing may be different from the master and the slave. In both cases, backup advances GTM and sequence value from the current value so they can be used as a restart point. B) You just restart GTM standby at the same or a different server. GTM standby will backup master status, writes restart point and then it continues restart point as A). Please note that it does not backup existing Barrier restart points so you may have to backup them manually. Regards; --- Koichi Suzuki 2013/12/27 Tomonari Katsumata <kat...@po...>: > Hi, > > I'm investigating about the behavior of GTM/GTM_standby. > Because I'm thinking about how making GTM more High Availability. > To clarify the behavior, I want to understand some things below. > > A)When do GTM/GTM_standby write data to their disk? > Currently I know that GTM writes data in below case. > > 1)Stopping GTM > 2)GXID/Sequence are over the limit > 3)Executing CREATE BARRIER > > Are there another case? > And then, what timing does GTM_standby write data at? > > B)When occurring something wrong against GTM/GTM_standby, how do they work? > > a)How does GTM work without write permission on the disk? > According to the situation 1),2),3), the behavior is changed? > 1)GTM continues to work(gtm_ctl returns ERROR) > 2)GTM Stopped abnormally > 3)GTM continues to work, but backup file is never created > > Are there another behavior? > > b)How does GTM_standby work without write permission on the disk? > - GTM_standby will stop abnormally? > - How GTM_standby work when executing promote in this situation? > > regards, > --------------- > NTT Software Corporation > Tomonari Katsumata > > > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers |
From: Tomonari K. <kat...@po...> - 2013-12-27 05:40:54
|
Hi, I'm investigating about the behavior of GTM/GTM_standby. Because I'm thinking about how making GTM more High Availability. To clarify the behavior, I want to understand some things below. A)When do GTM/GTM_standby write data to their disk? Currently I know that GTM writes data in below case. 1)Stopping GTM 2)GXID/Sequence are over the limit 3)Executing CREATE BARRIER Are there another case? And then, what timing does GTM_standby write data at? B)When occurring something wrong against GTM/GTM_standby, how do they work? a)How does GTM work without write permission on the disk? According to the situation 1),2),3), the behavior is changed? 1)GTM continues to work(gtm_ctl returns ERROR) 2)GTM Stopped abnormally 3)GTM continues to work, but backup file is never created Are there another behavior? b)How does GTM_standby work without write permission on the disk? - GTM_standby will stop abnormally? - How GTM_standby work when executing promote in this situation? regards, --------------- NTT Software Corporation Tomonari Katsumata |
From: Michael M. <me...@po...> - 2013-12-22 09:26:03
|
Koichi-san, > As you know, XC does not add any changes to configure, except for XC version number and related settings. > > If the error is from XC-specific code, they could be improved as well. > > Could you let me know how the build failed? I don’t have S390 build environment. It took a while to get answers from some claiming to be in the know but I finally got to talk to one of these guys and he said that there is apparently nothing you can do but using -fPIC instead of -fpic for s390 and maybe even others. He thinks the problem simply comes from the library exceeding the size limit for -fpic. If this holds true PostgreSQL proper may run into the same problem eventually. I don't have any s390 development experience myself, so I cannot verify this information easily, though. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org Jabber: michael.meskes at gmail dot com VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL |
From: Lin W. <lin...@gm...> - 2013-12-13 15:53:41
|
From: Masataka S. <pg...@gm...> - 2013-12-13 04:44:28
|
Hi, all. I got an error report that EXECUTE DIRECT queries with particular GROUP BY clause return strange binary results and I made brief analysis. The next query returns strang result. EXECUTE DIRECT ON (datanode1) $$SELECT relkind FROM pg_class GROUP BY relname, relkind$$; It should return a result like Table A, but it returns a result like Table B. *Table A* relkind --------- i r v t (4 rows) *Table B* relkind --------- h \x18 ` 8 \x10 X 0 x \x08 P ( p H A query to a regular table work well. cx=# create table tbl as select generate_series(1,10) as a, generate_series(1,10) as b; INSERT 0 10 cx=# execute direct on (datanode2) $$select b from tbl group by a,b$$; b ---- 7 10 4 3 (4 rows) On master_pg93_merge, this issue may cause psql disconnection. This query doesn't make coordinator to be aborted. cx=# execute direct on (datanode1) $$select tablename from pg_catalog.pg_tables group by schemaname,tablename$$; The connection to the server was lost. Attempting reset: Failed. !> And next queries return the right results. # execute direct on (datanode1) $$select relkind from pg_class group by relkind$$; # execute direct on (datanode1) $$select relname, relkind from pg_class group by relname, relkind$$; It seems that this problem is produced when SELECT statement to pg_catalog is issued and the selection target does not contain all of the columns listed in the GROUP BY clause. A datanode is returning the right result but a coordinator is returning wrong result. I think this issue is a bug of coordinator. Regards. |
From: Michael P. <mic...@gm...> - 2013-12-09 07:33:44
|
On Thu, Dec 5, 2013 at 11:22 AM, normandi <nor...@al...> wrote: > Hi all, > > When reading the Postgres-XC document, I got the following two questions > (the same in nature maybe :) ) > > Q1: Why XC need the GTM component? GTM needs what is called in the XC language global snapshot, that gives to every session anywhere in the cluster an image of the transactions running. This is essential to keep data consistent across the nodes and is well adapted for the definition of MVCC that sticks to PostgreSQL. > In my opinion,if a transaction is submitted at the coord node , the coord > should break it > up into a collection of subtransactions which will be submit to the > corresponding data nodes, > and the subtransaction will be resolved by datanode's local transaction > manager. And could you explain how do you manage consistency of data for replicated tables? > Coordinator can decide whether commit the transaction or not ,by means of > 2PC/3PC protocol. 2PC is already costly, and I am not going to imagine how your performance will drop particularly for replicated tables. > Q2: "same Timestamp view", how it will affect the App clients? What is > means ? When a transaction begins, the timestamp indicating when the transaction has begun is recorded on GTM as well, and is registered centrally by GTM. The same behavior applies for current_timestamp for example. clock_timestamp, on the contrary, fetch the timestamp of the local node clock. -- Michael |
From: normandi <nor...@al...> - 2013-12-05 02:22:22
|
Hi all, When reading the Postgres-XC document, I got the following two questions (the same in nature maybe :) ) Q1: Why XC need the GTM component? In my opinion,if a transaction is submitted at the coord node , the coord should break it up into a collection of subtransactions which will be submit to the corresponding data nodes,and the subtransaction will be resolved by datanode's local transaction manager. Coordinator can decide whether commit the transaction or not ,by means of 2PC/3PC protocol. Q2: "same Timestamp view", how it will affect the App clients? What is means ? Thanks a lot! Best Regards! |
From: Michael P. <mic...@gm...> - 2013-11-29 12:16:10
|
On Fri, Nov 29, 2013 at 6:15 PM, Pavan Deolasee <pav...@gm...> wrote: > I (actually my new compiler) noticed and complained about this which I > believe is most likely a mistake. Please see attached patch. I haven't > looked at the related code in detail, so don't know if this can cause any > problems, known or unknown. Indeed, I just went ahead, committed it and backpatched to 1.1 as well where the issue was introduced. > P.S. I can commit this, though not sure if my commit rights are still > intact. Not sure either... -- Michael |
From: Nikhil S. <ni...@st...> - 2013-11-29 09:46:48
|
Yeah, Mavericks OS X gives out a ton of additional, helpful info related to warnings (along with possible resolutions). Pretty neat. Regards, Nikhils On Fri, Nov 29, 2013 at 2:45 PM, Pavan Deolasee <pav...@gm...>wrote: > Hello, > > I (actually my new compiler) noticed and complained about this which I > believe is most likely a mistake. Please see attached patch. I haven't > looked at the related code in detail, so don't know if this can cause any > problems, known or unknown. > > Thanks, > Pavan > > P.S. I can commit this, though not sure if my commit rights are still > intact. > > -- > Pavan Deolasee > http://www.linkedin.com/in/pavandeolasee > > > ------------------------------------------------------------------------------ > Rapidly troubleshoot problems before they affect your business. Most IT > organizations don't have a clear picture of how application performance > affects their revenue. With AppDynamics, you get 100% visibility into your > Java,.NET, & PHP application. Start your 15-day FREE TRIAL of AppDynamics > Pro! > http://pubads.g.doubleclick.net/gampad/clk?id=84349351&iu=/4140/ostg.clktrk > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > -- StormDB - http://www.stormdb.com The Database Cloud |
From: Koichi S. <koi...@gm...> - 2013-11-27 07:29:10
|
Verry sorry that I didn't include the patch. I'll wait for a day for more input before commit. --- Koichi Suzuki 2013/11/25 Koichi Suzuki <koi...@gm...>: > The patch contains two fixes: > > 1. gtmPxyExtraConfig and gtmPxySpecificExtraConfig configuration > handling: They were not handled. > > 2. kill gtm slave handling: Internal command generation error fixed. > > The are intended to be in release 1.1.1 and 1.2. > > Further input is welcome. > > Regards; > --- > Koichi Suzuki |