Menu
▾
▴
Re: [Postgres-xc-developers] Query Planning bug ?
|
From: Tomonari K. <kat...@po...> - 2013-07-05 03:53:53
|
Hi Ashutosh, Abbas,
Sorry for slow response..
But still I can not run regression test.
>>Ashutosh
> see if copying query->rtable solves the issue. The changed code would
look
> like
>
> 3112 crte_context->crte_rtable =
> list_concat(crte_context->crte_rtable,
> 3113
> list_copy(query->rtable));
>
I've changed source as you said, and it seems resolving my problem(*).
(*) below query falt in endless loop.
------------------------------------------------------------------
EXECUTE DIRECT ON (data1) $$
SELECT
count(*)
FROM
(SELECT * FROM pg_locks l LEFT JOIN
(SELECT * FROM pg_stat_activity) s ON l.database = s.datid) a
$$
------------------------------------------------------------------
But I couldn't run regression test with this source too.
It stoped same test-case(xmlmap.sql).
>>Abbas
> Since you are getting a crash even without your patch there must be some
> thing wrong in the deployment.
> Can you try doing
> ./configure --enable-debug --enable-cassert CFLAGS='-O0'
> and then see whether its an assertion failure or not?
> Also can you see which SQL statement is causing the crash in xmlmap.sql?
>
I've tried the configure you said.
I didn't get any assertion failure, but regression test had
stoped in "test collate".
When Postgres-XC crashed with xmlmap.sql,
I found The SQL statement which is causing the crash.
----
DECLARE xc CURSOR WITH HOLD FOR SELECT * FROM testxmlschema.test1 ORDER
BY 1, 2;
----
I think these are another problems.
Because I get same result with or without patched Postgres-XC.
so I want to discuss on another mails.
regards,
------------
NTT Software Corporation
Tomonari Katsumata
(2013/07/03 12:13), Abbas Butt wrote:
> Since you are getting a crash even without your patch there must be some
> thing wrong in the deployment.
> Can you try doing
> ./configure --enable-debug --enable-cassert CFLAGS='-O0'
> and then see whether its an assertion failure or not?
> Also can you see which SQL statement is causing the crash in xmlmap.sql?
>
> On Mon, Jul 1, 2013 at 4:51 PM, Tomonari Katsumata <
> t.k...@gm...> wrote:
>
>> Hi,
>>
>> I've tried regression test, but I could not
>> get right result.
>> Both patched and un-patched Postgres-XC get
>> same result, so I think my process is bad.
>>
>> I run a gtm, a gtm_proxy, a coordinator, a datanode
>> in one server(CentOS6.4 x86_64 on VMWare),
>> and hit "make installcheck".
>> The configure option is [--enable-debug CFLAGS=""].
>>
>> What is right way to run the regression test?
>>
>> output is below.
>> ---------------------------------------------------------
>> test tablespace ... ok
>> test boolean ... ok
>> test char ... ok
>> test name ... ok
>> test varchar ... ok
>> test text ... ok
>> test int2 ... ok
>> test int4 ... ok
>> test int8 ... ok
>> test oid ... ok
>> test float4 ... ok
>> test float8 ... ok
>> test bit ... ok
>> test numeric ... ok
>> test txid ... ok
>> test uuid ... ok
>> test enum ... FAILED
>> test money ... ok
>> test rangetypes ... ok
>> test strings ... ok
>> test numerology ... ok
>> test point ... ok
>> test lseg ... ok
>> test box ... ok
>> test path ... ok
>> test polygon ... ok
>> test circle ... ok
>> test date ... ok
>> test time ... ok
>> test timetz ... ok
>> test timestamp ... ok
>> test timestamptz ... ok
>> test interval ... ok
>> test abstime ... ok
>> test reltime ... ok
>> test tinterval ... ok
>> test inet ... ok
>> test macaddr ... ok
>> test tstypes ... ok
>> test comments ... ok
>> test geometry ... ok
>> test horology ... ok
>> test regex ... ok
>> test oidjoins ... ok
>> test type_sanity ... ok
>> test opr_sanity ... ok
>> test insert ... ok
>> test create_function_1 ... ok
>> test create_type ... ok
>> test create_table ... ok
>> test create_function_2 ... ok
>> test create_function_3 ... ok
>> test copy ... ok
>> test copyselect ... ok
>> test create_misc ... ok
>> test create_operator ... ok
>> test create_index ... FAILED
>> test create_view ... ok
>> test create_aggregate ... ok
>> test create_cast ... ok
>> test constraints ... FAILED
>> test triggers ... ok
>> test inherit ... FAILED
>> test create_table_like ... ok
>> test typed_table ... ok
>> test vacuum ... ok
>> test drop_if_exists ... ok
>> test sanity_check ... ok
>> test errors ... ok
>> test select ... ok
>> test select_into ... ok
>> test select_distinct ... ok
>> test select_distinct_on ... ok
>> test select_implicit ... ok
>> test select_having ... ok
>> test subselect ... ok
>> test union ... ok
>> test case ... ok
>> test join ... FAILED
>> test aggregates ... FAILED
>> test transactions ... ok
>> test random ... ok
>> test portals ... ok
>> test arrays ... FAILED
>> test btree_index ... ok
>> test hash_index ... ok
>> test update ... ok
>> test delete ... ok
>> test namespace ... ok
>> test prepared_xacts ... ok
>> test privileges ... FAILED
>> test security_label ... ok
>> test collate ... FAILED
>> test misc ... ok
>> test rules ... ok
>> test select_views ... ok
>> test portals_p2 ... ok
>> test foreign_key ... ok
>> test cluster ... ok
>> test dependency ... ok
>> test guc ... ok
>> test bitmapops ... ok
>> test combocid ... ok
>> test tsearch ... ok
>> test tsdicts ... ok
>> test foreign_data ... ok
>> test window ... ok
>> test xmlmap ... FAILED (test process exited with exit
>> code 2)
>> test functional_deps ... FAILED (test process exited with exit
>> code 2)
>> test advisory_lock ... FAILED (test process exited with exit
>> code 2)
>> test json ... FAILED (test process exited with exit
>> code 2)
>> test plancache ... FAILED (test process exited with exit
>> code 2)
>> test limit ... FAILED (test process exited with exit
>> code 2)
>> test plpgsql ... FAILED (test process exited with exit
>> code 2)
>> test copy2 ... FAILED (test process exited with exit
>> code 2)
>> test temp ... FAILED (test process exited with exit
>> code 2)
>> test domain ... FAILED (test process exited with exit
>> code 2)
>> test rangefuncs ... FAILED (test process exited with exit
>> code 2)
>> test prepare ... FAILED (test process exited with exit
>> code 2)
>> test without_oid ... FAILED (test process exited with exit
>> code 2)
>> test conversion ... FAILED (test process exited with exit
>> code 2)
>> test truncate ... FAILED (test process exited with exit
>> code 2)
>> test alter_table ... FAILED (test process exited with exit
>> code 2)
>> test sequence ... FAILED (test process exited with exit
>> code 2)
>> test polymorphism ... FAILED (test process exited with exit
>> code 2)
>> test rowtypes ... FAILED (test process exited with exit
>> code 2)
>> test returning ... FAILED (test process exited with exit
>> code 2)
>> test largeobject ... FAILED (test process exited with exit
>> code 2)
>> test with ... FAILED (test process exited with exit
>> code 2)
>> test xml ... FAILED (test process exited with exit
>> code 2)
>> test stats ... FAILED (test process exited with exit
>> code 2)
>> test xc_create_function ... FAILED (test process exited with exit
>> code 2)
>> test xc_groupby ... FAILED (test process exited with exit
>> code 2)
>> test xc_distkey ... FAILED (test process exited with exit
>> code 2)
>> test xc_having ... FAILED (test process exited with exit
>> code 2)
>> test xc_temp ... FAILED (test process exited with exit
>> code 2)
>> test xc_remote ... FAILED (test process exited with exit
>> code 2)
>> test xc_node ... FAILED (test process exited with exit
>> code 2)
>> test xc_FQS ... FAILED (test process exited with exit
>> code 2)
>> test xc_FQS_join ... FAILED (test process exited with exit
>> code 2)
>> test xc_misc ... FAILED (test process exited with exit
>> code 2)
>> test xc_triggers ... FAILED (test process exited with exit
>> code 2)
>> test xc_trigship ... FAILED (test process exited with exit
>> code 2)
>> test xc_constraints ... FAILED (test process exited with exit
>> code 2)
>> test xc_copy ... FAILED (test process exited with exit
>> code 2)
>> test xc_alter_table ... FAILED (test process exited with exit
>> code 2)
>> test xc_sequence ... FAILED (test process exited with exit
>> code 2)
>> test xc_prepared_xacts ... FAILED (test process exited with exit
>> code 2)
>> test xc_notrans_block ... FAILED (test process exited with exit
>> code 2)
>> test xc_limit ... FAILED (test process exited with exit
>> code 2)
>> test xc_sort ... FAILED (test process exited with exit
>> code 2)
>> test xc_returning ... FAILED (test process exited with exit
>> code 2)
>> test xc_params ... FAILED (test process exited with exit
>> code 2)
>> =========================
>> 55 of 153 tests failed.
>> =========================
>>
>>
>> 2013/7/1 Tomonari Katsumata <kat...@po...>
>>
>>> Hi Ashutosh,
>>>
>>> OK, I'll try regression test.
>>> Please wait for the result.
>>>
>>> regards,
>>> ------------
>>> NTT Software Corporation
>>> Tomonari Katsumata
>>>
>>> (2013/07/01 17:06), Ashutosh Bapat wrote:
>>>> Hi Tomonori,
>>>>
>>>>
>>>>
>>>> On Mon, Jul 1, 2013 at 1:33 PM, Tomonari Katsumata <
>>>> kat...@po...> wrote:
>>>>
>>>>> Hi Ashutosh and all,
>>>>>
>>>>> Sorry for late response.
>>>>> I made a patch for resolving the problem I mentioned before.
>>>>>
>>>>> I thought the reason of this problem is parsing query twice.
>>>>> because of this, the rtable is made from same Lists and become
>>>>> cycliced List.
>>>>> I fixed this problem with making former List empty.
>>>>>
>>>>> I'm not sure this fix leads any anothre problems but
>>>>> the problem query I mentioned before works fine.
>>>>>
>>>>> This patch is against for
>>> "**a074cac9b6b507e6d4b58c5004673f**6cc65fcde1".
>>>>>
>>>> You can check the robustness of patch by running regression. Please let
>>> me
>>>> know what you see.
>>>>
>>>>
>>>>> regards,
>>>>> ------------------
>>>>> NTT Software Corporation
>>>>> Tomonari Katsumata
>>>>>
>>>>>
>>>>> (2013/06/17 18:53), Ashutosh Bapat wrote:
>>>>>
>>>>>> Hi Tomonari,
>>>>>> In which function have you taken this debug info? What is list1 and
>>> list2?
>>>>>>
>>>>>> On Mon, Jun 17, 2013 at 10:13 AM, Tomonari Katsumata <
>>>>>> kat...@po....**jp <kat...@po...
>>>>>> wrote:
>>>>>>
>>>>>> Hi Ashtosh,
>>>>>>> Sorry for slow response.
>>>>>>>
>>>>>>> I've watched the each lists at list_concat function.
>>>>>>>
>>>>>>> This function is called several times, but the lists before
>>>>>>> last infinitely roop are like below.
>>>>>>>
>>>>>>> [list1]
>>>>>>> (gdb) p *list1->head
>>>>>>> $18 = {data = {ptr_value = 0x17030e8, int_value = 24129768,
>>>>>>> oid_value = 24129768}, next = 0x170d418}
>>>>>>> (gdb) p *list1->head->next
>>>>>>> $19 = {data = {ptr_value = 0x17033d0, int_value = 24130512,
>>>>>>> oid_value = 24130512}, next = 0x170fd40}
>>>>>>> (gdb) p *list1->head->next->next
>>>>>>> $20 = {data = {ptr_value = 0x170ae58, int_value = 24161880,
>>>>>>> oid_value = 24161880}, next = 0x171e6c8}
>>>>>>> (gdb) p *list1->head->next->next->next
>>>>>>> $21 = {data = {ptr_value = 0x1702ca8, int_value = 24128680,
>>>>>>> oid_value = 24128680}, next = 0x171ed28}
>>>>>>> (gdb) p *list1->head->next->next->**next->next
>>>>>>> $22 = {data = {ptr_value = 0x170af68, int_value = 24162152,
>>>>>>> oid_value = 24162152}, next = 0x171f3a0}
>>>>>>> (gdb) p *list1->head->next->next->**next->next->next
>>>>>>> $23 = {data = {ptr_value = 0x170b0a8, int_value = 24162472,
>>>>>>> oid_value = 24162472}, next = 0x170b7c0}
>>>>>>> (gdb) p *list1->head->next->next->**next->next->next->next
>>>>>>> $24 = {data = {ptr_value = 0x17035f0, int_value = 24131056,
>>>>>>> oid_value = 24131056}, next = 0x1720998}
>>>>>>> ---- from ---
>>>>>>> (gdb) p *list1->head->next->next->**next->next->next->next->next
>>>>>>> $25 = {data = {ptr_value = 0x17209b8, int_value = 24250808,
>>>>>>> oid_value = 24250808}, next = 0x1721190}
>>>>>>> (gdb) p
>>> *list1->head->next->next->**next->next->next->next->next->**next
>>>>>>> $26 = {data = {ptr_value = 0x17211b0, int_value = 24252848,
>>>>>>> oid_value = 24252848}, next = 0x1721988}
>>>>>>> (gdb) p *list1->head->next->next->**next->next->next->next->next->**
>>>>>>> next->next
>>>>>>> $27 = {data = {ptr_value = 0x17219a8, int_value = 24254888,
>>>>>>> oid_value = 24254888}, next = 0x1722018}
>>>>>>> (gdb) p
>>>>>>> *list1->head->next->next->**next->next->next->next->next->**
>>>>>>> next->next->next
>>>>>>> $28 = {data = {ptr_value = 0x1722038, int_value = 24256568,
>>>>>>> oid_value = 24256568}, next = 0x1722820}
>>>>>>> (gdb) p
>>>>>>>
>>>>>>> *list1->head->next->next->**next->next->next->next->next->**
>>>>>>> next->next->next->next
>>>>>>> $29 = {data = {ptr_value = 0x1722840, int_value = 24258624,
>>>>>>> oid_value = 24258624}, next = 0x0}
>>>>>>> ---- to ----
>>>>>>>
>>>>>>> [list2]
>>>>>>> (gdb) p *list2->head
>>>>>>> $31 = {data = {ptr_value = 0x17209b8, int_value = 24250808,
>>>>>>> oid_value = 24250808}, next = 0x1721190}
>>>>>>> (gdb) p *list2->head->next
>>>>>>> $32 = {data = {ptr_value = 0x17211b0, int_value = 24252848,
>>>>>>> oid_value = 24252848}, next = 0x1721988}
>>>>>>> (gdb) p *list2->head->next->next
>>>>>>> $33 = {data = {ptr_value = 0x17219a8, int_value = 24254888,
>>>>>>> oid_value = 24254888}, next = 0x1722018}
>>>>>>> (gdb) p *list2->head->next->next->next
>>>>>>> $34 = {data = {ptr_value = 0x1722038, int_value = 24256568,
>>>>>>> oid_value = 24256568}, next = 0x1722820}
>>>>>>> (gdb) p *list2->head->next->next->**next->next
>>>>>>> $35 = {data = {ptr_value = 0x1722840, int_value = 24258624,
>>>>>>> oid_value = 24258624}, next = 0x0}
>>>>>>> ----
>>>>>>>
>>>>>>> list1's last five elements are same with list2's all elements.
>>>>>>> (in above example, between "from" and "to" in list1 equal all of
>>> list2)
>>>>>>> This is cause of infinitely roop, but I can not
>>>>>>> watch deeper.
>>>>>>> Because some values from gdb are optimized and un-displayed.
>>>>>>> I tried compile with CFLAGS=O0, but I can't.
>>>>>>>
>>>>>>> What can I do more ?
>>>>>>>
>>>>>>> regards,
>>>>>>> ------------------
>>>>>>> NTT Software Corporation
>>>>>>> Tomonari Katsumata
>>>>>>>
>>>>>>> (2013/06/12 21:04), Ashutosh Bapat wrote:
>>>>>>> > Hi Tomonari,
>>>>>>> > Can you please check the list's sanity before calling
>>>>>>> pgxc_collect_RTE()
>>>>>>> > and at every point in the minions of this function. My primary
>>>>>>> suspect
>>>>>>> is
>>>>>>> > the line pgxcplan.c:3094. We should copy the list before
>>>>>>> concatenating it.
>>>>>>> >
>>>>>>> >
>>>>>>> > On Wed, Jun 12, 2013 at 2:26 PM, Tomonari Katsumata <
>>>>>>> > kat...@po....**jp<
>>> kat...@po...>>
>>>>>>> wrote:
>>>>>>> >
>>>>>>> >> Hi Ashutosh,
>>>>>>> >>
>>>>>>> >> Thank you for the response.
>>>>>>> >>
>>>>>>> >> (2013/06/12 14:43), Ashutosh Bapat wrote:
>>>>>>> >> >> Hi,
>>>>>>> >> >> >
>>>>>>> >> >> > I've investigated this problem(BUG:3614369).
>>>>>>> >> >> >
>>>>>>> >> >> > I caught the cause of it, but I can not
>>>>>>> >> >> > find where to fix.
>>>>>>> >> >> >
>>>>>>> >> >> > The problem occurs when "pgxc_collect_RTE_walker" is
>>> called
>>>>>>> >> infinitely.
>>>>>>> >> >> > It seems that rtable(List of RangeTable) become cyclic
>>> List.
>>>>>>> >> >> > I'm not sure where the List is made.
>>>>>>> >> >> >
>>>>>>> >> >> >
>>>>>>> >> > I guess, we are talking about EXECUTE DIRECT statement that
>>> you
>>>>>>> have
>>>>>>> >> > mentioned earlier.
>>>>>>> >>
>>>>>>> >> Yes, that's right.
>>>>>>> >> I'm talking about EXECUTE DIRECT statement like below.
>>>>>>> >> ---
>>>>>>> >> EXECUTE DIRECT ON (data1) $$
>>>>>>> >> SELECT
>>>>>>> >> count(*)
>>>>>>> >> FROM
>>>>>>> >> (SELECT * FROM pg_locks l LEFT JOIN
>>>>>>> >> (SELECT * FROM pg_stat_activity) s ON l.database = s.datid) a
>>>>>>> >> $$
>>>>>>> >> ---
>>>>>>> >>
>>>>>>> >> > The function pgxc_collect_RTE_walker() is a recursive
>>>>>>> >> > function. The condition to end the recursion is if the given
>>>>>>> node is
>>>>>>> >> NULL.
>>>>>>> >> > We have to look at if that condition is met and if not why.
>>>>>>> >> >
>>>>>>> >> I investigated it deeper, and I noticed that
>>>>>>> >> the infinitely loop happens at the function
>>> "range_table_walker()".
>>>>>>> >>
>>>>>>> >> Please see below trace.
>>>>>>> >> ===========================
>>>>>>> >> Breakpoint 1, range_table_walker (rtable=0x15e7968,
>>> walker=0x612c70
>>>>>>> >> <pgxc_collect_RTE_walker>, context=0x7fffd2de31c0,
>>>>>>> >> flags=0) at nodeFuncs.c:1908
>>>>>>> >> 1908 in nodeFuncs.c
>>>>>>> >>
>>>>>>> >> (gdb) p *rtable
>>>>>>> >> $10 = {type = T_List, length = 5, head = 0x15e7998, tail =
>>>>>>> 0x15e9820}
>>>>>>> >> (gdb) p *rtable->head
>>>>>>> >> $11 = {data = {ptr_value = 0x15e79b8, int_value = 22968760,
>>>>>>> oid_value =
>>>>>>> >> 22968760}, next = 0x15e8190}
>>>>>>> >> (gdb) p *rtable->head->next
>>>>>>> >> $12 = {data = {ptr_value = 0x15e81b0, int_value = 22970800,
>>>>>>> oid_value =
>>>>>>> >> 22970800}, next = 0x15e8988}
>>>>>>> >> (gdb) p *rtable->head->next->next
>>>>>>> >> $13 = {data = {ptr_value = 0x15e89a8, int_value = 22972840,
>>>>>>> oid_value =
>>>>>>> >> 22972840}, next = 0x15e9018}
>>>>>>> >> (gdb) p *rtable->head->next->next->**next
>>>>>>> >> $14 = {data = {ptr_value = 0x15e9038, int_value = 22974520,
>>>>>>> oid_value =
>>>>>>> >> 22974520}, next = 0x15e9820}
>>>>>>> >> (gdb) p *rtable->head->next->next->**next->next
>>>>>>> >> $15 = {data = {ptr_value = 0x15e9840, int_value = 22976576,
>>>>>>> oid_value =
>>>>>>> >> 22976576}, next = 0x15e7998}
>>>>>>> >> ===========================
>>>>>>> >>
>>>>>>> >> The line which starts with "$15" has 0x15e7998 as its next
>>> data.
>>>>>>> >> But it is the head pointer(see the line which starts with $10).
>>>>>>> >>
>>>>>>> >> And in range_table_walker(), the function is called
>>> recursively.
>>>>>>> >> --------
>>>>>>> >> ...
>>>>>>> >> if (!(flags & QTW_IGNORE_RANGE_TABLE))
>>>>>>> >> {
>>>>>>> >> if (range_table_walker(query->**rtable,
>>> walker,
>>>>>>> context,
>>>>>>> >> flags))
>>>>>>> >> return true;
>>>>>>> >> }
>>>>>>> >> ...
>>>>>>> >> --------
>>>>>>> >>
>>>>>>> >> We should make rtable right or deal with "flags" properly.
>>>>>>> >> But I can't find where to do it...
>>>>>>> >>
>>>>>>> >> What do you think ?
>>>>>>> >>
>>>>>>> >> regards,
>>>>>>> >> ---------
>>>>>>> >> NTT Software Corporation
>>>>>>> >> Tomonari Katsumata
>>>>>>> >>
>>>>>>> >>
>>>>>>> >>
>>>>>>> >>
>>>>>>> >>
>>>>>>>
>>>>>>> ------------------------------**------------------------------**
>>>>>>> ------------------
>>>>>>> >> This SF.net email is sponsored by Windows:
>>>>>>> >>
>>>>>>> >> Build for Windows Store.
>>>>>>> >>
>>>>>>> >> http://p.sf.net/sfu/windows-**dev2dev<
>>> http://p.sf.net/sfu/windows-dev2dev>
>>>>>>> >> ______________________________**_________________
>>>>>>> >> Postgres-xc-developers mailing list
>>>>>>> >> Postgres-xc-developers@lists.**sourceforge.net<
>>> Pos...@li...>
>>>>>>> >> https://lists.sourceforge.net/**lists/listinfo/postgres-xc-**
>>>>>>> developers<
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers>
>>>>>>> >>
>>>>>>> >
>>>>>>> >
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------**------------------------------**
>>>>>>> ------------------
>>>>>>> This SF.net email is sponsored by Windows:
>>>>>>>
>>>>>>> Build for Windows Store.
>>>>>>>
>>>>>>> http://p.sf.net/sfu/windows-**dev2dev<
>>> http://p.sf.net/sfu/windows-dev2dev>
>>>>>>> ______________________________**_________________
>>>>>>> Postgres-xc-developers mailing list
>>>>>>> Postgres-xc-developers@lists.**sourceforge.net<
>>> Pos...@li...>
>>> https://lists.sourceforge.net/**lists/listinfo/postgres-xc-**developers<
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers>
>>>>>>>
>>>>>
>>> ------------------------------------------------------------------------------
>>>>> This SF.net email is sponsored by Windows:
>>>>>
>>>>> Build for Windows Store.
>>>>>
>>>>> http://p.sf.net/sfu/windows-dev2dev
>>>>> _______________________________________________
>>>>> Postgres-xc-developers mailing list
>>>>> Pos...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>
>>>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> This SF.net email is sponsored by Windows:
>>>
>>> Build for Windows Store.
>>>
>>> http://p.sf.net/sfu/windows-dev2dev
>>> _______________________________________________
>>> Postgres-xc-developers mailing list
>>> Pos...@li...
>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by Windows:
>>
>> Build for Windows Store.
>>
>> http://p.sf.net/sfu/windows-dev2dev
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
>
>
> _______________________________________________
> Postgres-xc-developers mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
|
