Re: [Postgres-xc-developers] GTM/GTM Proxy invalid handles and failed to get snapshot issues

[Koichi S. <koi...@gm...>]

I'd like to commit this patch unless there're no more discussion or input.

Regards;
---
Koichi Suzuki


2013/8/8 Koichi Suzuki <koi...@gm...>

>
>
>
> 2013/8/6 Nikhil Sontakke <ni...@st...>
>
>>
>>
>>
>>> Can GTM/proxy cleanup the pending list when it detects the death of a
>>> connection, just like GTM does its cleanup?
>>>
>>>
>> Well, the pending requests are not stored in a per client list. We could
>> go and look for this specific client in the list though and clean it up.
>> But remember that the client might have given a set of instructions to the
>> proxy and disappeared legitimately. So processing all requests in order
>> before processing the disconnection seems much cleaner.
>>
>
> Yes, it makes sense.    BTW, all the pending and processed list for each
> select() cycle is cleaned up.
> ---
> Koichi Suzuki
>
>
>>
>> Regards,
>> Nikhils
>>
>>
>>>
>>> On Tue, Aug 6, 2013 at 11:03 AM, Nikhil Sontakke <ni...@st...>wrote:
>>>
>>>>
>>>> A naive question,
>>>>> Can we also cleanup the pending list?
>>>>>
>>>>>
>>>> Umm, the processed commands are put onto the thr_processed_commands
>>>> list and freed up eventually. Is that what you were asking?
>>>>
>>>> Regards,
>>>> Nikhils
>>>>
>>>>>
>>>>>  On Tue, Aug 6, 2013 at 9:56 AM, Nikhil Sontakke <ni...@st...>wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We have had a long standing issue with a GTM Proxy in the mix. It has
>>>>>> been reported here:
>>>>>>
>>>>>>
>>>>>> http://sourceforge.net/mailarchive/forum.php?thread_name=CAB7nPqQ51fabopxCOHPrRM%3D2WTJMikfZUPHNEW942DDHmED7bQ%40mail.gmail.com&forum_name=postgres-xc-developers
>>>>>>
>>>>>> I think I found the root cause:
>>>>>>
>>>>>> Here's how things work:
>>>>>>
>>>>>> The client connects to proxy. The proxy internally connects to the
>>>>>> GTM.
>>>>>>
>>>>>> Some commands from clients are sent immediately to the GTM by the
>>>>>> proxy.
>>>>>>
>>>>>> Some commands are put in the pending list on the proxy for batch
>>>>>> processing later.
>>>>>>
>>>>>> Now, if the client goes away before processing pending commands, the
>>>>>> proxy is immediately telling the GTM so. That initiates a cleanup on the
>>>>>> GTM side and removes the XIDs associated with this client. After this
>>>>>> cleanup when the GTM receives a command like "MSG_SNAPSHOT_GET" or
>>>>>> "MSG_TXN_COMMIT_MULTI" etc. from the pending list from the proxy, then it's
>>>>>> going to get confused.
>>>>>>
>>>>>> A way to handle this is to put "disconnected" messages also onto the
>>>>>> pending list. We then process these "disconnected" messages LAST from the
>>>>>> pending list.
>>>>>>
>>>>>> I have tried this fix and run a number of tests (using pgbench) and
>>>>>> things seem to be ok. I do not see the invalid handle and snapshot errors
>>>>>> on the GTM side with this patch in place.
>>>>>>
>>>>>> PFA, patches for 1.0, 1.1 and head attached with this email.
>>>>>>
>>>>>> Regards,
>>>>>> Nikhils
>>>>>> --
>>>>>> StormDB - http://www.stormdb.com
>>>>>> The Database Cloud
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Get your SQL database under version control now!
>>>>>> Version control is standard for application code, but databases havent
>>>>>> caught up. So what steps can you take to put your SQL databases under
>>>>>> version control? Why should you start doing it? Read more to find out.
>>>>>>
>>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>>>>>> _______________________________________________
>>>>>> Postgres-xc-developers mailing list
>>>>>> Pos...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Wishes,
>>>>> Ashutosh Bapat
>>>>> EntepriseDB Corporation
>>>>> The Postgres Database Company
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> StormDB - http://www.stormdb.com
>>>> The Database Cloud
>>>>
>>>
>>>
>>>
>>> --
>>> Best Wishes,
>>> Ashutosh Bapat
>>> EntepriseDB Corporation
>>> The Postgres Database Company
>>>
>>
>>
>>
>> --
>> StormDB - http://www.stormdb.com
>> The Database Cloud
>>
>>
>> ------------------------------------------------------------------------------
>> Get your SQL database under version control now!
>> Version control is standard for application code, but databases havent
>> caught up. So what steps can you take to put your SQL databases under
>> version control? Why should you start doing it? Read more to find out.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>