Thanks. I wrote a preliminary patch and found it seems to work fine.
I'm now inserting sleep between retries if we have to retry many times.
Will see the patch soon.
Regards;
---
Koichi Suzuki
2013/10/23 Nikhil Sontakke <ni...@st...>
> +1
>
> I had diagnosed this same issue quite a while ago. We should wait for the
> buffer to drain out before adding more into it. The buffer size was going
> beyond 1GB in size pretty quickly!
>
> Regards,
> Nikhils
>
>
> On Wed, Oct 23, 2013 at 10:35 AM, Koichi Suzuki <koi...@gm...>wrote:
>
>> I've found that in copy command, we have a risk to overflow the buffer in
>> coordinator if datanode is very slow (for various reasons).
>>
>> When we issue copy command against tens of gigabytes of data, coordinator
>> send all of then without synch. When some of the data fails to send, it
>> adds next data and tries to send all of them. When a datanode is very
>> slow (for any reason), the data will continue to be cashed at the
>> coordinator and finally overflows.
>>
>> The patch avoids this issue. When amount of unsent data exceeds the
>> criteria, coordinator will keep retry until this amount is below the second
>> criteria.
>>
>> This can be applied to all the releases, as well as the master.
>>
>> Regards;
>> ---
>> Koichi Suzuki
>>
>>
>> ------------------------------------------------------------------------------
>> October Webinars: Code for Performance
>> Free Intel webinars can help you accelerate application performance.
>> Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most
>> from
>> the latest Intel processors and coprocessors. See abstracts and register >
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Postgres-xc-developers mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers
>>
>>
>
>
> --
> StormDB - http://www.stormdb.com
> The Database Cloud
>
|
