Re: [Postgres-xc-general] ERROR: Failed to get pooled connections

[Koichi S. <koi...@gm...>]

Yes, reading from datanode slaves will enhance read scalability.

In terms of reading from datanode slave, I think we still need a
couple of improvements:

1. If you want to connect directly to a slave, current datanode
expects all the connections are from coordinators and they supply GXID
and snapshot from GTM, which psql or current libpq don't do.   If
datanode is in recovery mode and standby_mode is on, then it should
use XID and snapshot from WAL, which is now overridden by GXID and
snapshot from coordinators/GTM.

2. If you want to connect via coordinator, which is not supported yet
and we need coordinator extension.

3. If you want to visit multiple datanodes, you may get different
visibility from datanode to datanode, because synchronous replication
implies time lag from "receiving" WAL records to "replaying" them.
The time lag  may be different from datanode to datanode and the query
result could be incorrect.  I guess "BARRIER" may work to synchronize
the visibility among the datanode but we may need another visibility
control infrastructure for hot standby.

Any more inputs are welcome.

Regards;
----------
Koichi Suzuki


2012/7/31 Mason Sharp <ma...@st...>:
> On Tue, Jul 31, 2012 at 5:19 AM, Vladimir Stavrinov
> <vst...@gm...> wrote:
>> On Tue, Jul 31, 2012 at 05:35:45PM +0900, Michael Paquier wrote:
>>
>>>    The main problem that I see here is that replicating data
>>>    asynchronously breaks MVCC.
>>
>> May I cite myself?
>>
>>         When read request come in, it should go to replicated node if and only if
>>         requested data exists there, otherwise such request should go to distributed
>>         node where those data in question exists in any case.
>>
>>>    So you are never sure that the data will be here or not on your
>>>    background nodes.
>>
>> If we control where the data stored in distributed nodes, why not to control
>> the state of replicated nodes? In both cases we should know what data is where.
>>
>
> If a data node has one or more sync rep standbys, it should be
> theoretically possible to read balance those if that intelligence is
> added to the coordinator. It would not matter if that data is in a
> "distributed" or "replicated" table.
>
> If it were asynchronous, there would be more tracking that would have
> to be done to know if it is safe to load balance.
>
> In some tests we have done at StormDB, the extra overhead for sync rep
> is small, so you might as well use sync rep. With multiple replicas,
> in sync rep mode it will continue working even there is a failure with
> one of the replicas.
>
>
>>>    Ideas are of course always welcome, but if you want to add some new
>>>    features you will need to be more specific.
>>
>> I don't think what we discussing here is simply feature that may be added
>> with an patch. The idea to move storage control on cluster level touches
>> the basics and concept of XC.
>
> Yeah, that is sounding different than what is done here. I am not sure
> I understand what your requirements are and what exactly it is you
> need.  If it is about HA, there are a lot of basics in place to build
> out HA from that will probably meet your needs.
>
>>
>> --
>>
>> ***************************
>> ##  Vladimir Stavrinov
>> ##  vst...@gm...
>> ***************************
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Postgres-xc-general mailing list
>> Pos...@li...
>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
>
>
>
> --
> Mason Sharp
>
> StormDB - http://www.stormdb.com
> The Database Cloud
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Postgres-xc-general mailing list
> Pos...@li...
> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general