Menu
▾
▴
Re: [Postgres-xc-developers] Patch to lock cluster
|
From: Abbas B. <abb...@en...> - 2013-03-04 09:14:20
|
On Mon, Mar 4, 2013 at 2:00 PM, Amit Khandekar < ami...@en...> wrote: > On 1 March 2013 18:45, Abbas Butt <abb...@en...> wrote: > > > > > > On Fri, Mar 1, 2013 at 5:48 PM, Amit Khandekar > > <ami...@en...> wrote: > >> > >> On 19 February 2013 12:37, Abbas Butt <abb...@en...> > wrote: > >> > > >> > Hi, > >> > Attached please find a patch that locks the cluster so that dump can > be > >> > taken to be restored on the new node to be added. > >> > > >> > To lock the cluster the patch adds a new GUC parameter called > >> > xc_lock_for_backup, however its status is maintained by the pooler. > The > >> > reason is that the default behavior of XC is to release connections as > >> > soon > >> > as a command is done and it uses PersistentConnections GUC to control > >> > the > >> > behavior. We in this case however need a status that is independent of > >> > the > >> > setting of PersistentConnections. > >> > > >> > Assume we have two coordinator cluster, the patch provides this > >> > behavior: > >> > > >> > Case 1: set and show > >> > ==================== > >> > psql test -p 5432 > >> > set xc_lock_for_backup=yes; > >> > show xc_lock_for_backup; > >> > xc_lock_for_backup > >> > -------------------- > >> > yes > >> > (1 row) > >> > > >> > Case 2: set from one client show from other > >> > ================================== > >> > psql test -p 5432 > >> > set xc_lock_for_backup=yes; > >> > (From another tab) > >> > psql test -p 5432 > >> > show xc_lock_for_backup; > >> > xc_lock_for_backup > >> > -------------------- > >> > yes > >> > (1 row) > >> > > >> > Case 3: set from one, quit it, run again and show > >> > ====================================== > >> > psql test -p 5432 > >> > set xc_lock_for_backup=yes; > >> > \q > >> > psql test -p 5432 > >> > show xc_lock_for_backup; > >> > xc_lock_for_backup > >> > -------------------- > >> > yes > >> > (1 row) > >> > > >> > Case 4: set on one coordinator, show from other > >> > ===================================== > >> > psql test -p 5432 > >> > set xc_lock_for_backup=yes; > >> > (From another tab) > >> > psql test -p 5433 > >> > show xc_lock_for_backup; > >> > xc_lock_for_backup > >> > -------------------- > >> > yes > >> > (1 row) > >> > > >> > pg_dump and pg_dumpall seem to work fine after locking the cluster for > >> > backup but I would test these utilities in detail next. > >> > > >> > Also I have yet to look in detail that standard_ProcessUtility is the > >> > only > >> > place that updates the portion of catalog that is dumped. There may be > >> > some > >> > other places too that need to be blocked for catalog updates. > >> > > >> > The patch adds no extra warnings and regression shows no extra > failure. > >> > > >> > Comments are welcome. > >> > >> Abbas wrote on another thread: > >> > >> > Amit wrote on another thread: > >> >> I haven't given a thought on the earlier patch you sent for cluster > >> >> lock > >> >> implementation; may be we can discuss this on that thread, but just a > >> >> quick > >> >> question: > >> >> > >> >> Does the cluster-lock command wait for the ongoing DDL commands to > >> >> finish > >> >> ? If not, we have problems. The subsequent pg_dump would not contain > >> >> objects > >> >> created by these particular DDLs. > >> > > >> > > >> > Suppose you have a two coordinator cluster. Assume one client > connected > >> > to > >> > each. Suppose one client issues a lock cluster command and the other > >> > issues > >> > a DDL. Is this what you mean by an ongoing DDL? If true then answer to > >> > your > >> > question is Yes. > >> > > >> > Suppose you have a prepared transaction that has a DDL in it, again if > >> > this > >> > can be considered an on going DDL, then again answer to your question > is > >> > Yes. > >> > > >> > Suppose you have a two coordinator cluster. Assume one client > connected > >> > to > >> > each. One client starts a transaction and issues a DDL, the second > >> > client > >> > issues a lock cluster command, the first commits the transaction. If > >> > this is > >> > an ongoing DDL, then the answer to your question is No. > >> > >> Yes this last scenario is what I meant: A DDL has been executed on > nodes, > >> but > >> not committed, when the cluster lock command is run and then pg_dump > >> immediately > >> starts its transaction before the DDL is committed. Here pg_dump does > >> not see the new objects that would be created. > > Come to think of it, there would always be a small interval where the > concurrency issue would remain. Can you please give an example to clarify. > If we were to totally get rid of this > concurrency issue, we need to have some kind of lock. For e.g. the > object access hook function will have shared acces lock on this object > (may be on pg_depend because it is always used for objcet > creation/drop ??) and the lock-cluster command will try to get > exclusive lock on the same. This of course should be done after we are > sure object access hook is called on all types of objects. > > > >> > >> I myself am not sure how would we prevent this from happening. There > >> are two callback hooks that might be worth considering though: > >> 1. Transaction End callback (CallXactCallbacks) > >> 2. Object creation/drop hook (InvokeObjectAccessHook) > >> > >> Suppose we create an object creation/drop hook function that would : > >> 1. store the current transaction id in a global objects_created list > >> if the cluster is not locked, > >> 2. or else if the cluster is locked, this hook would ereport() saying > >> "cannot create catalog objects in this mode". > >> > >> And then during transaction commit , a new transaction callback hook > will: > >> 1. Check the above objects_created list to see if the current > >> transaction has any objects created/dropped. > >> 2. If found and if the cluster-lock is on, it will again ereport() > >> saying "cannot create catalog objects in this mode" > >> > >> Thinking more on the object creation hook, we can even consider this > >> as a substitute for checking the cluster-lock status in > >> standardProcessUtility(). But I am not sure whether this hook does get > >> called on each of the catalog objects. At least the code comments say > >> it does. > > > > > > These are very good ideas, Thanks, I will work on those lines and will > > report back. > > > >> > >> > >> > >> > >> > But its a matter of > >> > deciding which camp are we going to put COMMIT in, the allow camp, or > >> > the > >> > deny camp. I decided to put it in allow camp, because I have not yet > >> > written > >> > any code to detect whether a transaction being committed has a DDL in > it > >> > or > >> > not, and stopping all transactions from committing looks too > restrictive > >> > to > >> > me. > >> > >> > >> > > >> > Do you have some other meaning of an ongoing DDL? > >> > >> > >> > >> > > >> > -- > >> > Abbas > >> > Architect > >> > EnterpriseDB Corporation > >> > The Enterprise PostgreSQL Company > >> > > >> > Phone: 92-334-5100153 > >> > > >> > Website: www.enterprisedb.com > >> > EnterpriseDB Blog: http://blogs.enterprisedb.com/ > >> > Follow us on Twitter: http://www.twitter.com/enterprisedb > >> > > >> > This e-mail message (and any attachment) is intended for the use of > >> > the individual or entity to whom it is addressed. This message > >> > contains information from EnterpriseDB Corporation that may be > >> > privileged, confidential, or exempt from disclosure under applicable > >> > law. If you are not the intended recipient or authorized to receive > >> > this for the intended recipient, any use, dissemination, distribution, > >> > retention, archiving, or copying of this communication is strictly > >> > prohibited. If you have received this e-mail in error, please notify > >> > the sender immediately by reply e-mail and delete this message. > >> > > >> > > >> > > ------------------------------------------------------------------------------ > >> > Everyone hates slow websites. So do we. > >> > Make your web apps faster with AppDynamics > >> > Download AppDynamics Lite for free today: > >> > http://p.sf.net/sfu/appdyn_d2d_feb > >> > _______________________________________________ > >> > Postgres-xc-developers mailing list > >> > Pos...@li... > >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > >> > > > > > > > > > > > -- > > -- > > Abbas > > Architect > > EnterpriseDB Corporation > > The Enterprise PostgreSQL Company > > > > Phone: 92-334-5100153 > > > > Website: www.enterprisedb.com > > EnterpriseDB Blog: http://blogs.enterprisedb.com/ > > Follow us on Twitter: http://www.twitter.com/enterprisedb > > > > This e-mail message (and any attachment) is intended for the use of > > the individual or entity to whom it is addressed. This message > > contains information from EnterpriseDB Corporation that may be > > privileged, confidential, or exempt from disclosure under applicable > > law. If you are not the intended recipient or authorized to receive > > this for the intended recipient, any use, dissemination, distribution, > > retention, archiving, or copying of this communication is strictly > > prohibited. If you have received this e-mail in error, please notify > > the sender immediately by reply e-mail and delete this message. > -- -- Abbas Architect EnterpriseDB Corporation The Enterprise PostgreSQL Company Phone: 92-334-5100153 Website: www.enterprisedb.com EnterpriseDB Blog: http://blogs.enterprisedb.com/ Follow us on Twitter: http://www.twitter.com/enterprisedb This e-mail message (and any attachment) is intended for the use of the individual or entity to whom it is addressed. This message contains information from EnterpriseDB Corporation that may be privileged, confidential, or exempt from disclosure under applicable law. If you are not the intended recipient or authorized to receive this for the intended recipient, any use, dissemination, distribution, retention, archiving, or copying of this communication is strictly prohibited. If you have received this e-mail in error, please notify the sender immediately by reply e-mail and delete this message. |
