Menu
▾
▴
postgres-xc-developers
|
From: Nikhil S. <ni...@st...> - 2013-02-17 05:25:24
|
Hi Suzuki-san, > It's so helpful to have this patch. Will review and test it before committing. > Any comments on this patch? I do not see it in the commit logs. Regards, Nikhils > Best; > --- > Koichi Suzuki > > On Wed, 6 Feb 2013 14:25:48 +0530 > Nikhil Sontakke <ni...@st...> wrote: > >> > Also, gtm_ctl -w option >> > seems not work properly. >> > >> >> PFA, a patch to fix "gtm_ctl -w" behavior in GIT HEAD. I also tested >> "-w -t nsecs" behavior and it seems to work as well. This patch can be >> easily be backported to 1.0 if desired. I hope all these obnoxious gtm >> startup issues are resolved now. Scripting was a bit painful because >> of these issues. >> >> Regards, >> Nikhils >> >> > I will take a look at these issue and fix at least in the next major >> > release. Daemonization should be back-ported to 1.0.x. >> > >> > Kind Regards; >> > ---------- >> > Koichi Suzuki >> > >> > >> > 2012/12/27 Michael Meskes <me...@po...>: >> >> I still haven't found the time to dig into this, but could anyone please >> >> explain to me how gtm is supposed to start up? A simple grep command doesn't >> >> seem to find any call to setsid() in gtm subdirs. Could you please point me to >> >> the code where gtm daemonizes? Because if it doesn't correctly, there is no >> >> suprise gtm_ctl doesn't come back. Or in other words, this is a logical point >> >> to start looking. >> >> >> >> Michael >> >> -- >> >> Michael Meskes >> >> Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) >> >> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org >> >> Jabber: michael.meskes at gmail dot com >> >> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL >> >> >> >> ------------------------------------------------------------------------------ >> >> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, >> >> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current >> >> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft >> >> MVPs and experts. ON SALE this month only -- learn more at: >> >> http://p.sf.net/sfu/learnmore_122712 >> >> _______________________________________________ >> >> Postgres-xc-developers mailing list >> >> Pos...@li... >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> >> >> -- >> StormDB - http://www.stormdb.com >> The Database Cloud >> Postgres-XC Support and Service -- StormDB - http://www.stormdb.com The Database Cloud Postgres-XC Support and Service |
|
From: Koichi S. <ko...@in...> - 2013-02-18 02:06:23
|
Thanks. I tested the patch both on master and REL1_0_STABLE and found they look to work fine. I will commit it to both branches. I found there are two more issues on GTM. 1) If we kill GTM process, GTM will begin to assing GXID values with the one of the proveious GTM run. It is bad and harmful behaviour. To improve it, GTM should write the next gxid value to gtm.control. When gxid value reaches this backup, then the backup should be refreshed with the new one. That way, we can safely restart GTM even if it was killed. When GTM stops normally, it should write the next gxid value to gtm.control. 2) We have BARRIER to synchronize PITR recovery line. However, GTM does not have this capability. In this case, GTM's sequence definition could be inconsisitent if ALTER SEQUENCE or DROP SEQUENCE was issued after the recovery point. Have WAL in GTM could be a general choice but it seems to be too much for current GTM feature. Instead, we can have GTM.control backups when CREATE BARRIER is issued. Backup files will be qualified by BARRIER id so that DBA can choose which backup to use at the end of PITR. Regards; --- Koichi On Sun, 17 Feb 2013 10:27:37 +0530 Nikhil Sontakke <ni...@st...> wrote: > Hi Suzuki-san, > > > It's so helpful to have this patch. Will review and test it before committing. > > > > Any comments on this patch? I do not see it in the commit logs. > > Regards, > Nikhils > > > Best; > > --- > > Koichi Suzuki > > > > On Wed, 6 Feb 2013 14:25:48 +0530 > > Nikhil Sontakke <ni...@st...> wrote: > > > >> > Also, gtm_ctl -w option > >> > seems not work properly. > >> > > >> > >> PFA, a patch to fix "gtm_ctl -w" behavior in GIT HEAD. I also tested > >> "-w -t nsecs" behavior and it seems to work as well. This patch can be > >> easily be backported to 1.0 if desired. I hope all these obnoxious gtm > >> startup issues are resolved now. Scripting was a bit painful because > >> of these issues. > >> > >> Regards, > >> Nikhils > >> > >> > I will take a look at these issue and fix at least in the next major > >> > release. Daemonization should be back-ported to 1.0.x. > >> > > >> > Kind Regards; > >> > ---------- > >> > Koichi Suzuki > >> > > >> > > >> > 2012/12/27 Michael Meskes <me...@po...>: > >> >> I still haven't found the time to dig into this, but could anyone please > >> >> explain to me how gtm is supposed to start up? A simple grep command doesn't > >> >> seem to find any call to setsid() in gtm subdirs. Could you please point me to > >> >> the code where gtm daemonizes? Because if it doesn't correctly, there is no > >> >> suprise gtm_ctl doesn't come back. Or in other words, this is a logical point > >> >> to start looking. > >> >> > >> >> Michael > >> >> -- > >> >> Michael Meskes > >> >> Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) > >> >> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org > >> >> Jabber: michael.meskes at gmail dot com > >> >> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL > >> >> > >> >> ------------------------------------------------------------------------------ > >> >> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, > >> >> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current > >> >> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft > >> >> MVPs and experts. ON SALE this month only -- learn more at: > >> >> http://p.sf.net/sfu/learnmore_122712 > >> >> _______________________________________________ > >> >> Postgres-xc-developers mailing list > >> >> Pos...@li... > >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > >> > >> > >> > >> -- > >> StormDB - http://www.stormdb.com > >> The Database Cloud > >> Postgres-XC Support and Service > > > > -- > StormDB - http://www.stormdb.com > The Database Cloud > Postgres-XC Support and Service > |
|
From: Nikhil S. <ni...@st...> - 2013-02-18 14:52:24
|
> Thanks. I tested the patch both on master and REL1_0_STABLE and found they look to work fine. I will commit it to both branches. > Thanks Suzuki-san. > I found there are two more issues on GTM. > Yeah, these are very important from the correct functioning of the cluster. And both of them need to be tackled. And we need to think properly for a decent mechanism to do this. Specifically, the mechanism should not become a performance bottleneck. And yeah GTM WAL does seem to be an overkill. Maybe, we can have a GTM thread which writes the current GXID every 1000 increments. In case GTM dies and restarts in between, it can read the control file and start with the current value + 1000 as the next valid value. For sequences, again a similar mechanism can be though of. I admit I haven't looked at the current sequence handling. But we can introduce logic to track current saved value and current delta. Again if GTM dies and restarts, we can start with saved value + delta to be clear of existing sequence values. Thoughts? Any other better suggestions? Regards, Nikhils > 1) If we kill GTM process, GTM will begin to assing GXID values with the one of the proveious GTM run. It is bad and harmful behaviour. To improve it, GTM should write the next gxid value to gtm.control. When gxid value reaches this backup, then the backup should be refreshed with the new one. That way, we can safely restart GTM even if it was killed. When GTM stops normally, it should write the next gxid value to gtm.control. > > 2) We have BARRIER to synchronize PITR recovery line. However, GTM does not have this capability. In this case, GTM's sequence definition could be inconsisitent if ALTER SEQUENCE or DROP SEQUENCE was issued after the recovery point. Have WAL in GTM could be a general choice but it seems to be too much for current GTM feature. Instead, we can have GTM.control backups when CREATE BARRIER is issued. Backup files will be qualified by BARRIER id so that DBA can choose which backup to use at the end of PITR. > > Regards; > --- > Koichi > > On Sun, 17 Feb 2013 10:27:37 +0530 > Nikhil Sontakke <ni...@st...> wrote: > >> Hi Suzuki-san, >> >> > It's so helpful to have this patch. Will review and test it before committing. >> > >> >> Any comments on this patch? I do not see it in the commit logs. >> >> Regards, >> Nikhils >> >> > Best; >> > --- >> > Koichi Suzuki >> > >> > On Wed, 6 Feb 2013 14:25:48 +0530 >> > Nikhil Sontakke <ni...@st...> wrote: >> > >> >> > Also, gtm_ctl -w option >> >> > seems not work properly. >> >> > >> >> >> >> PFA, a patch to fix "gtm_ctl -w" behavior in GIT HEAD. I also tested >> >> "-w -t nsecs" behavior and it seems to work as well. This patch can be >> >> easily be backported to 1.0 if desired. I hope all these obnoxious gtm >> >> startup issues are resolved now. Scripting was a bit painful because >> >> of these issues. >> >> >> >> Regards, >> >> Nikhils >> >> >> >> > I will take a look at these issue and fix at least in the next major >> >> > release. Daemonization should be back-ported to 1.0.x. >> >> > >> >> > Kind Regards; >> >> > ---------- >> >> > Koichi Suzuki >> >> > >> >> > >> >> > 2012/12/27 Michael Meskes <me...@po...>: >> >> >> I still haven't found the time to dig into this, but could anyone please >> >> >> explain to me how gtm is supposed to start up? A simple grep command doesn't >> >> >> seem to find any call to setsid() in gtm subdirs. Could you please point me to >> >> >> the code where gtm daemonizes? Because if it doesn't correctly, there is no >> >> >> suprise gtm_ctl doesn't come back. Or in other words, this is a logical point >> >> >> to start looking. >> >> >> >> >> >> Michael >> >> >> -- >> >> >> Michael Meskes >> >> >> Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) >> >> >> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org >> >> >> Jabber: michael.meskes at gmail dot com >> >> >> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> >> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, >> >> >> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current >> >> >> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft >> >> >> MVPs and experts. ON SALE this month only -- learn more at: >> >> >> http://p.sf.net/sfu/learnmore_122712 >> >> >> _______________________________________________ >> >> >> Postgres-xc-developers mailing list >> >> >> Pos...@li... >> >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> >> >> >> >> >> >> -- >> >> StormDB - http://www.stormdb.com >> >> The Database Cloud >> >> Postgres-XC Support and Service >> >> >> >> -- >> StormDB - http://www.stormdb.com >> The Database Cloud >> Postgres-XC Support and Service >> -- StormDB - http://www.stormdb.com The Database Cloud Postgres-XC Support and Service |
|
From: Koichi S. <koi...@gm...> - 2013-02-18 15:00:12
|
Hi, 2013/2/18 Nikhil Sontakke <ni...@st...>: >> Thanks. I tested the patch both on master and REL1_0_STABLE and found they look to work fine. I will commit it to both branches. >> > > Thanks Suzuki-san. > >> I found there are two more issues on GTM. >> > > Yeah, these are very important from the correct functioning of the > cluster. And both of them need to be tackled. > > And we need to think properly for a decent mechanism to do this. > Specifically, the mechanism should not become a performance > bottleneck. And yeah GTM WAL does seem to be an overkill. > > Maybe, we can have a GTM thread which writes the current GXID every > 1000 increments. In case GTM dies and restarts in between, it can read > the control file and start with the current value + 1000 as the next > valid value. Yes, I was thinking the same idea. Number of increments can be a configuration parameter. Having dedicated thread could be a good idea which does not bring too much overhead to GTM opeation. > > For sequences, again a similar mechanism can be though of. I admit I > haven't looked at the current sequence handling. But we can introduce > logic to track current saved value and current delta. Again if GTM > dies and restarts, we can start with saved value + delta to be clear > of existing sequence values. Yes, exactly. We should take sequence in to account too. We need simple mechanism to synchronize coordinator/datanode PITR and GTM backup. GTM has sufficient infrastructure to backup its own status. I'm thinking to extend Barrier mechanism to trigger this gtm.control backup. > > Thoughts? Any other better suggestions? Regards; --- Koichi Suzuki > > Regards, > Nikhils > > >> 1) If we kill GTM process, GTM will begin to assing GXID values with the one of the proveious GTM run. It is bad and harmful behaviour. To improve it, GTM should write the next gxid value to gtm.control. When gxid value reaches this backup, then the backup should be refreshed with the new one. That way, we can safely restart GTM even if it was killed. When GTM stops normally, it should write the next gxid value to gtm.control. >> >> 2) We have BARRIER to synchronize PITR recovery line. However, GTM does not have this capability. In this case, GTM's sequence definition could be inconsisitent if ALTER SEQUENCE or DROP SEQUENCE was issued after the recovery point. Have WAL in GTM could be a general choice but it seems to be too much for current GTM feature. Instead, we can have GTM.control backups when CREATE BARRIER is issued. Backup files will be qualified by BARRIER id so that DBA can choose which backup to use at the end of PITR. >> >> Regards; >> --- >> Koichi >> >> On Sun, 17 Feb 2013 10:27:37 +0530 >> Nikhil Sontakke <ni...@st...> wrote: >> >>> Hi Suzuki-san, >>> >>> > It's so helpful to have this patch. Will review and test it before committing. >>> > >>> >>> Any comments on this patch? I do not see it in the commit logs. >>> >>> Regards, >>> Nikhils >>> >>> > Best; >>> > --- >>> > Koichi Suzuki >>> > >>> > On Wed, 6 Feb 2013 14:25:48 +0530 >>> > Nikhil Sontakke <ni...@st...> wrote: >>> > >>> >> > Also, gtm_ctl -w option >>> >> > seems not work properly. >>> >> > >>> >> >>> >> PFA, a patch to fix "gtm_ctl -w" behavior in GIT HEAD. I also tested >>> >> "-w -t nsecs" behavior and it seems to work as well. This patch can be >>> >> easily be backported to 1.0 if desired. I hope all these obnoxious gtm >>> >> startup issues are resolved now. Scripting was a bit painful because >>> >> of these issues. >>> >> >>> >> Regards, >>> >> Nikhils >>> >> >>> >> > I will take a look at these issue and fix at least in the next major >>> >> > release. Daemonization should be back-ported to 1.0.x. >>> >> > >>> >> > Kind Regards; >>> >> > ---------- >>> >> > Koichi Suzuki >>> >> > >>> >> > >>> >> > 2012/12/27 Michael Meskes <me...@po...>: >>> >> >> I still haven't found the time to dig into this, but could anyone please >>> >> >> explain to me how gtm is supposed to start up? A simple grep command doesn't >>> >> >> seem to find any call to setsid() in gtm subdirs. Could you please point me to >>> >> >> the code where gtm daemonizes? Because if it doesn't correctly, there is no >>> >> >> suprise gtm_ctl doesn't come back. Or in other words, this is a logical point >>> >> >> to start looking. >>> >> >> >>> >> >> Michael >>> >> >> -- >>> >> >> Michael Meskes >>> >> >> Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) >>> >> >> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org >>> >> >> Jabber: michael.meskes at gmail dot com >>> >> >> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, PostgreSQL >>> >> >> >>> >> >> ------------------------------------------------------------------------------ >>> >> >> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, >>> >> >> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current >>> >> >> with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft >>> >> >> MVPs and experts. ON SALE this month only -- learn more at: >>> >> >> http://p.sf.net/sfu/learnmore_122712 >>> >> >> _______________________________________________ >>> >> >> Postgres-xc-developers mailing list >>> >> >> Pos...@li... >>> >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> >> >>> >> >>> >> >>> >> -- >>> >> StormDB - http://www.stormdb.com >>> >> The Database Cloud >>> >> Postgres-XC Support and Service >>> >>> >>> >>> -- >>> StormDB - http://www.stormdb.com >>> The Database Cloud >>> Postgres-XC Support and Service >>> > > > > -- > StormDB - http://www.stormdb.com > The Database Cloud > Postgres-XC Support and Service |
|
From: Mason S. <ma...@st...> - 2013-02-18 15:42:26
|
On Mon, Feb 18, 2013 at 10:00 AM, Koichi Suzuki <koi...@gm...>wrote: > Hi, > > 2013/2/18 Nikhil Sontakke <ni...@st...>: > >> Thanks. I tested the patch both on master and REL1_0_STABLE and found > they look to work fine. I will commit it to both branches. > >> > > > > Thanks Suzuki-san. > > > >> I found there are two more issues on GTM. > >> > > > > Yeah, these are very important from the correct functioning of the > > cluster. And both of them need to be tackled. > > > > And we need to think properly for a decent mechanism to do this. > > Specifically, the mechanism should not become a performance > > bottleneck. And yeah GTM WAL does seem to be an overkill. > > > > Maybe, we can have a GTM thread which writes the current GXID every > > 1000 increments. In case GTM dies and restarts in between, it can read > > the control file and start with the current value + 1000 as the next > > valid value. > > Yes, I was thinking the same idea. Number of increments can be a > configuration > parameter. Having dedicated thread could be a good idea which does not > bring > too much overhead to GTM opeation. > > > > > For sequences, again a similar mechanism can be though of. I admit I > > haven't looked at the current sequence handling. But we can introduce > > logic to track current saved value and current delta. Again if GTM > > dies and restarts, we can start with saved value + delta to be clear > > of existing sequence values. > We also need to be mindful of XID wraparound. > > Yes, exactly. We should take sequence in to account too. We need > simple mechanism to synchronize coordinator/datanode PITR and GTM > backup. GTM has sufficient infrastructure to backup its own status. > I'm thinking to extend Barrier mechanism to trigger this gtm.control > backup. > Yes, sequences are trickier. A single transaction can increment multiple sequences. Barrier seems like one good way to do this periodically, but we may be missing info and may have to query the individual data nodes in order to get the actual latest sequence values. We may instead also consider a WAL-type of mechanism where we write the latest sequence values that a transaction updated at transaction commit. With GTM Proxy grouping requests together, too, and by grouping from the other threads, maybe a single GTM WAL fsync can write out quite a lot of information. > > > > > Thoughts? Any other better suggestions? > > Regards; > --- > Koichi Suzuki > > > > > Regards, > > Nikhils > > > > > >> 1) If we kill GTM process, GTM will begin to assing GXID values with > the one of the proveious GTM run. It is bad and harmful behaviour. To > improve it, GTM should write the next gxid value to gtm.control. When > gxid value reaches this backup, then the backup should be refreshed with > the new one. That way, we can safely restart GTM even if it was killed. > When GTM stops normally, it should write the next gxid value to gtm.control. > >> > >> 2) We have BARRIER to synchronize PITR recovery line. However, GTM > does not have this capability. In this case, GTM's sequence definition > could be inconsisitent if ALTER SEQUENCE or DROP SEQUENCE was issued after > the recovery point. Have WAL in GTM could be a general choice but it > seems to be too much for current GTM feature. Instead, we can have > GTM.control backups when CREATE BARRIER is issued. Backup files will be > qualified by BARRIER id so that DBA can choose which backup to use at the > end of PITR. > >> > >> Regards; > >> --- > >> Koichi > >> > >> On Sun, 17 Feb 2013 10:27:37 +0530 > >> Nikhil Sontakke <ni...@st...> wrote: > >> > >>> Hi Suzuki-san, > >>> > >>> > It's so helpful to have this patch. Will review and test it before > committing. > >>> > > >>> > >>> Any comments on this patch? I do not see it in the commit logs. > >>> > >>> Regards, > >>> Nikhils > >>> > >>> > Best; > >>> > --- > >>> > Koichi Suzuki > >>> > > >>> > On Wed, 6 Feb 2013 14:25:48 +0530 > >>> > Nikhil Sontakke <ni...@st...> wrote: > >>> > > >>> >> > Also, gtm_ctl -w option > >>> >> > seems not work properly. > >>> >> > > >>> >> > >>> >> PFA, a patch to fix "gtm_ctl -w" behavior in GIT HEAD. I also tested > >>> >> "-w -t nsecs" behavior and it seems to work as well. This patch can > be > >>> >> easily be backported to 1.0 if desired. I hope all these obnoxious > gtm > >>> >> startup issues are resolved now. Scripting was a bit painful because > >>> >> of these issues. > >>> >> > >>> >> Regards, > >>> >> Nikhils > >>> >> > >>> >> > I will take a look at these issue and fix at least in the next > major > >>> >> > release. Daemonization should be back-ported to 1.0.x. > >>> >> > > >>> >> > Kind Regards; > >>> >> > ---------- > >>> >> > Koichi Suzuki > >>> >> > > >>> >> > > >>> >> > 2012/12/27 Michael Meskes <me...@po...>: > >>> >> >> I still haven't found the time to dig into this, but could > anyone please > >>> >> >> explain to me how gtm is supposed to start up? A simple grep > command doesn't > >>> >> >> seem to find any call to setsid() in gtm subdirs. Could you > please point me to > >>> >> >> the code where gtm daemonizes? Because if it doesn't correctly, > there is no > >>> >> >> suprise gtm_ctl doesn't come back. Or in other words, this is a > logical point > >>> >> >> to start looking. > >>> >> >> > >>> >> >> Michael > >>> >> >> -- > >>> >> >> Michael Meskes > >>> >> >> Michael at Fam-Meskes dot De, Michael at Meskes dot > (De|Com|Net|Org) > >>> >> >> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot > Org > >>> >> >> Jabber: michael.meskes at gmail dot com > >>> >> >> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, > PostgreSQL > >>> >> >> > >>> >> >> > ------------------------------------------------------------------------------ > >>> >> >> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, > CSS, > >>> >> >> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills > current > >>> >> >> with LearnDevNow - 3,200 step-by-step video tutorials by > Microsoft > >>> >> >> MVPs and experts. ON SALE this month only -- learn more at: > >>> >> >> http://p.sf.net/sfu/learnmore_122712 > >>> >> >> _______________________________________________ > >>> >> >> Postgres-xc-developers mailing list > >>> >> >> Pos...@li... > >>> >> >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> StormDB - http://www.stormdb.com > >>> >> The Database Cloud > >>> >> Postgres-XC Support and Service > >>> > >>> > >>> > >>> -- > >>> StormDB - http://www.stormdb.com > >>> The Database Cloud > >>> Postgres-XC Support and Service > >>> > > > > > > > > -- > > StormDB - http://www.stormdb.com > > The Database Cloud > > Postgres-XC Support and Service > > > ------------------------------------------------------------------------------ > The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, > is your hub for all things parallel software development, from weekly > thought > leadership blogs to news, videos, case studies, tutorials, tech docs, > whitepapers, evaluation guides, and opinion stories. Check out the most > recent posts - join the conversation now. > http://goparallel.sourceforge.net/ > _______________________________________________ > Postgres-xc-developers mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > -- Mason Sharp StormDB - http://www.stormdb.com The Database Cloud Postgres-XC Support and Services |
|
From: Nikhil S. <ni...@st...> - 2013-02-18 17:01:54
|
> > > Yes, sequences are trickier. A single transaction can increment multiple > sequences. Barrier seems like one good way to do this periodically, but we > may be missing info and may have to query the individual data nodes in order > to get the actual latest sequence values. > I am not in favor of relying on barrier for GTM stuff. Just implementing currval+delta for both XID and sequences seems OK enough to me and it keeps things within the GTM scheme of things. > We may instead also consider a WAL-type of mechanism where we write the > latest sequence values that a transaction updated at transaction commit. > With GTM Proxy grouping requests together, too, and by grouping from the > other threads, maybe a single GTM WAL fsync can write out quite a lot of > information. > IMHO, WAL types implementation seems overkill to me. Configurable delta based logic to write to control file can whet everyone's appetite. Regards, Nikhils > >> >> >> > >> > Thoughts? Any other better suggestions? >> >> Regards; >> --- >> Koichi Suzuki >> >> > >> > Regards, >> > Nikhils >> > >> > >> >> 1) If we kill GTM process, GTM will begin to assing GXID values with >> >> the one of the proveious GTM run. It is bad and harmful behaviour. To >> >> improve it, GTM should write the next gxid value to gtm.control. When gxid >> >> value reaches this backup, then the backup should be refreshed with the new >> >> one. That way, we can safely restart GTM even if it was killed. When GTM >> >> stops normally, it should write the next gxid value to gtm.control. >> >> >> >> 2) We have BARRIER to synchronize PITR recovery line. However, GTM >> >> does not have this capability. In this case, GTM's sequence definition >> >> could be inconsisitent if ALTER SEQUENCE or DROP SEQUENCE was issued after >> >> the recovery point. Have WAL in GTM could be a general choice but it seems >> >> to be too much for current GTM feature. Instead, we can have GTM.control >> >> backups when CREATE BARRIER is issued. Backup files will be qualified by >> >> BARRIER id so that DBA can choose which backup to use at the end of PITR. >> >> >> >> Regards; >> >> --- >> >> Koichi >> >> >> >> On Sun, 17 Feb 2013 10:27:37 +0530 >> >> Nikhil Sontakke <ni...@st...> wrote: >> >> >> >>> Hi Suzuki-san, >> >>> >> >>> > It's so helpful to have this patch. Will review and test it before >> >>> > committing. >> >>> > >> >>> >> >>> Any comments on this patch? I do not see it in the commit logs. >> >>> >> >>> Regards, >> >>> Nikhils >> >>> >> >>> > Best; >> >>> > --- >> >>> > Koichi Suzuki >> >>> > >> >>> > On Wed, 6 Feb 2013 14:25:48 +0530 >> >>> > Nikhil Sontakke <ni...@st...> wrote: >> >>> > >> >>> >> > Also, gtm_ctl -w option >> >>> >> > seems not work properly. >> >>> >> > >> >>> >> >> >>> >> PFA, a patch to fix "gtm_ctl -w" behavior in GIT HEAD. I also >> >>> >> tested >> >>> >> "-w -t nsecs" behavior and it seems to work as well. This patch can >> >>> >> be >> >>> >> easily be backported to 1.0 if desired. I hope all these obnoxious >> >>> >> gtm >> >>> >> startup issues are resolved now. Scripting was a bit painful >> >>> >> because >> >>> >> of these issues. >> >>> >> >> >>> >> Regards, >> >>> >> Nikhils >> >>> >> >> >>> >> > I will take a look at these issue and fix at least in the next >> >>> >> > major >> >>> >> > release. Daemonization should be back-ported to 1.0.x. >> >>> >> > >> >>> >> > Kind Regards; >> >>> >> > ---------- >> >>> >> > Koichi Suzuki >> >>> >> > >> >>> >> > >> >>> >> > 2012/12/27 Michael Meskes <me...@po...>: >> >>> >> >> I still haven't found the time to dig into this, but could >> >>> >> >> anyone please >> >>> >> >> explain to me how gtm is supposed to start up? A simple grep >> >>> >> >> command doesn't >> >>> >> >> seem to find any call to setsid() in gtm subdirs. Could you >> >>> >> >> please point me to >> >>> >> >> the code where gtm daemonizes? Because if it doesn't correctly, >> >>> >> >> there is no >> >>> >> >> suprise gtm_ctl doesn't come back. Or in other words, this is a >> >>> >> >> logical point >> >>> >> >> to start looking. >> >>> >> >> >> >>> >> >> Michael >> >>> >> >> -- >> >>> >> >> Michael Meskes >> >>> >> >> Michael at Fam-Meskes dot De, Michael at Meskes dot >> >>> >> >> (De|Com|Net|Org) >> >>> >> >> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot >> >>> >> >> Org >> >>> >> >> Jabber: michael.meskes at gmail dot com >> >>> >> >> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, >> >>> >> >> PostgreSQL >> >>> >> >> >> >>> >> >> >> >>> >> >> ------------------------------------------------------------------------------ >> >>> >> >> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, >> >>> >> >> CSS, >> >>> >> >> MVC, Windows 8 Apps, JavaScript and much more. Keep your skills >> >>> >> >> current >> >>> >> >> with LearnDevNow - 3,200 step-by-step video tutorials by >> >>> >> >> Microsoft >> >>> >> >> MVPs and experts. ON SALE this month only -- learn more at: >> >>> >> >> http://p.sf.net/sfu/learnmore_122712 >> >>> >> >> _______________________________________________ >> >>> >> >> Postgres-xc-developers mailing list >> >>> >> >> Pos...@li... >> >>> >> >> >> >>> >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >>> >> >> >>> >> >> >>> >> >> >>> >> -- >> >>> >> StormDB - http://www.stormdb.com >> >>> >> The Database Cloud >> >>> >> Postgres-XC Support and Service >> >>> >> >>> >> >>> >> >>> -- >> >>> StormDB - http://www.stormdb.com >> >>> The Database Cloud >> >>> Postgres-XC Support and Service >> >>> >> > >> > >> > >> > -- >> > StormDB - http://www.stormdb.com >> > The Database Cloud >> > Postgres-XC Support and Service >> >> >> ------------------------------------------------------------------------------ >> The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, >> is your hub for all things parallel software development, from weekly >> thought >> leadership blogs to news, videos, case studies, tutorials, tech docs, >> whitepapers, evaluation guides, and opinion stories. Check out the most >> recent posts - join the conversation now. >> http://goparallel.sourceforge.net/ >> >> _______________________________________________ >> Postgres-xc-developers mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > -- > Mason Sharp > > > StormDB - http://www.stormdb.com > The Database Cloud > Postgres-XC Support and Services -- StormDB - http://www.stormdb.com The Database Cloud Postgres-XC Support and Service |
|
From: Mason S. <ma...@st...> - 2013-02-18 18:43:38
|
On Mon, Feb 18, 2013 at 11:53 AM, Nikhil Sontakke <ni...@st...>wrote: > > > > > > Yes, sequences are trickier. A single transaction can increment multiple > > sequences. Barrier seems like one good way to do this periodically, but > we > > may be missing info and may have to query the individual data nodes in > order > > to get the actual latest sequence values. > > > > I am not in favor of relying on barrier for GTM stuff. Just > implementing currval+delta for both XID and sequences seems OK enough > to me and it keeps things within the GTM scheme of things. > I think the barrier makes sense when doing a barrier with the rest of the cluster components. Then each element is recoverable to the same point in time. > > > We may instead also consider a WAL-type of mechanism where we write the > > latest sequence values that a transaction updated at transaction commit. > > With GTM Proxy grouping requests together, too, and by grouping from the > > other threads, maybe a single GTM WAL fsync can write out quite a lot of > > information. > > > > IMHO, WAL types implementation seems overkill to me. Configurable > delta based logic to write to control file can whet everyone's > appetite. > Most important is XID. With a valid XID, we can get the current sequence value from the individual nodes. We could consider optionally that when a data node connects to GTM that GTM acknowledges and then requests back all of the sequence values from the node. Then GTM can take the maximum from all the nodes to be able to determine what the current value is cluster-wide. No need for an external tool. For the nodes, we can implement it as a new stored function. Regards, Mason > > Regards, > Nikhils > > > > > >> > >> > >> > > >> > Thoughts? Any other better suggestions? > >> > >> Regards; > >> --- > >> Koichi Suzuki > >> > >> > > >> > Regards, > >> > Nikhils > >> > > >> > > >> >> 1) If we kill GTM process, GTM will begin to assing GXID values with > >> >> the one of the proveious GTM run. It is bad and harmful behaviour. > To > >> >> improve it, GTM should write the next gxid value to gtm.control. > When gxid > >> >> value reaches this backup, then the backup should be refreshed with > the new > >> >> one. That way, we can safely restart GTM even if it was killed. > When GTM > >> >> stops normally, it should write the next gxid value to gtm.control. > >> >> > >> >> 2) We have BARRIER to synchronize PITR recovery line. However, GTM > >> >> does not have this capability. In this case, GTM's sequence > definition > >> >> could be inconsisitent if ALTER SEQUENCE or DROP SEQUENCE was issued > after > >> >> the recovery point. Have WAL in GTM could be a general choice but > it seems > >> >> to be too much for current GTM feature. Instead, we can have > GTM.control > >> >> backups when CREATE BARRIER is issued. Backup files will be > qualified by > >> >> BARRIER id so that DBA can choose which backup to use at the end of > PITR. > >> >> > >> >> Regards; > >> >> --- > >> >> Koichi > >> >> > >> >> On Sun, 17 Feb 2013 10:27:37 +0530 > >> >> Nikhil Sontakke <ni...@st...> wrote: > >> >> > >> >>> Hi Suzuki-san, > >> >>> > >> >>> > It's so helpful to have this patch. Will review and test it > before > >> >>> > committing. > >> >>> > > >> >>> > >> >>> Any comments on this patch? I do not see it in the commit logs. > >> >>> > >> >>> Regards, > >> >>> Nikhils > >> >>> > >> >>> > Best; > >> >>> > --- > >> >>> > Koichi Suzuki > >> >>> > > >> >>> > On Wed, 6 Feb 2013 14:25:48 +0530 > >> >>> > Nikhil Sontakke <ni...@st...> wrote: > >> >>> > > >> >>> >> > Also, gtm_ctl -w option > >> >>> >> > seems not work properly. > >> >>> >> > > >> >>> >> > >> >>> >> PFA, a patch to fix "gtm_ctl -w" behavior in GIT HEAD. I also > >> >>> >> tested > >> >>> >> "-w -t nsecs" behavior and it seems to work as well. This patch > can > >> >>> >> be > >> >>> >> easily be backported to 1.0 if desired. I hope all these > obnoxious > >> >>> >> gtm > >> >>> >> startup issues are resolved now. Scripting was a bit painful > >> >>> >> because > >> >>> >> of these issues. > >> >>> >> > >> >>> >> Regards, > >> >>> >> Nikhils > >> >>> >> > >> >>> >> > I will take a look at these issue and fix at least in the next > >> >>> >> > major > >> >>> >> > release. Daemonization should be back-ported to 1.0.x. > >> >>> >> > > >> >>> >> > Kind Regards; > >> >>> >> > ---------- > >> >>> >> > Koichi Suzuki > >> >>> >> > > >> >>> >> > > >> >>> >> > 2012/12/27 Michael Meskes <me...@po...>: > >> >>> >> >> I still haven't found the time to dig into this, but could > >> >>> >> >> anyone please > >> >>> >> >> explain to me how gtm is supposed to start up? A simple grep > >> >>> >> >> command doesn't > >> >>> >> >> seem to find any call to setsid() in gtm subdirs. Could you > >> >>> >> >> please point me to > >> >>> >> >> the code where gtm daemonizes? Because if it doesn't > correctly, > >> >>> >> >> there is no > >> >>> >> >> suprise gtm_ctl doesn't come back. Or in other words, this is > a > >> >>> >> >> logical point > >> >>> >> >> to start looking. > >> >>> >> >> > >> >>> >> >> Michael > >> >>> >> >> -- > >> >>> >> >> Michael Meskes > >> >>> >> >> Michael at Fam-Meskes dot De, Michael at Meskes dot > >> >>> >> >> (De|Com|Net|Org) > >> >>> >> >> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) > dot > >> >>> >> >> Org > >> >>> >> >> Jabber: michael.meskes at gmail dot com > >> >>> >> >> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, > >> >>> >> >> PostgreSQL > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > ------------------------------------------------------------------------------ > >> >>> >> >> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, > HTML5, > >> >>> >> >> CSS, > >> >>> >> >> MVC, Windows 8 Apps, JavaScript and much more. Keep your > skills > >> >>> >> >> current > >> >>> >> >> with LearnDevNow - 3,200 step-by-step video tutorials by > >> >>> >> >> Microsoft > >> >>> >> >> MVPs and experts. ON SALE this month only -- learn more at: > >> >>> >> >> http://p.sf.net/sfu/learnmore_122712 > >> >>> >> >> _______________________________________________ > >> >>> >> >> Postgres-xc-developers mailing list > >> >>> >> >> Pos...@li... > >> >>> >> >> > >> >>> >> >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> -- > >> >>> >> StormDB - http://www.stormdb.com > >> >>> >> The Database Cloud > >> >>> >> Postgres-XC Support and Service > >> >>> > >> >>> > >> >>> > >> >>> -- > >> >>> StormDB - http://www.stormdb.com > >> >>> The Database Cloud > >> >>> Postgres-XC Support and Service > >> >>> > >> > > >> > > >> > > >> > -- > >> > StormDB - http://www.stormdb.com > >> > The Database Cloud > >> > Postgres-XC Support and Service > >> > >> > >> > ------------------------------------------------------------------------------ > >> The Go Parallel Website, sponsored by Intel - in partnership with > Geeknet, > >> is your hub for all things parallel software development, from weekly > >> thought > >> leadership blogs to news, videos, case studies, tutorials, tech docs, > >> whitepapers, evaluation guides, and opinion stories. Check out the most > >> recent posts - join the conversation now. > >> http://goparallel.sourceforge.net/ > >> > >> _______________________________________________ > >> Postgres-xc-developers mailing list > >> Pos...@li... > >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers > > > > > > > > > > -- > > Mason Sharp > > > > > > StormDB - http://www.stormdb.com > > The Database Cloud > > Postgres-XC Support and Services > > > > -- > StormDB - http://www.stormdb.com > The Database Cloud > Postgres-XC Support and Service > -- Mason Sharp StormDB - http://www.stormdb.com The Database Cloud Postgres-XC Support and Services |
|
From: Koichi S. <koi...@gm...> - 2013-02-19 02:26:47
|
2013/2/19 Mason Sharp <ma...@st...>: > > > On Mon, Feb 18, 2013 at 11:53 AM, Nikhil Sontakke <ni...@st...> > wrote: >> >> > >> > >> > Yes, sequences are trickier. A single transaction can increment multiple >> > sequences. Barrier seems like one good way to do this periodically, but >> > we >> > may be missing info and may have to query the individual data nodes in >> > order >> > to get the actual latest sequence values. >> > >> >> I am not in favor of relying on barrier for GTM stuff. Just >> implementing currval+delta for both XID and sequences seems OK enough >> to me and it keeps things within the GTM scheme of things. > > > I think the barrier makes sense when doing a barrier with the rest of the > cluster components. Then each element is recoverable to the same point in > time. Backing up gtm.control triggerd by BARRIER is not associated with WAL at all. My idea is 1) add new message to GTM to report BARRIER is going to be recorded, with barrier ID. 2) When gtm (and gtm slave) receives this message, gtm/gtm slave creates gtm.control backup. Important thing is to synchronize PITR recovery point with gtm.control. > >> >> >> > We may instead also consider a WAL-type of mechanism where we write the >> > latest sequence values that a transaction updated at transaction commit. >> > With GTM Proxy grouping requests together, too, and by grouping from the >> > other threads, maybe a single GTM WAL fsync can write out quite a lot of >> > information. >> > >> >> IMHO, WAL types implementation seems overkill to me. Configurable >> delta based logic to write to control file can whet everyone's >> appetite. Same to me. > > > Most important is XID. With a valid XID, we can get the current sequence > value from the individual nodes. Well, this will save some value consumed by aborted TXs. This is wasted in vanilla PostgreSQL too and it does not violate sequence consistency. > > We could consider optionally that when a data node connects to GTM that GTM > acknowledges and then requests back all of the sequence values from the > node. Then GTM can take the maximum from all the nodes to be able to > determine what the current value is cluster-wide. No need for an external > tool. For the nodes, we can implement it as a new stored function. I don't think we need this now. As Mason wrote, yes we have to consider GXID wraparound. Regards; --- Koichi > > Regards, > > Mason > > >> >> >> Regards, >> Nikhils >> >> >> > >> >> >> >> >> >> > >> >> > Thoughts? Any other better suggestions? >> >> >> >> Regards; >> >> --- >> >> Koichi Suzuki >> >> >> >> > >> >> > Regards, >> >> > Nikhils >> >> > >> >> > >> >> >> 1) If we kill GTM process, GTM will begin to assing GXID values with >> >> >> the one of the proveious GTM run. It is bad and harmful behaviour. >> >> >> To >> >> >> improve it, GTM should write the next gxid value to gtm.control. >> >> >> When gxid >> >> >> value reaches this backup, then the backup should be refreshed with >> >> >> the new >> >> >> one. That way, we can safely restart GTM even if it was killed. >> >> >> When GTM >> >> >> stops normally, it should write the next gxid value to gtm.control. >> >> >> >> >> >> 2) We have BARRIER to synchronize PITR recovery line. However, GTM >> >> >> does not have this capability. In this case, GTM's sequence >> >> >> definition >> >> >> could be inconsisitent if ALTER SEQUENCE or DROP SEQUENCE was issued >> >> >> after >> >> >> the recovery point. Have WAL in GTM could be a general choice but >> >> >> it seems >> >> >> to be too much for current GTM feature. Instead, we can have >> >> >> GTM.control >> >> >> backups when CREATE BARRIER is issued. Backup files will be >> >> >> qualified by >> >> >> BARRIER id so that DBA can choose which backup to use at the end of >> >> >> PITR. >> >> >> >> >> >> Regards; >> >> >> --- >> >> >> Koichi >> >> >> >> >> >> On Sun, 17 Feb 2013 10:27:37 +0530 >> >> >> Nikhil Sontakke <ni...@st...> wrote: >> >> >> >> >> >>> Hi Suzuki-san, >> >> >>> >> >> >>> > It's so helpful to have this patch. Will review and test it >> >> >>> > before >> >> >>> > committing. >> >> >>> > >> >> >>> >> >> >>> Any comments on this patch? I do not see it in the commit logs. >> >> >>> >> >> >>> Regards, >> >> >>> Nikhils >> >> >>> >> >> >>> > Best; >> >> >>> > --- >> >> >>> > Koichi Suzuki >> >> >>> > >> >> >>> > On Wed, 6 Feb 2013 14:25:48 +0530 >> >> >>> > Nikhil Sontakke <ni...@st...> wrote: >> >> >>> > >> >> >>> >> > Also, gtm_ctl -w option >> >> >>> >> > seems not work properly. >> >> >>> >> > >> >> >>> >> >> >> >>> >> PFA, a patch to fix "gtm_ctl -w" behavior in GIT HEAD. I also >> >> >>> >> tested >> >> >>> >> "-w -t nsecs" behavior and it seems to work as well. This patch >> >> >>> >> can >> >> >>> >> be >> >> >>> >> easily be backported to 1.0 if desired. I hope all these >> >> >>> >> obnoxious >> >> >>> >> gtm >> >> >>> >> startup issues are resolved now. Scripting was a bit painful >> >> >>> >> because >> >> >>> >> of these issues. >> >> >>> >> >> >> >>> >> Regards, >> >> >>> >> Nikhils >> >> >>> >> >> >> >>> >> > I will take a look at these issue and fix at least in the next >> >> >>> >> > major >> >> >>> >> > release. Daemonization should be back-ported to 1.0.x. >> >> >>> >> > >> >> >>> >> > Kind Regards; >> >> >>> >> > ---------- >> >> >>> >> > Koichi Suzuki >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > 2012/12/27 Michael Meskes <me...@po...>: >> >> >>> >> >> I still haven't found the time to dig into this, but could >> >> >>> >> >> anyone please >> >> >>> >> >> explain to me how gtm is supposed to start up? A simple grep >> >> >>> >> >> command doesn't >> >> >>> >> >> seem to find any call to setsid() in gtm subdirs. Could you >> >> >>> >> >> please point me to >> >> >>> >> >> the code where gtm daemonizes? Because if it doesn't >> >> >>> >> >> correctly, >> >> >>> >> >> there is no >> >> >>> >> >> suprise gtm_ctl doesn't come back. Or in other words, this is >> >> >>> >> >> a >> >> >>> >> >> logical point >> >> >>> >> >> to start looking. >> >> >>> >> >> >> >> >>> >> >> Michael >> >> >>> >> >> -- >> >> >>> >> >> Michael Meskes >> >> >>> >> >> Michael at Fam-Meskes dot De, Michael at Meskes dot >> >> >>> >> >> (De|Com|Net|Org) >> >> >>> >> >> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) >> >> >>> >> >> dot >> >> >>> >> >> Org >> >> >>> >> >> Jabber: michael.meskes at gmail dot com >> >> >>> >> >> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, >> >> >>> >> >> PostgreSQL >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> ------------------------------------------------------------------------------ >> >> >>> >> >> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, >> >> >>> >> >> HTML5, >> >> >>> >> >> CSS, >> >> >>> >> >> MVC, Windows 8 Apps, JavaScript and much more. Keep your >> >> >>> >> >> skills >> >> >>> >> >> current >> >> >>> >> >> with LearnDevNow - 3,200 step-by-step video tutorials by >> >> >>> >> >> Microsoft >> >> >>> >> >> MVPs and experts. ON SALE this month only -- learn more at: >> >> >>> >> >> http://p.sf.net/sfu/learnmore_122712 >> >> >>> >> >> _______________________________________________ >> >> >>> >> >> Postgres-xc-developers mailing list >> >> >>> >> >> Pos...@li... >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> -- >> >> >>> >> StormDB - http://www.stormdb.com >> >> >>> >> The Database Cloud >> >> >>> >> Postgres-XC Support and Service >> >> >>> >> >> >>> >> >> >>> >> >> >>> -- >> >> >>> StormDB - http://www.stormdb.com >> >> >>> The Database Cloud >> >> >>> Postgres-XC Support and Service >> >> >>> >> >> > >> >> > >> >> > >> >> > -- >> >> > StormDB - http://www.stormdb.com >> >> > The Database Cloud >> >> > Postgres-XC Support and Service >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> The Go Parallel Website, sponsored by Intel - in partnership with >> >> Geeknet, >> >> is your hub for all things parallel software development, from weekly >> >> thought >> >> leadership blogs to news, videos, case studies, tutorials, tech docs, >> >> whitepapers, evaluation guides, and opinion stories. Check out the most >> >> recent posts - join the conversation now. >> >> http://goparallel.sourceforge.net/ >> >> >> >> _______________________________________________ >> >> Postgres-xc-developers mailing list >> >> Pos...@li... >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >> > >> > >> > >> > >> > -- >> > Mason Sharp >> > >> > >> > StormDB - http://www.stormdb.com >> > The Database Cloud >> > Postgres-XC Support and Services >> >> >> >> -- >> StormDB - http://www.stormdb.com >> The Database Cloud >> Postgres-XC Support and Service > > > > > -- > Mason Sharp > > StormDB - http://www.stormdb.com > The Database Cloud > Postgres-XC Support and Services |
|
From: Nikhil S. <ni...@st...> - 2013-02-19 05:24:10
|
>> >> I think the barrier makes sense when doing a barrier with the rest of the >> cluster components. Then each element is recoverable to the same point in >> time. Oh yes. I did not mean that BARRIER activity should not track GTM stuff. I only meant that GTM behavior should be done independent of BARRIER calls as well. > > Backing up gtm.control triggerd by BARRIER is not associated with WAL > at all. My idea is > 1) add new message to GTM to report BARRIER is going to be recorded, > with barrier ID. > 2) When gtm (and gtm slave) receives this message, gtm/gtm slave > creates gtm.control backup. > > Important thing is to synchronize PITR recovery point with gtm.control. > Yeah, +1 for this approach to integrate GTM with BARRIER activity. >> Most important is XID. With a valid XID, we can get the current sequence >> value from the individual nodes. > > Well, this will save some value consumed by aborted TXs. This is > wasted in vanilla PostgreSQL too and it does not violate sequence > consistency. > Yeah, this loose requirement also makes our life easier. We can just append an appropriate delta to get the next "sane" value for both GXID and sequences. Unless again if I am mistaken about the semantics or something. Regards, Nikhils >> >> We could consider optionally that when a data node connects to GTM that GTM >> acknowledges and then requests back all of the sequence values from the >> node. Then GTM can take the maximum from all the nodes to be able to >> determine what the current value is cluster-wide. No need for an external >> tool. For the nodes, we can implement it as a new stored function. > > I don't think we need this now. > > As Mason wrote, yes we have to consider GXID wraparound. > > Regards; > --- > Koichi > >> >> Regards, >> >> Mason >> >> >>> >>> >>> Regards, >>> Nikhils >>> >>> >>> > >>> >> >>> >> >>> >> > >>> >> > Thoughts? Any other better suggestions? >>> >> >>> >> Regards; >>> >> --- >>> >> Koichi Suzuki >>> >> >>> >> > >>> >> > Regards, >>> >> > Nikhils >>> >> > >>> >> > >>> >> >> 1) If we kill GTM process, GTM will begin to assing GXID values with >>> >> >> the one of the proveious GTM run. It is bad and harmful behaviour. >>> >> >> To >>> >> >> improve it, GTM should write the next gxid value to gtm.control. >>> >> >> When gxid >>> >> >> value reaches this backup, then the backup should be refreshed with >>> >> >> the new >>> >> >> one. That way, we can safely restart GTM even if it was killed. >>> >> >> When GTM >>> >> >> stops normally, it should write the next gxid value to gtm.control. >>> >> >> >>> >> >> 2) We have BARRIER to synchronize PITR recovery line. However, GTM >>> >> >> does not have this capability. In this case, GTM's sequence >>> >> >> definition >>> >> >> could be inconsisitent if ALTER SEQUENCE or DROP SEQUENCE was issued >>> >> >> after >>> >> >> the recovery point. Have WAL in GTM could be a general choice but >>> >> >> it seems >>> >> >> to be too much for current GTM feature. Instead, we can have >>> >> >> GTM.control >>> >> >> backups when CREATE BARRIER is issued. Backup files will be >>> >> >> qualified by >>> >> >> BARRIER id so that DBA can choose which backup to use at the end of >>> >> >> PITR. >>> >> >> >>> >> >> Regards; >>> >> >> --- >>> >> >> Koichi >>> >> >> >>> >> >> On Sun, 17 Feb 2013 10:27:37 +0530 >>> >> >> Nikhil Sontakke <ni...@st...> wrote: >>> >> >> >>> >> >>> Hi Suzuki-san, >>> >> >>> >>> >> >>> > It's so helpful to have this patch. Will review and test it >>> >> >>> > before >>> >> >>> > committing. >>> >> >>> > >>> >> >>> >>> >> >>> Any comments on this patch? I do not see it in the commit logs. >>> >> >>> >>> >> >>> Regards, >>> >> >>> Nikhils >>> >> >>> >>> >> >>> > Best; >>> >> >>> > --- >>> >> >>> > Koichi Suzuki >>> >> >>> > >>> >> >>> > On Wed, 6 Feb 2013 14:25:48 +0530 >>> >> >>> > Nikhil Sontakke <ni...@st...> wrote: >>> >> >>> > >>> >> >>> >> > Also, gtm_ctl -w option >>> >> >>> >> > seems not work properly. >>> >> >>> >> > >>> >> >>> >> >>> >> >>> >> PFA, a patch to fix "gtm_ctl -w" behavior in GIT HEAD. I also >>> >> >>> >> tested >>> >> >>> >> "-w -t nsecs" behavior and it seems to work as well. This patch >>> >> >>> >> can >>> >> >>> >> be >>> >> >>> >> easily be backported to 1.0 if desired. I hope all these >>> >> >>> >> obnoxious >>> >> >>> >> gtm >>> >> >>> >> startup issues are resolved now. Scripting was a bit painful >>> >> >>> >> because >>> >> >>> >> of these issues. >>> >> >>> >> >>> >> >>> >> Regards, >>> >> >>> >> Nikhils >>> >> >>> >> >>> >> >>> >> > I will take a look at these issue and fix at least in the next >>> >> >>> >> > major >>> >> >>> >> > release. Daemonization should be back-ported to 1.0.x. >>> >> >>> >> > >>> >> >>> >> > Kind Regards; >>> >> >>> >> > ---------- >>> >> >>> >> > Koichi Suzuki >>> >> >>> >> > >>> >> >>> >> > >>> >> >>> >> > 2012/12/27 Michael Meskes <me...@po...>: >>> >> >>> >> >> I still haven't found the time to dig into this, but could >>> >> >>> >> >> anyone please >>> >> >>> >> >> explain to me how gtm is supposed to start up? A simple grep >>> >> >>> >> >> command doesn't >>> >> >>> >> >> seem to find any call to setsid() in gtm subdirs. Could you >>> >> >>> >> >> please point me to >>> >> >>> >> >> the code where gtm daemonizes? Because if it doesn't >>> >> >>> >> >> correctly, >>> >> >>> >> >> there is no >>> >> >>> >> >> suprise gtm_ctl doesn't come back. Or in other words, this is >>> >> >>> >> >> a >>> >> >>> >> >> logical point >>> >> >>> >> >> to start looking. >>> >> >>> >> >> >>> >> >>> >> >> Michael >>> >> >>> >> >> -- >>> >> >>> >> >> Michael Meskes >>> >> >>> >> >> Michael at Fam-Meskes dot De, Michael at Meskes dot >>> >> >>> >> >> (De|Com|Net|Org) >>> >> >>> >> >> Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) >>> >> >>> >> >> dot >>> >> >>> >> >> Org >>> >> >>> >> >> Jabber: michael.meskes at gmail dot com >>> >> >>> >> >> VfL Borussia! Força Barça! Go SF 49ers! Use Debian GNU/Linux, >>> >> >>> >> >> PostgreSQL >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> ------------------------------------------------------------------------------ >>> >> >>> >> >> Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, >>> >> >>> >> >> HTML5, >>> >> >>> >> >> CSS, >>> >> >>> >> >> MVC, Windows 8 Apps, JavaScript and much more. Keep your >>> >> >>> >> >> skills >>> >> >>> >> >> current >>> >> >>> >> >> with LearnDevNow - 3,200 step-by-step video tutorials by >>> >> >>> >> >> Microsoft >>> >> >>> >> >> MVPs and experts. ON SALE this month only -- learn more at: >>> >> >>> >> >> http://p.sf.net/sfu/learnmore_122712 >>> >> >>> >> >> _______________________________________________ >>> >> >>> >> >> Postgres-xc-developers mailing list >>> >> >>> >> >> Pos...@li... >>> >> >>> >> >> >>> >> >>> >> >> >>> >> >>> >> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> >>> >> -- >>> >> >>> >> StormDB - http://www.stormdb.com >>> >> >>> >> The Database Cloud >>> >> >>> >> Postgres-XC Support and Service >>> >> >>> >>> >> >>> >>> >> >>> >>> >> >>> -- >>> >> >>> StormDB - http://www.stormdb.com >>> >> >>> The Database Cloud >>> >> >>> Postgres-XC Support and Service >>> >> >>> >>> >> > >>> >> > >>> >> > >>> >> > -- >>> >> > StormDB - http://www.stormdb.com >>> >> > The Database Cloud >>> >> > Postgres-XC Support and Service >>> >> >>> >> >>> >> >>> >> ------------------------------------------------------------------------------ >>> >> The Go Parallel Website, sponsored by Intel - in partnership with >>> >> Geeknet, >>> >> is your hub for all things parallel software development, from weekly >>> >> thought >>> >> leadership blogs to news, videos, case studies, tutorials, tech docs, >>> >> whitepapers, evaluation guides, and opinion stories. Check out the most >>> >> recent posts - join the conversation now. >>> >> http://goparallel.sourceforge.net/ >>> >> >>> >> _______________________________________________ >>> >> Postgres-xc-developers mailing list >>> >> Pos...@li... >>> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-developers >>> > >>> > >>> > >>> > >>> > -- >>> > Mason Sharp >>> > >>> > >>> > StormDB - http://www.stormdb.com >>> > The Database Cloud >>> > Postgres-XC Support and Services >>> >>> >>> >>> -- >>> StormDB - http://www.stormdb.com >>> The Database Cloud >>> Postgres-XC Support and Service >> >> >> >> >> -- >> Mason Sharp >> >> StormDB - http://www.stormdb.com >> The Database Cloud >> Postgres-XC Support and Services -- StormDB - http://www.stormdb.com The Database Cloud Postgres-XC Support and Service |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X