Menu
▾
▴
postgres-xc-general
|
From: Marco G. <mar...@gm...> - 2012-09-06 09:37:38
|
Hey all, I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition? If not so, what is an alternative way to add a coordinator and datanode? Thanks and regards Marco |
|
From: Koichi S. <koi...@gm...> - 2012-09-06 10:00:19
|
Thanks for the question. With XC-1.0.x, we cannot add a node at runtime. This feature is included in the current development and will be available with the next major release. The safest way is to backup the whole database with pg_dump, reinitialize all the nodes including the additional ones, configure the cluster (CREATE NODE and ALTER NODE) and then restore the data with pg_restore. You may need to create each table in advance so that it can be distributed/replicated among all the nodes. New feature of node addition will include: 1) Create new coordinators and datanodes on new servers at runtime, which copies all the necessary catalog such as role, database, etc., from existing components, 2) Add them to the cluster, 3) Reballance tables with new components. It's very helpful if you submit your requirement for this feature. Regards; ---------- Koichi Suzuki 2012/9/6 "Marco Güttler" <mar...@gm...>: > Hey all, > > I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition? > If not so, what is an alternative way to add a coordinator and datanode? > > Thanks and regards > Marco > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general |
|
From: Mephysto <mep...@gm...> - 2012-09-06 10:12:23
|
Hi Koichi, this feature could be very helpful for me too. Is there a forecast for release date of next major release of postgres XC? Thanks in advance. Mephysto On 6 September 2012 12:00, Koichi Suzuki <koi...@gm...> wrote: > Thanks for the question. > > With XC-1.0.x, we cannot add a node at runtime. This feature is > included in the current development and will be available with the > next major release. > > The safest way is to backup the whole database with pg_dump, > reinitialize all the nodes including the additional ones, configure > the cluster (CREATE NODE and ALTER NODE) and then restore the data > with pg_restore. You may need to create each table in advance so > that it can be distributed/replicated among all the nodes. > > New feature of node addition will include: > 1) Create new coordinators and datanodes on new servers at runtime, > which copies all the necessary catalog such as role, database, etc., > from existing components, > 2) Add them to the cluster, > 3) Reballance tables with new components. > > It's very helpful if you submit your requirement for this feature. > > Regards; > ---------- > Koichi Suzuki > > > 2012/9/6 "Marco Güttler" <mar...@gm...>: > > Hey all, > > > > I have a question concerning node addition (coordinator and datanode) at > runtime. Is it possible to add a node at runtime? If so, how to do node > addition? > > If not so, what is an alternative way to add a coordinator and datanode? > > > > Thanks and regards > > Marco > > > > > ------------------------------------------------------------------------------ > > Live Security Virtual Conference > > Exclusive live event will cover all the ways today's security and > > threat landscape has changed and how IT managers can respond. Discussions > > will include endpoint security, mobile security and the latest in malware > > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > > _______________________________________________ > > Postgres-xc-general mailing list > > Pos...@li... > > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > |
|
From: Koichi S. <koi...@gm...> - 2012-09-06 10:14:17
|
Well, next April to May is my wish. ---------- Koichi Suzuki 2012/9/6 Mephysto <mep...@gm...>: > Hi Koichi, > this feature could be very helpful for me too. Is there a forecast for > release date of next major release of postgres XC? > > Thanks in advance. > > Mephysto > > On 6 September 2012 12:00, Koichi Suzuki <koi...@gm...> wrote: >> >> Thanks for the question. >> >> With XC-1.0.x, we cannot add a node at runtime. This feature is >> included in the current development and will be available with the >> next major release. >> >> The safest way is to backup the whole database with pg_dump, >> reinitialize all the nodes including the additional ones, configure >> the cluster (CREATE NODE and ALTER NODE) and then restore the data >> with pg_restore. You may need to create each table in advance so >> that it can be distributed/replicated among all the nodes. >> >> New feature of node addition will include: >> 1) Create new coordinators and datanodes on new servers at runtime, >> which copies all the necessary catalog such as role, database, etc., >> from existing components, >> 2) Add them to the cluster, >> 3) Reballance tables with new components. >> >> It's very helpful if you submit your requirement for this feature. >> >> Regards; >> ---------- >> Koichi Suzuki >> >> >> 2012/9/6 "Marco Güttler" <mar...@gm...>: >> > Hey all, >> > >> > I have a question concerning node addition (coordinator and datanode) at >> > runtime. Is it possible to add a node at runtime? If so, how to do node >> > addition? >> > If not so, what is an alternative way to add a coordinator and datanode? >> > >> > Thanks and regards >> > Marco >> > >> > >> > ------------------------------------------------------------------------------ >> > Live Security Virtual Conference >> > Exclusive live event will cover all the ways today's security and >> > threat landscape has changed and how IT managers can respond. >> > Discussions >> > will include endpoint security, mobile security and the latest in >> > malware >> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> > _______________________________________________ >> > Postgres-xc-general mailing list >> > Pos...@li... >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >> >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > |
|
From: Mason S. <ma...@st...> - 2012-09-06 14:37:50
|
On Thu, Sep 6, 2012 at 6:00 AM, Koichi Suzuki <koi...@gm...> wrote: > Thanks for the question. > > With XC-1.0.x, we cannot add a node at runtime. This feature is > included in the current development and will be available with the > next major release. I may have missed emails discussing the design. Can you please summarize the design? We could have GTM manage node membership. For example, a new candidate data node registers itself as available to GTM (we may want some kind of GTM authentication instead of merely connecting to the ip:port). ADD NODE is executed on one single coordinator, which validates against GTM, updates the status, adds it to the cluster and notifies the other Coordinators. There would no longer be any need for the manual DDL that is done now. This could be expanded in the future to help support multiple read-only replicas for read intensive workloads, where they are identified by distribution segment and registered in a similar fashion. (Instead of just node id, we use segment id, where a node can be associated with a segment, either as the primary or a read-only slave.) > > The safest way is to backup the whole database with pg_dump, > reinitialize all the nodes including the additional ones, configure > the cluster (CREATE NODE and ALTER NODE) and then restore the data > with pg_restore. You may need to create each table in advance so > that it can be distributed/replicated among all the nodes. > > New feature of node addition will include: > 1) Create new coordinators and datanodes on new servers at runtime, > which copies all the necessary catalog such as role, database, etc., > from existing components, > 2) Add them to the cluster, > 3) Reballance tables with new components. > > It's very helpful if you submit your requirement for this feature. > > Regards; > ---------- > Koichi Suzuki > > > 2012/9/6 "Marco Güttler" <mar...@gm...>: >> Hey all, >> >> I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition? >> If not so, what is an alternative way to add a coordinator and datanode? >> >> Thanks and regards >> Marco >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Postgres-xc-general mailing list > Pos...@li... > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general -- Mason Sharp StormDB - http://www.stormdb.com The Database Cloud |
|
From: Koichi S. <koi...@gm...> - 2012-09-06 16:32:46
|
Design discussion has not been made extensively. I just gave my idea to the developer personaly. I'm planning to make more discussion before posting to this ML. It is not just simple and will take a bit more (a week or so). Regards; ---------- Koichi Suzuki 2012/9/6 Mason Sharp <ma...@st...>: > On Thu, Sep 6, 2012 at 6:00 AM, Koichi Suzuki <koi...@gm...> wrote: >> Thanks for the question. >> >> With XC-1.0.x, we cannot add a node at runtime. This feature is >> included in the current development and will be available with the >> next major release. > > I may have missed emails discussing the design. Can you please > summarize the design? > > We could have GTM manage node membership. For example, a new candidate > data node registers itself as available to GTM (we may want some kind > of GTM authentication instead of merely connecting to the ip:port). > ADD NODE is executed on one single coordinator, which validates > against GTM, updates the status, adds it to the cluster and notifies > the other Coordinators. There would no longer be any need for the > manual DDL that is done now. > > This could be expanded in the future to help support multiple > read-only replicas for read intensive workloads, where they are > identified by distribution segment and registered in a similar > fashion. (Instead of just node id, we use segment id, where a node can > be associated with a segment, either as the primary or a read-only > slave.) > >> >> The safest way is to backup the whole database with pg_dump, >> reinitialize all the nodes including the additional ones, configure >> the cluster (CREATE NODE and ALTER NODE) and then restore the data >> with pg_restore. You may need to create each table in advance so >> that it can be distributed/replicated among all the nodes. >> >> New feature of node addition will include: >> 1) Create new coordinators and datanodes on new servers at runtime, >> which copies all the necessary catalog such as role, database, etc., >> from existing components, >> 2) Add them to the cluster, >> 3) Reballance tables with new components. >> >> It's very helpful if you submit your requirement for this feature. >> >> Regards; >> ---------- >> Koichi Suzuki >> >> >> 2012/9/6 "Marco Güttler" <mar...@gm...>: >>> Hey all, >>> >>> I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition? >>> If not so, what is an alternative way to add a coordinator and datanode? >>> >>> Thanks and regards >>> Marco >>> >>> ------------------------------------------------------------------------------ >>> Live Security Virtual Conference >>> Exclusive live event will cover all the ways today's security and >>> threat landscape has changed and how IT managers can respond. Discussions >>> will include endpoint security, mobile security and the latest in malware >>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >>> _______________________________________________ >>> Postgres-xc-general mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. Discussions >> will include endpoint security, mobile security and the latest in malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> _______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general > > > > -- > Mason Sharp > > StormDB - http://www.stormdb.com > The Database Cloud |
|
From: Mason S. <ma...@st...> - 2012-09-07 12:05:58
|
On Thu, Sep 6, 2012 at 12:32 PM, Koichi Suzuki <koi...@gm...> wrote: > Design discussion has not been made extensively. I just gave my idea > to the developer personaly. I'm planning to make more discussion > before posting to this ML. It is not just simple and will take a > bit more (a week or so). OK, thanks, I understand. In my previous email I was initially just focusing on the node registration part, and not the rebalancing part. > > Regards; > ---------- > Koichi Suzuki > > > 2012/9/6 Mason Sharp <ma...@st...>: >> On Thu, Sep 6, 2012 at 6:00 AM, Koichi Suzuki <koi...@gm...> wrote: >>> Thanks for the question. >>> >>> With XC-1.0.x, we cannot add a node at runtime. This feature is >>> included in the current development and will be available with the >>> next major release. >> >> I may have missed emails discussing the design. Can you please >> summarize the design? >> >> We could have GTM manage node membership. For example, a new candidate >> data node registers itself as available to GTM (we may want some kind >> of GTM authentication instead of merely connecting to the ip:port). >> ADD NODE is executed on one single coordinator, which validates >> against GTM, updates the status, adds it to the cluster and notifies >> the other Coordinators. There would no longer be any need for the >> manual DDL that is done now. >> >> This could be expanded in the future to help support multiple >> read-only replicas for read intensive workloads, where they are >> identified by distribution segment and registered in a similar >> fashion. (Instead of just node id, we use segment id, where a node can >> be associated with a segment, either as the primary or a read-only >> slave.) >> >>> >>> The safest way is to backup the whole database with pg_dump, >>> reinitialize all the nodes including the additional ones, configure >>> the cluster (CREATE NODE and ALTER NODE) and then restore the data >>> with pg_restore. You may need to create each table in advance so >>> that it can be distributed/replicated among all the nodes. >>> >>> New feature of node addition will include: >>> 1) Create new coordinators and datanodes on new servers at runtime, >>> which copies all the necessary catalog such as role, database, etc., >>> from existing components, >>> 2) Add them to the cluster, >>> 3) Reballance tables with new components. >>> >>> It's very helpful if you submit your requirement for this feature. >>> >>> Regards; >>> ---------- >>> Koichi Suzuki >>> >>> >>> 2012/9/6 "Marco Güttler" <mar...@gm...>: >>>> Hey all, >>>> >>>> I have a question concerning node addition (coordinator and datanode) at runtime. Is it possible to add a node at runtime? If so, how to do node addition? >>>> If not so, what is an alternative way to add a coordinator and datanode? >>>> >>>> Thanks and regards >>>> Marco >>>> >>>> ------------------------------------------------------------------------------ >>>> Live Security Virtual Conference >>>> Exclusive live event will cover all the ways today's security and >>>> threat landscape has changed and how IT managers can respond. Discussions >>>> will include endpoint security, mobile security and the latest in malware >>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >>>> _______________________________________________ >>>> Postgres-xc-general mailing list >>>> Pos...@li... >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >>> >>> ------------------------------------------------------------------------------ >>> Live Security Virtual Conference >>> Exclusive live event will cover all the ways today's security and >>> threat landscape has changed and how IT managers can respond. Discussions >>> will include endpoint security, mobile security and the latest in malware >>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >>> _______________________________________________ >>> Postgres-xc-general mailing list >>> Pos...@li... >>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >> >> >> >> -- >> Mason Sharp >> >> StormDB - http://www.stormdb.com >> The Database Cloud -- Mason Sharp StormDB - http://www.stormdb.com The Database Cloud |
Thanks for helping keep SourceForge clean.
X