Re: [Postgres-xc-general] Our general use case

[Vladimir S. <vst...@gm...>]

On Fri, Oct 26, 2012 at 12:36:25PM -0700, Roger Mayes wrote:

>    they ever were to, I can press a button and have another server up,
>    of the exact same configuration, in minutes flat, restored,
>    potentially, to an image created the previous night.  We can

This is good for standalone system, but with cluster those images of all
nodes should be synchronized. 

More over cluster inside virtual machines is something exotic. If all
Your nodes are running on the same hardware host, what for You need
cluster?  Run standalone system without virtual machine and You've got
more capacity. Or simply pay for more resources for single virtual
machine. The same if Your XC nodes are running on different hardware
nodes: You will use small part of its resources for which You have paid.
What for do You need virtual machines there? They are needed for Your
provider to share resources with his customers, but not for You. 

I am running XC on virtual machines but for testing and debugging only.

>    HA is important for us, scalability is definitely the more pressing
>    matter:  Without HA, we might someday go out of business - without
>    some claim to scalability, we can't get into business to begin
>    with.

I really enjoy with Your maxim. Your philosophy is applicable for
everything existing in this wonderful world. We can't loose something we
don't have. So first we want to have, then we don't want to loose. And
it is not about priorities, it is about "be or not to be". There even
nice song exists about this. But what You wrote below proves: You need
HA first of all.

>    Can you do log shipping, hot backups, and recover a cluster to a
>    point in time?  If not what is the quickest/best backup/recovery
>    procedure?  Whatever it is, it is something I'll need to get
>    scripted and working (I mean I'll write

logs should be handled on every node, it is not so simple.

>    Can you (1) do a full dump, then (2) kill, drop and rebuild the
>    cluster, and then (3) restore the entire cluster using pg_restore
>    (or psql .. < dumpfile) through a coordinator?  This would be a

Yes, without this it become inviable. The only question is about size of
data base and level of load. Depend on this backup procedure may impact
production operation. I am doing such things piping dump directly to
network without any temporary files on intermidiate storage.

>    found myself doing things with dump files, like pulling pieces of
>    them into temporary tables and then deleting and/or updating rows
>    in existing tables based on the data in the dump-loaded temporary
>    tables.  I'm imagining that these general recovery scenarios are
>    not particularly complicated by the fact that I'm working with an
>    xc cluster - as long as I'm going through a coordinator, the
>    effects of the sql statements should replicate or distribute across
>    the cluster depending on how the table was set up.. right?

You are right. This is usual data manipulation with sql and if XC claims
it is transparent, as long as it is true it should working.

>    I can probably create a temporary table on a single datanode,
>    through a coordinator, just by telling it to distribute that table,
>    and only list the one datanode I want it on, right?  Then I can do

No difference, sure.

>    Hmm, so I wonder what I actually would do if a datanode went down,
>    or if the gtm server went down.  Obviously I wouldn't want to lose
>    all the data in the other nodes.  I wonder how complicated it would
>    be and how long it would take

At this point HA comes to help. But it is not alternative for backup. We
should do usual backup always.

>    to get things back up and running again.  I guess I'd better
>    familiarize myself with the docs on backup and recovery.
>    Are they up to date with pgxc 1.0.1?

Yes, You can see this on top of every page. But name of link pointing to
it is wrong.


***************************
###  Vladimir Stavrinov
###  vst...@gm...
***************************