Re: [Spock Proxy Devel] Different Shards on the same server

[steve <iam...@gm...>]

>> I'm looking at possibly moving to AWS/EC2, and was looking at scalr
>> (http://code.google.com/p/scalr/) as a way to manage the spinning up
>> of and auto-managing instances. It would be awesome if Spock Proxy
>> were directly supported there.
>
> Indeed - we also use a lot of EC2 and have discussed things like this.  But
> so far just talk.  If we do anything I'll let you know.

Great. I just found the scalr project this week, and seems to be one
of the only ones that is open source (I don't think RightScale is, for
example).

I would like to have shard migration from server to server, including
having a shard run on a server already serving a shard. For either the
mast or slave. I'm sure an algorithm could be created that did
rebalancing, and could be plugged into scalr or similar to include
creating (or terminating) new servers if the rebalancing required it.

> Very  interesting, we have  discussed doing something similar but simpler
> just one  master and one slave on a single server.  The issue is we use
> replication for the universal data and  you don't want  to get into a spot
> where you can't replicate (because MySQL will only support one  master - at
> least for now).  Also in out case we want the master  DB to have all the
>  machine resources (memory, disk IO, ...).

I hear you on memory... but on IO I think that you find that you can
ditch hardware raid for software mirrors and multiple mysql instances
pointing to the different mirrors -- for orthogonal data. I don't know
enough about EC2 to know how things will perform there though...

But it is also helpful when mysql does not scale well to a lot of
cores -- you can use CPU affinity for the master process (say 4 of 8
cores) and have four other mysql slaves running on the other 4 cores.
This works well if you like many replicas of each shard.