Re: [Openscrobbler-devel] On Replication

[Mr. D. <mr...@gm...>]

Err, replied to wrong place ...

On Mar 4, 2005, at 12:33 AM, Mr. Deep wrote:

> My initial reaction to the replication issue was to simply have pairs 
> of servers, such that they would constantly update one-another in case 
> either of them went down, but I decided that was really lame and we 
> need something better.
>
> I think that we need incremental backups (of either raw data or final 
> statistics) sent to the main server, these should be done on a regular 
> basis (weekly?) or possibly be done somewhat continuously.  Also, 
> every client should have multiple servers that it tries to submit data 
> to in case the main one fails.
>
> In general, I think i really need to read the lovely docs that you've 
> created before I go babbling on like an idiot (specifically, the 
> chicken w/head cut off variety).  My main thought at the moment has to 
> do with each client-server generating an incremental update that is 
> sent to the main sever, which then applies it, this incremental update 
> tells the main sever how to update itself to reflect the new 
> submissions that the client-server has received.  The main server 
> would store tables, etc in a way that would make accessing the data as 
> fast as possible.  The rate at which these incremental updates would 
> be determined by the # of individual song submissions that have been 
> received.
>
> On Mar 3, 2005, at 12:05 AM, Jonathan Dance wrote:
>
>> Another serious issue we face is that of replication. This really has 
>> two facets:
>>
>> Backups: what if a server has a hardware failure, or, for one reason 
>> or another, disappears forever? It would be best if the user data is 
>> (also) stored somewhere else.
>>
>> Temporary failure: what if a server goes down temporarily? Ideally, 
>> the system should be able to handle this situation as well.
>>
>> I don't have many good answers for these. Some possible answers 
>> (which does not necessarily cover both facets):
>> - Each server sends us a backup of itself every X time period.
>> - Each server replicates itself (somehow) to another server. In case 
>> the first server fails, the second server starts serving the users of 
>> the first server, in addition to its own. If the first server comes 
>> back, the second server stops serving those users. If the server 
>> never comes back, the second server moves some users somewhere else. 
>> (This is basically some kind of dynamic cluster-to-cluster 
>> user-handling system, where each system pushes users around as 
>> necessary. Also, the data is always in at least two places.)
>>
>> Lots of fun stuff to think about!
>>
>> -------------------
>> There was a significant typo in my last e-mail:
>>
>> This perfect for "clustering" where each server is responsible for 
>> any number of servers. => This is perfect for "clustering" where each 
>> server is responsible for any number of users.
>> -------------------
>>
>> --JD
>>
>>
>>
>> -------------------------------------------------------
>> SF email is sponsored by - The IT Product Guide
>> Read honest & candid reviews on hundreds of IT Products from real 
>> users.
>> Discover which products truly live up to the hype. Start reading now.
>> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
>> _______________________________________________
>> Openscrobbler-devel mailing list
>> Ope...@li...
>> https://lists.sourceforge.net/lists/listinfo/openscrobbler-devel
>>
>