Re: [Openscrobbler-devel] On Replication

[Mr. D. <mr...@gm...>]

Hm, I guess that the cluster-servers also serving pages makes good 
sense.  So the central server will handle inter-cluster statistics (top 
artist, etc), and the clusters will handle user-level statistics.

On Mar 4, 2005, at 12:44 AM, Jonathan Dance wrote:

> On Mar 4, 2005, at 12:33 AM, Mr. Deep wrote:
>
>> My initial reaction to the replication issue was to simply have pairs 
>> of servers, such that they would constantly update one-another in 
>> case either of them went down, but I decided that was really lame and 
>> we need something better.
>>
>> I think that we need incremental backups (of either raw data or final 
>> statistics) sent to the main server, these should be done on a 
>> regular basis (weekly?) or possibly be done somewhat continuously.  
>> Also, every client should have multiple servers that it tries to 
>> submit data to in case the main one fails.
>
> The biggest difference from my thoughts here is this is a lot more 
> data than I was imagining a central server ever having.
>
>> In general, I think i really need to read the lovely docs that you've 
>> created before I go babbling on like an idiot (specifically, the 
>> chicken w/head cut off variety).
>
> Well... these don't exist. The docs in SVN mostly deal with the code 
> itself: guidelines, how it works, etc etc.
>
>> My main thought at the moment has to do with each client-server 
>> generating an incremental update that is sent to the main sever, 
>> which then applies it, this incremental update tells the main sever 
>> how to update itself to reflect the new submissions that the 
>> client-server has received.
>
> As before... I think the difference we're seeing is that in my head, 
> "clusters" would not only receive/record submissions, but also serve 
> web pages. If each user is assigned to a cluster, then they can get 
> all their detailed, personalized stats from that server. Aggregated 
> stats can be obtained from a central source and served, or the central 
> source can serve them.
>
> There seems to be three paths:
> - The central server would be completely transparent and is purely in 
> the background. No user would use it directly. Requests are served 
> from clusters.
> - The central server does some of the requests, while "clusters" do 
> others.
> - The central server handles all requests
>
>> The main server would store tables, etc in a way that would make 
>> accessing the data as fast as possible.  The rate at which these 
>> incremental updates would be determined by the # of individual song 
>> submissions that have been received.
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real 
> users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Openscrobbler-devel mailing list
> Ope...@li...
> https://lists.sourceforge.net/lists/listinfo/openscrobbler-devel
>