Re: [Moosefs-users] process planing

[Michal B. <mic...@ge...>]

Hi Josef!

Thanks for your interest in MooseFS! 

Our experience tells us that the best algorithm for this is just: "random".
Things like "load" or "network bandwidth" may change very quickly and basing
on these facts to choose chunkserver may bring the opposite result. But of
course if you want to experiment we'd gladly hear your observations.

Creating new chunk:

At the beginning of "chunk_multi_modify" function there is: 
servcount = matocsserv_getservers_wrandom(ptrs,goal);

And "matocsserv_getservers_wrandom" function writes to "ptrs" array servers
on which the new chunk should be created and returns its quantity. Parameter
"goal" tells how many servers should be inserted to the array. In reply
function return number of servers, typically it should be the value equal to
the "goal" parameter but if there is too few servers available, the value
would be lower and array "ptrs" would contain all the servers.


Servers for clients are returned in order by "chunk_getversionandlocations"
function. Generally speaking this function returns random servers and one
exception is returning IP of the client server at the first position (if the
client is also a chunk). In the future we would add here a more complicated
algorithm to take into account "rack/switch awareness". For the read process
the clients uses the first server and for the write sets the chain in the
given order.

While reading the client can by itself change the server for the next one.
The client checks how many operations it has with the chunkserver and may
choose the one with the smallest number of operations. 


I hope it helps and you can have some interesting experiments with the code.


If you need any further assistance please let us know.

Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01





-----Original Message-----
From: Josef [mailto:pe...@p-...] 
Sent: Monday, December 13, 2010 2:02 PM
To: moo...@li...
Subject: [Moosefs-users] process planing

  Hello,
     I've been studying moosefs source codes to find a planing 
mechanism, where to place new chunks, but due to a lack of comments it's 
quite difficult. Would someone recommand me, where to search? The same 
would interest me for reading, if there is more copies which one the 
client chooses to read from.
     Ideal planning mechanism should take into account factors as 
chunkserver load, disk usage, avilable network bandwidth and so on. My 
topic of disertation thesis at the CTU in Prague is process planing in 
distributed systems, so if there is a lack of such mechanism, it would 
interest me to help.

Josef

----------------------------------------------------------------------------
--
Oracle to DB2 Conversion Guide: Learn learn about native support for PL/SQL,
new data types, scalar functions, improved concurrency, built-in packages, 
OCI, SQL*Plus, data movement tools, best practices and more.
http://p.sf.net/sfu/oracle-sfdev2dev 
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users