moosefs-users

[Moosefs-users] chunkserver priority?

[Giovanni T. <gt...@li...>]

Hi,

I was wondering about the ability to configure a priority flag for each 
available chunkserver:

on my setup I have two servers, a master+chunk and a metalogger+chunk, 
and a bunch of desktop client (now ~50, but they will probably increase 
in the future). Each client has a spare disk of 40/80/160 GB, that are 
mostly unused.

What if I use each desktop client as a small chunk server?

The problem is that even with an high number of goal, maybe some chunks 
will be placed on nodes that are currently offline on a particular 
moment of the day (mostly part-time worker here).

If I was able to configure as high priority the 2 big chunkserver and as 
low priority the small chunkservers, I would be pretty sure that at 
least the first two copies of a chunk are always on the servers.

Any comment is greatly appreciated.

Thanks.

-- 
Giovanni Toraldo
http://www.libersoft.it/

Re: [Moosefs-users] chunkserver priority?

[Michal B. <mic...@ge...>]

Hi!

This is quite an interesting suggestion of computers usage but not that easy
to implement. We could think of "auxiliary chunkserver" but we should how to
interpret goal value. Does goal=2 mean that we need to have 2 copies on the
main chunkservers and one extra on the auxiliary one or one at the main and
one at the auxiliary? How would the master know that auxiliary chunkserver
would be permanently unavailable? 

So for now we put it in the section "nice ideas" and we'll come back to it.


Kind regards
Michał 

-----Original Message-----
From: Giovanni Toraldo [mailto:gt...@li...] 
Sent: Friday, February 25, 2011 3:41 PM
To: moo...@li...
Subject: [Moosefs-users] chunkserver priority?

Hi,

I was wondering about the ability to configure a priority flag for each 
available chunkserver:

on my setup I have two servers, a master+chunk and a metalogger+chunk, 
and a bunch of desktop client (now ~50, but they will probably increase 
in the future). Each client has a spare disk of 40/80/160 GB, that are 
mostly unused.

What if I use each desktop client as a small chunk server?

The problem is that even with an high number of goal, maybe some chunks 
will be placed on nodes that are currently offline on a particular 
moment of the day (mostly part-time worker here).

If I was able to configure as high priority the 2 big chunkserver and as 
low priority the small chunkservers, I would be pretty sure that at 
least the first two copies of a chunk are always on the servers.

Any comment is greatly appreciated.

Thanks.

-- 
Giovanni Toraldo
http://www.libersoft.it/

----------------------------------------------------------------------------
--
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT
data 
generated by your applications, servers and devices whether physical,
virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] chunkserver priority?

[Giovanni T. <gt...@li...>]

Il 01/03/2011 14:21, Michal Borychowski ha scritto:
> This is quite an interesting suggestion of computers usage but not that easy
> to implement. We could think of "auxiliary chunkserver" but we should how to
> interpret goal value. Does goal=2 mean that we need to have 2 copies on the
> main chunkservers and one extra on the auxiliary one or one at the main and
> one at the auxiliary? How would the master know that auxiliary chunkserver
> would be permanently unavailable? 

The auxiliary chunkserver should be intended as extra copies for
lowering the I/O and network load on main chunkserver.

I have servers and main switches on Gigabit Ethernet and Fibre Channel,
but desktop clients are 10/100 on 10/100 switches, so the bandwidth is
capped to 10/100 in each room (5-6 clients on each room).

The reason I cannot use *now* the local desktop hd as auxiliary
chunckserver it's because I haven't the assurance that if all the
auxiliary chunckserver are down, all chunks are available on the main
chunkservers.

On the other hand, another solution could be to implement client-side
disk caching, like fs-cache, in the same way nfsv4 do (via cachefilesd).

> So for now we put it in the section "nice ideas" and we'll come back to it.

Thank you Michał.

-- 
Giovanni Toraldo
http://www.libersoft.it/