Re: [Moosefs-users] CGI curiosity

[Markus K. <mar...@tu...>]

On Monday 09 April 2012, Steve Wilson wrote:
> On 04/09/2012 02:51 PM, Steve Thompson wrote:
> > mfs 1.6.20
> >
> > I have marked a disk for removal with * in the mfshdd.cfg file. There are
> > approximately 1.6 million chunks on this disk, and so far about 1.3
> > million chunks have been replicated elsewhere. All files have goal = 2.
> >
> > When viewing the CGI with Firefox on a Windows box, it shows 1.3 million
> > in blue on the valid copies = 3 column of the goal = 2 line and nothing
> > in the valid copies = 1 line. This number is increasing.
> >
> > When viewing the CGI with Firefox on a Linux box, it shows 300,000 in
> > orange on the valid copies = 1 column of the goal = 2 line, and nothing
> > in the valid copies = 3 line. This number is decreasing.
> >
> > The CGI is running on the mfsmaster in both cases. Why the difference?
> >
> > BTW, this has taken 5 days so far. A little slow, methinks.
> >
> > Steve
>
> Regarding the speed issue, have you modified the default
> CHUNKS_WRITE_REP_LIMIT and CHUNKS_READ_REP_LIMIT in mfsmaster.cfg?
> Following suggestions on the list, I have mine permanently set to:
>      CHUNKS_WRITE_REP_LIMIT = 5
>      CHUNKS_READ_REP_LIMIT = 15
> instead of the default:
>      CHUNKS_WRITE_REP_LIMIT = 1
>      CHUNKS_READ_REP_LIMIT = 5


I tried this settings today and it worked very good. Thanks for sharing your 
configuration.


For me today a similar question followed up:
How to find out if all chunks are migrated from a chunk server?

At the moment I can see:

chunk server marked for removal: 
chunks=4545
All chunks state matrix 'regular':
goal=2: 1=1, 2=8028
All chunks state matrix 'all':
goal=2: 1=1, 2=3483, 3=4545

So I know that I can remove the chunk server now because chunks state 
matrix 'all' says I have a overgoal of 4545 which is exactly the same number 
of chunks on my chunk server. 

At the moment I only run some test with goal=2. In production we will have 
different goals for different types of data. My plan is to use all desktop 
and laboratory hosts in our environment as a chunk server. Ones a year we 
need to reboot our laboratory hosts into windows for about two weeks. Which 
means I have to mark all partitions on this chunk servers for removal at the 
same time. Which I guess will make it hard to say which one is ready to 
reboot. At least not before all chunks of all chunk server are migrated.

It would be very nice to see the status at the disk status table. At the 
moment I only can see status 'ok' or 'marked for removal'.
Nice would be see something like: 'marked for removal, migration in progress' 
and 'marked for removal, migration finished'


Markus
-- 
Markus Köberl
Graz University of Technology
Signal Processing and Speech Communication Laboratory
E-mail: mar...@tu...