[Moosefs-users] Different disk space taken with 2 chunk servers and goal=2

[Flow J. <fl...@gm...>]

Hi,

We've been using moosefs in production environment and now we have about 
4 million files stored. Here's the information reported by the CGI script:

version     total space     avail space     trash space     trash files 
     reserved space     reserved files     all fs objects     
directories     files     chunks     all chunk copies     regular chunk 
copies
1.6.19     974 GiB     537 GiB     0 B     0     0 B     0     4602256 
     331677     4027193     4002874     8005748     8005748

Everything looks good except for the space taken on 2 chunk servers, 
they are not equal:

mfschunkserver1    13.187.243.149    9422    1.6.19     4002874    257 
GiB    523 GiB
mfschunkserver2    13.187.243.153    9422    1.6.19     4002874    180 
GiB    451 GiB

We have set goal=2 for all the files so chunks should have been mirrored 
on these 2 servers, right? But why we still have different disk space 
taken on these servers?

Here's the disk layout for these 2 server:

1    13.187.243.149:9422:/mnt/mfs_chunk1/    3532242    no errors    ok 
     -    -    -    -    -    -    -    -     204 GiB    229 GiB
2    13.187.243.149:9422:/mnt/mfs_chunk3/    470632    no errors    ok 
     -    -    -    -    -    -    -    -     53 GiB    293 GiB
3    13.187.243.153:9422:/mnt/mfs_chunk1/    2002902    no errors    ok 
     -    -    -    -    -    -    -    -     90 GiB    226 GiB
4    13.187.243.153:9422:/mnt/mfs_chunk2/    1999972    no errors    ok 
     -    -    -    -    -    -    -    -     90 GiB    226 GiB

Most of the chunks stored on 13.187.243.149 were replicated from another 
server which has been removed from the cluster. The disk on that server 
was set to "marked for removal" and we waited until the replication 
finished.

Can anyone help to explain if this is normal and can we make the space 
taken by the 2 servers equal?

Thanks
Flow