Re: [Moosefs-users] mfs-master[4166]: CS(10.10.10.10) packet too long (226064141/50000000)

[Stas O. <sta...@gm...>]

>
>  Sure ! But the problem is more general than 32bit machines : just catching
> the "no more memory available" error would be great, since it can happen on
> both 32bit and 64bit machines. For example, on our 64 bit machine with a 64
> bit compiled mfsmaster binary, metadata has became such big that mfsmaster
> crashed, and we can't even restore it since it takes too much memory, and
> ends with a segmentation fault :
>
> [root@mfsmaster ~]# mfsmetarestore -a -d /data/MFS/
> loading objects (files,directories,etc.) ... ok
> loading names ... ok
> loading deletion timestamps ... ok
> checking filesystem consistency ... ok
> loading chunks data ... Segmentation fault
> [root@mfsmaster ~]#
>
> [root@mfsmaster ~]# strace mfsmetarestore -a -d /data/MFS/
>   [...]
> read(3,
> "\0\0\0\0\36\347\314\0\0\0\1\0\0\0\0\0\0\0\0\0\35\347\314\0\0\0\1\0\0\0\0\0"...,
> 4096) = 4096
> mmap2(NULL, 561152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
> = -1 ENOMEM (Cannot allocate memory)
> brk(0xb08e8000)                         = 0xffffffffb0844000
> mmap2(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> 0) = -1 ENOMEM (Cannot allocate memory)
> mmap2(NULL, 2097152, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> mmap2(NULL, 1048576, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> mmap2(NULL, 2097152, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> mmap2(NULL, 1048576, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE,
> -1, 0) = -1 ENOMEM (Cannot allocate memory)
> --- SIGSEGV (Segmentation fault) @ 0 (0) ---
> +++ killed by SIGSEGV +++
> [root@mfsmaster ~]#
>
> *[MB] We’ll look into it*
>
>
>
Another suggestion:

Perhaps it's possible to measure the total available memory to MFS master /
logger, and show via chart how much is left?
Similar to how disk space is measured today per chunk servers.

That would allow to plan the memory expansion in advance, and not to be
pressed to locate and add more memory modules when the MFSg master / logger
has crashed (or even normally stopped once this added) due to insufficient
memory.

Regards.