[MooseFS-Users] mfsmaster won't start - loading edge: 2502428->5211944 error: empty name

[Ricardo J. B. <ric...@do...>]

Hi all,

My mfsmaster crashed crashed today and when trying to start it I get this error:

loading edge: 2502428->5211944 error: empty name


I tried with metadata.mfs.back, metadata.mfs.back.1 from master, and also
metadata_ml.mfs.back and metadata_ml.mfs.back.1 from metalogger, all of them
fail with the same error.


My mfsmaster was 3.0.100 when it crashed, I upgraded to 3.0.114 but it's the same:

# mfsmaster -v
version: 3.0.114-1 ; build: 1257


Any hints to solve this much appreciated, as this is a production cluster.


I have dumped all the metadata files and all of them have this:

# mfsmetadump metadata.mfs.back > metadata.mfs.dump
# grep 2502428 metadata.mfs.dump | grep -1 5211944
EDGE|p:   2502428|c:   5154814|i:0x7FFFFFFFF92D4A30|n:1519914956.H419257P2952.c106-dr.dattaweb.com:2,S
EDGE|p:   2502428|c:   5211944|i:0x000000095F864814|n:
EDGE|p:   2502428|c:   2920292|i:0x7FFFFFFFF947F4F2|n:1519809696.H296554P8952.c105-dr.dattaweb.com:2,S


Full output of 'mfsmaster -xx -a -i -c /etc/mfs/mfsmaster.cfg':

open files limit has been set to: 16384
working directory: /mnt/mailmfs/mfs
lockfile created and locked
initializing mfsmaster modules ...
exports file has been loaded
topology file has been loaded
write replication limit in old format - change limits to new format
read replication limit in old format - change limits to new format
loading metadata ...
found valid metadata file: metadata.mfs.back.1 (version: 16411110888 ; id: 59AEE396E5130F20)
found invalid metadata file (wrong header): metadata.crc
found valid metadata file: metadata.mfs.back (version: 16412023641 ; id: 59AEE396E5130F20)
chosen most recent metadata file: metadata.mfs.back (version: 16412023641 ; id: 59AEE396E5130F20)
loading sessions data ... ok (0.0000)
loading storage classes data ... ok (0.0000)
loading objects (files,directories,etc.) ... ok (1.1999)
loading names ... 
loading edge: 2502428->5211944 error: empty name
cleaning metadata ...
cleaning objects ... done
cleaning names ... done
cleaning deletion timestamps ... done
cleaning quota definitions ... done
cleaning chunks data ...done
cleaning xattr data ...done
cleaning posix_acl data ...done
cleaning flock locks data ...done
cleaning posix locks data ...done
cleaning chunkservers data ...done
cleaning open files data ...done
cleaning sessions data ...done
cleaning storage classes data ...done
cleaning dictionary data ...done
metadata have been cleaned
error loading metadata file (metadata.mfs.back): ENOENT (No such file or directory)
init: metadata manager failed !!!
error occurred during initialization - exiting


-- 
Ricardo J. Barberis
Senior SysAdmin / IT Architect
DonWeb
La Actitud Es Todo
www.DonWeb.com
  _____