[MooseFS-Users] Very odd issue starting master (community version)

[Alex C. <ac...@in...>]

Hi,

A really weird issue. mfsmaster is not starting, complaining that 
metadata.mfs does not exist, when it actually does. Note that the 
machine was forcibly rebooted just before this started.

[root@glenrock ~]# mfsmaster start
open files limit has been set to: 16384
working directory: /var/lib/mfs
lockfile created and locked
initializing mfsmaster modules ...
exports file has been loaded
topology file has been loaded
loading metadata ...
loading sessions data ... ok (0.0000)
loading label data ... ok (0.0000)
loading objects (files,directories,etc.) ... ok (0.1155)
loading names ... ok (0.2793)
loading deletion timestamps ... ok (0.0000)
loading quota definitions ... ok (0.0000)
loading xattr data ... ok (0.0000)
loading posix_acl data ... ok (0.0289)
loading open files data ... ok (0.0000)
loading flock_locks data ... ok (0.0000)
loading posix_locks data ... loading posix_locks: lock on closed file !!!
cleaning metadata ...
cleaning objects ... done
cleaning names ... done
cleaning deletion timestamps ... done
cleaning quota definitions ... done
cleaning chunks data ...done
cleaning xattr data ...done
cleaning posix_acl data ...done
cleaning flock locks data ...done
cleaning posix locks data ...done
cleaning chunkservers data ...done
cleaning open files data ...done
cleaning sessions data ...done
cleaning label sets data ...done
metadata have been cleaned
error loading metadata.mfs - try using option '-a'
init: metadata manager failed !!!
error occured during initialization - exiting


[root@glenrock ~]# mfsmaster -a
open files limit has been set to: 16384
working directory: /var/lib/mfs
lockfile created and locked
initializing mfsmaster modules ...
exports file has been loaded
topology file has been loaded
loading metadata ...
loading sessions data ... ok (0.0000)
loading label data ... ok (0.0000)
loading objects (files,directories,etc.) ... ok (0.1160)
loading names ... ok (0.2792)
loading deletion timestamps ... ok (0.0000)
loading quota definitions ... ok (0.0000)
loading xattr data ... ok (0.0000)
loading posix_acl data ... ok (0.0291)
loading open files data ... ok (0.0000)
loading flock_locks data ... ok (0.0000)
loading posix_locks data ... loading posix_locks: lock on closed file !!!
cleaning metadata ...
cleaning objects ... done
cleaning names ... done
cleaning deletion timestamps ... done
cleaning quota definitions ... done
cleaning chunks data ...done
cleaning xattr data ...done
cleaning posix_acl data ...done
cleaning flock locks data ...done
cleaning posix locks data ...done
cleaning chunkservers data ...done
cleaning open files data ...done
cleaning sessions data ...done
cleaning label sets data ...done
metadata have been cleaned
error loading metadata file (metadata.mfs): ENOENT (No such file or 
directory)
init: metadata manager failed !!!
error occured during initialization - exiting

[root@glenrock ~]# ls -l /var/lib/mfs/metadata.mfs
-rw-r----- 1 mfs mfs 65582156 Apr 15 11:23 /var/lib/mfs/metadata.mfs

Can anyone help as this cluster was running fine before and we need to 
have it working again by Monday.

We do have several metaloggers that have been running the whole time if 
that helps.

Best regards

Alex

--
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
This email is not intended to, nor should it be taken to, constitute advice.
The information provided is correct to our knowledge & belief and must not
be used as a substitute for obtaining tax, regulatory, investment, legal or
any other appropriate advice.

"Transact" is operated by Integrated Financial Arrangements Ltd.
29 Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608 5300.
(Registered office: as above; Registered in England and Wales under
number: 3727592). Authorised and regulated by the Financial Conduct
Authority (entered on the Financial Services Register; no. 190856).