Re: [MooseFS-Users] MooseFS and encryption?

[Marin B. <li...@ol...>]

Hi,  
  
 I would really love to see TLS or Kerberos support added to MooseFS, and md5   
 password hashes replaced by proper HMAC. The lack of such features makes   
 MooseFS vulnerable to several classes of exploits, mostly similar to   
 those affecting unencrypted NFS. As of now, using MooseFS on unsecure /   
 public networks is fairly dangerous.  
  
 Here is what we did to help securing the whole cluster:  
  
 - We emulate TLS with IPsec. We encrypt MooseFS traffic with IPsec between   
 all members of the cluster (including clients). Masters and chunk servers   
 are on the same subnet and use IPsec in transport mode, while clients use   
 tunnel mode through our gateways, even internally. The overhead is   
 minimal with recent CPUs and NICs.  
  
 - We use certificates to authenticate IPsec peers. This allows us to easily   
 revoke the access of a compromised system by revoking its certificate.  
  
 - We use secrets to authenticate masters and chunk servers. It's not secure,   
 but doesn't harm.  
  
 - Out of the box, MooseFS relies on DNS to locate the cluster master by   
 querying the 'mfsmaster' host name. This makes the cluster vulnerable to   
 DDoS and DNS spoofing. We use DNSSEC to authenticate DNS answers.  
  
 - We encrypt metadata (master / metaloggers) and data (chunk servers) with OS   
 level encryption. We use GELI on FreeBSD.  
  
 - Sensitive information may be further encrypted on client-side. We use PEFS   
 on FreeBSD, which is able to encrypt selected files or folders.  
  
 - Since user authentication happens on client-side, a smart user with   
 administrative access may use bogus UID/GID to bypass POSIX file system   
 permissions. Kerberos authentication would not let such a scenario   
 happen, but since it is not an option, we decided to consider file system   
 permissions as unsecure and not to rely on them. Instead, we handle   
 access control with exports: we use one export per shared directory and   
 ACL. Each export is protected by a md5 hash and we rely on the export   
 password to authenticate remote systems. Each export line includes a   
 'mapall' option which maps any remote UIDs/GIDs to an unprivilegied local   
 user/group whose permissions are limited to the shared location only.   
 This configuration protects the cluster from most exploits, except if the   
 export password is leaked.  
  
 Hope that helps,  
  
 Marin.