#70 Full transparent MiG home access in jobs with sshfs

[Anonymous]

Originally created by: jonas.ba...@gmail.com
Originally owned by: jonas.ba...@gmail.com

To whom is this feature relevant?
-users with jobs using many or big files

Please provide a one line description of the feature:
-MiG user home mounted in job dir for full access

How would the new feature work (appear in use)?
the MiG user home would be available as just a sub-directory in the job directory during job execution.

How would the new feature be implemented?
The job hand out would include generation of a job specific ssh-key without passphrase used to temporarily mount the user home inside the job dir on the resource.

Please provide any additional information below.
The generated pub key will have to be temporarily inserted in the user dictionary of our grid sftp server or a separate sftp instance for jobs only. The private key must then be securely pushed to the resource home so that the job script can use it to sshfs mount the user home and possible delete the key before the job kicks in. Upon job exit/timeout the directory must be unmounted.
Basically this is an alternative implementation of our transparent remote file access model but without any extra daemons or ports and with caching moved to the OS. It obviously requires sshfs on the resource and outbound access to the sftp server of the MiG server.
Please note that the user can manually create the exact same setup explicitly already, but we could make it simpler and transparent.

[Anonymous]

Originally posted by: jonas.ba...@gmail.com

Additional notes based on talk with Martin:
One drawback of full mount is that it potentially makes it easier for resource owners to snoop on user files. With the current setup or the Transparent Remote File Access model they would have to intercept a job and guess file names.
Since we use our own sftp server we could perhaps just mark the job keys and restrict sftp access with those keys to only open/read/write/close and get the same protection.

Another way to limit snooping could be to only mount dirs listed in INPUTFILES.

We could force unmount on the server side as a result of job finish/timeout/kill.

Finally we could modify TRFA to use sftp instead of https and deploy it without the need for a separate https server. It requires a C implementation of sftp with keys though.