Menu
▾
▴
#45 Splitting DATAPATH
The need for a slash at the end of datapath causes immense frustrations in beginners. Also, the impossibility of knowing if DATAPATH is a directory or directory+prefix causes all sorts of problems and lack of sanity checks and directory permission checks and informative error messages. DATAPATH should be split into a M8R_DPATH_DIR and M8R_DPATH_PREFIX. (or use RSF_ instead of M8R_ if it makes more sense) The latter would be very useful for eliminating binary filenames aliasing problems, as well as for selective cleanup in datapath.
Discussion
Anonymous
I dont think it should be too hard to make it so that DATAPATH checks for the existence of a directory...
I agree, that we should change the name of DATAPATH to RSF_DATAPATH. DATAPATH is common enough that we could inadvertantly cause issues for users of other packages.
DATAPATH has a huge backwards compatibility issue, because people set it in their $HOME/.bashrc, and we cannot push a change through the env.sh script, as we would with another variable. There is also the issue that SEPlib uses DATAPATH as well, as do other programs written elsewhere. So regardless of the implementation mechanism, we will have to support both solutions, and maintain backwards compatibility for something like ten years, and gently persuade people of the benefits of the new approach.
Speaking of benefits -- the great benefit that I see is in having the RSF_DATAPATH_PREFIX standalone variable. That is what would guarantee elimination of data aliasing forever in an easy fashion, because the user can set it at the beginning of each project, or it can be generated from the basepath of the header (i.e. for /absolute/path/to/file.rsf , the RSF_DATAPATH_PREFIX can be by default "absolute.path.to"). Also -- the selective cleanup when the disks fill up!
IDEA: Actually, to make our life easy, we do not even need to replace the current mechanism. We can leave DATAPATH for the time being as it is, and if RSF_DATAPATH_PREFIX is set, append it to DATAPATH. Then we can start advertising so people start using this mechanism, and after two or three years, when all people inclined to set prefixes for their datapaths have shifted to the new mechanism, we can start assuming in the codebase that DATAPATH is a directory, and every time someone raises an error, direct them to use the new mechanism.