Menu

#96 sarra does not copy some files that it should.

SarraBeta
closed-fixed
psilva
None
1
2018-06-02
2018-01-15
psilva
No

When sr_sarra (or sr_subscribe) downloads a file, it checks locally if the file exists, and is different from the one to download. Sometimes, the program looks in the directory of the last file downloaded, rather than in the directory where the new file will be downloaded.

If it happens that the previous directory contained a file with the same name and content, sarracenia will decide not to download the data to the new location. This sounds rare but is actually common in some uses cases:

  • when multiple clients download the same data to different directories. (duplicated feeds for transition, such as RADAR.)
  • when many directory trees have files with the same name and content for tracking purposes (Maestro jobs.)

This issue was identified and resolve while working on the HPC mirroring. it is addressed by commit
79d5c62ddaf68132495a194646162475c4a17d34 on January 5th, and included in 2.18.01a2 release. The code was fixed to use the full path instead of just the file name.

The check remains open because a number of systems are still affected by this bug, and no version with the fix has made it to operations yet.

Discussion

  • psilva

    psilva - 2018-04-13

    This is waiting for the new Michel stream (post 2.17.07) to be deployed everywhere... at least 2.18.04 needs to get to ddsr.

     
  • psilva

    psilva - 2018-06-02
    • status: open --> closed-fixed
     

Log in to post a comment.

MongoDB Logo MongoDB