Menu

#66 Write the renamed filename in the sr_sender log

SarraBeta
closed
psilva
None
5
2017-07-29
2016-11-25
No

Hi,
When we are using a DESTFNSCRIPT to rename a file in sr_sender, the new name is not written in the log unless we are in loglevel debug.

In examples below, the DESTFNSCRIPT is replacing WUL by EC if the filename contain AQ:WUL:AIRNOW:AQCSV

example withou debug:
2016-11-25 06:31:16,616 [INFO] msg_log received: 20161125063116.77 http://ddsr3.cmc.ec.gc.ca/ 20161125/MSC-ECONET/AIRNOW/20161125_0631_aq.txt:AQ:WUL:AIRNOW:AQCSV topic=v02.post.20161125.MSC-ECONET.AIRNOW lag=0.539445 mtime=20161125063116.77 parts=1,26452,1,0,0 sum=d,a31b57b4699c8108d7dd3882092b4c8f from_cluster=DDSR.CMC source=MSC-ECONET to_clusters=DDSR.CMC,DDI.CMC,CMC,SCIENCE,EDM
2016-11-25 06:31:17,106 [INFO] Sends: 20161125_0631_aq.txt:AQ:WUL:AIRNOW:AQCSV into //home/can000/project/ade/indata/raw/airnow/20161125_0631_aq.txt:AQ:WUL:AIRNOW:AQCSV 0-26451

example in debug mode:
sr_sender_cmc-ade-airnow_0001.log:2016-11-25 14:48:08,136 [DEBUG] sending/copying /apps/sarra/public_data/20161125/MSC-ECONET/AIRNOW/20161125_1431_aq.txt:AQ:WUL:AIRNOW:AQCSV
sr_sender_cmc-ade-airnow_0001.log:2016-11-25 14:48:08,501 [INFO] Sends: 20161125_1431_aq.txt:AQ:WUL:AIRNOW:AQCSV into //home/can000/project/ade/indata/raw/airnow/20161125_1431_aq.txt:AQ:WUL:AIRNOW:AQCSV 0-26463
sr_sender_cmc-ade-airnow_0001.log:2016-11-25 14:48:08,502 [DEBUG] sr_sftp put /apps/sarra/public_data/20161125/MSC-ECONET/AIRNOW/20161125_1431_aq.txt:AQ:WUL:AIRNOW:AQCSV 20161125_1431_aq.txt:AQ:EC:AIRNOW:AQCSV.tmp 0 0 0
sr_sender_cmc-ade-airnow_0001.log:2016-11-25 14:48:08,515 [DEBUG] sr_sftp rename 20161125_1431_aq.txt:AQ:EC:AIRNOW:AQCSV.tmp 20161125_1431_aq.txt:AQ:EC:AIRNOW:AQCSV

We need to have the infos (renamed filename) in the log.

Thanks

Lucie

Discussion

  • psilva

    psilva - 2016-11-26

    This bug should already be fixed in 11a4. If you are running an older version, please confirm byt testing that version.

     
  • psilva

    psilva - 2016-12-01
    • assigned_to: psilva
     
  • psilva

    psilva - 2016-12-10
    • assigned_to: psilva --> Khosrow Ebrahimpour
     
  • psilva

    psilva - 2016-12-10

    khosrow please confirm in your stuff if you see the destination file name in the current daily builds. Lucie cannot test because she is using the main systems on 7a3. it should be easy
    for you to confirm. If you see the destination files there, then can close-fixed this bug.

     
  • psilva

    psilva - 2016-12-11
    • Group: deferred --> v2.16.00b
     
  • Khosrow Ebrahimpour

    Here's what we get in the logs:

    2016-12-08 19:47:46,743 [INFO] Sends: 20160301_ddi1_panam.tar.gz  into /data/tmp/ 0-24185844343
    2016-12-08 22:08:07,312 [INFO] post_log notice=20161208194744.615 sftp://someuser@example.cmc.ec.gc.ca//20160301_ddi1_panam.tar.gz headers={'flow': 'khosrow.ebrahimpo
    ur@canada.ca,20160301.tar.gz,2016/03', 'sum': 'd,4eb6043546034de7c50dc96942a90e2c', 'to_clusters': 'test_cluster', 'filename': '20160301_ddi1_panam.tar.gz', '
    parts': '1,24185844344,1,0,0', 'mtime': '20160303074115.126', 'atime': '20161208193751.926'}
    

    I see filename: in the post field, which corresponds to the name of the posted file.

    Please note that we do not use a DESTFNSCRIPT in our setup.

     
  • psilva

    psilva - 2016-12-14

    the post_log is created when posting the notification after the file has been written, so DESTFN has been run. It should be correct.

     
  • psilva

    psilva - 2017-04-28
    • assigned_to: Khosrow Ebrahimpour --> psilva
     
  • psilva

    psilva - 2017-04-28

    I think this is fixed long ago, but the server where we care about it has not gotten the new version yet. once it is there we will confirm the fix and marke it closed.
    also: Khosrow has left the building.

     
  • psilva

    psilva - 2017-05-29

    no it was never fixed. now it is.

     
  • psilva

    psilva - 2017-06-23

    released in 2.17.06

     
  • psilva

    psilva - 2017-07-29
    • status: open --> closed
     

Log in to post a comment.

MongoDB Logo MongoDB