postreceive running multiple times
Encrypted UDP based FTP with multicast
Brought to you by:
dennisbush
Menu
▾
▴
I'm trying out the postrecieve function. Is it working well other than the fact that it seems to be firing more than once, per the log below. In some situations I think I have seen it fire up to 4 files for a single file push.
root@43ea21c6d696:/var/log# cat /tmp/1 | egrep -v "COMPLETE|Source ID|packet"
2017/07/18 22:31:42.841477: UFTP version 4.9.3 Copyright (C) 2001-2017 Dennis A. Bush
2017/07/18 22:31:42.841530: Loaded 512 bit RSA key with fingerprint A8:A0:C2:B0:3B:6A:95:D4:AB:04:87:84:03:FE:55:E4:B8:C6:FA:A6
2017/07/18 22:31:47.415595: [5288F3C6/00:0]: Received request from 0A021502 at 172.17.0.1 (172.17.0.1)
2017/07/18 22:31:47.415624: [5288F3C6/00:0]: Using private multicast address 230.5.5.121
2017/07/18 22:31:47.415641: [5288F3C6/00:0]: grtt = 0.532216
2017/07/18 22:31:47.415663: [5288F3C6/00:0]: send time: 1500417107.417815
2017/07/18 22:31:47.415673: [5288F3C6/00:0]: receive time: 1500417107.415510
2017/07/18 22:31:47.415751: [5288F3C6/00:0]: REGISTER sent
2017/07/18 22:31:47.415768: [5288F3C6/00:0]: send time: 1500417107.418013
2017/07/18 22:31:48.916149: [5288F3C6/00:0]: Registration confirmed
2017/07/18 22:31:49.487366: [5288F3C6/00:0001]: Name of file to receive: ucpe_packages/bash.sh
2017/07/18 22:31:49.487432: [5288F3C6/00:0001]: Bytes: 4594973, Blocks: 3535, Sections: 1
2017/07/18 22:31:49.487446: [5288F3C6/00:0001]: small section size: 3535, big section size: 3535, # big sections: 0
2017/07/18 22:31:49.487612: Free space: 52639621120
2017/07/18 22:31:49.487678: [5288F3C6/00:0001]: FILEINFO_ACK sent
2017/07/18 22:31:49.487693: [5288F3C6/00:0001]: send time: 1500417109.490534
2017/07/18 22:31:49.487717: [5288F3C6/00:0001]: FILEINFO_ACK sent
2017/07/18 22:31:49.487730: [5288F3C6/00:0001]: send time: 1500417109.490593
2017/07/18 22:32:08.167421: [5288F3C6/00:0001]: Got DONE message for section 0
2017/07/18 22:32:08.169971: [5288F3C6/00:0001]: Wrote blocks 0 - 3534 to disk from cache
2017/07/18 22:32:08.173201: [5288F3C6/00:0001]: File transfer complete
2017/07/18 22:32:08.173264: [5288F3C6/00:0001]: starting file close
2017/07/18 22:32:08.173279: [5288F3C6/00:0001]: done file close
2017/07/18 22:32:08.173290: [5288F3C6/00:0001]: Running postreceive: /bin/run_blueprint.sh -I 5288F3C6 /tmp/ucpe_packages/bash.sh
2017/07/18 22:32:08.173580: [5288F3C6/00:0001]: Got DONE message for section 0
2017/07/18 22:32:08.173626: [5288F3C6/00:0001]: File transfer complete
2017/07/18 22:32:08.173671: [5288F3C6/00:0001]: Running postreceive: /bin/run_blueprint.sh -I 5288F3C6 /tmp/ucpe_packages/bash.sh
2017/07/18 22:32:08.188986: [5288F3C6/00:0]: Got DONE message for group
2017/07/18 22:32:08.189012: [5288F3C6/00:0]: Group complete
2017/07/18 22:32:09.189088: [5288F3C6/00:0]: Completion unconfirmed by server
Any ideas?!
I'm wondering, based on 2 lots of "Got DONE message for section 0" whether it's not just the postreceive but the whole final phase that is running twice. I will own up that I am using the proxies in a slight unusual way that MAY be causing this...
Looks like a bug. Thanks for catching this.
The postreceive script should only be run the first time a DONE is received for a given file. All other cleanup tasks are only done once, such as closing the file (as noted in the logs) and temp directory handling, so the call to the postreceive script needs to go in the same place.
I'll include the fix in the next release.
Last edit: Dennis Bush 2017-07-19
Dennis - thanks for the quick response. I am running my tests a different way and not using the Proxy and I'm only seeing the script call once.
For interest my setup is this:
Running Open vSwitch to connect up all the docker1 networks so they can talk to each other (with a GRE tunnel between OVS on each host)
On Host 1 running uftp -I docker1 to inject the traffic into the docker1 network on host1 which gets propagated across all the different hosts
In this way I am able to run scale testing to many clients using docker containers as isolated uftp end points across several hosts.