Missing Retransmission of Last Block When ACK Is Not Received
Debian patches recommended to be applied upstream
Hey Andi , Martin , i am seeing the similar issue where my pcre2 methods are not getting detected and am seeing" undefined reference to pcre2_match_data_create_from_pattern_'" issues and many others , as the original git has the pcre support https://sourceforge.net/projects/atftp/files/ . I wanted to check how do you guys have resolved this issue in here. I tried to add pcre2.h from https://github.com/php/php-src/blob/master/ext/pcre/pcre2lib/pcre2.h but it didnt worked for me
Recommended Way to Report a Potential Security Issue
Thanks for the patches! In total this would be the following: --------------------------------- test/test.sh --------------------------------- index bae3f71..0c2e3e2 100755 @@ -575,7 +575,7 @@ stop_and_clean(){ echo ## +3 is for "Test tsize option ..." and "Test PCRE mapped download ... " ## +2 for diskspace tests: - local M=$(grep "/tmp/" "$TDIR"/multicast*.log | wc -l) + local D M=$(grep "/tmp/" "$TDIR"/multicast*.log 2> /dev/null | wc -l) $INTERACTIVE && D=2 cat <<EOF
Looks good to me. -------------------------------- test/test.sh --------------------------------- index bae3f71..3661bec 100755 @@ -575,7 +575,7 @@ stop_and_clean(){ echo ## +3 is for "Test tsize option ..." and "Test PCRE mapped download ... " ## +2 for diskspace tests: - local M=$(grep "/tmp/" "$TDIR"/multicast*.log | wc -l) + local D M=$(grep "/tmp/" "$TDIR"/multicast*.log | wc -l) $INTERACTIVE && D=2 cat <<EOF
I guess this bug can be closed with the latest release.
The release has happened, I guess this bug can be closed. Thanks!
I guess this bug is fixed and can be closed.
I guess this bug can now be closed.
I guess this bug can now be closed.
By default Ubuntu compile packages with LTO enabled, which I think Debian is not yet using. There's two course of actions, either we disable it on our Deb, or upstream can evaluate if they want to have this compile flag enabled by default and fix the issues in code (something like the mentioned fix above).
tried via the console directly: same problem.
On Ubuntu 22.04: 0.7.git20210915-4 crashes unable to start any client
Got the same problem with Ubuntu 22.04. Need help
I can confirm the issue on jammy. However, as far as I can tell, it's the same code. I did not investigate it in detail, but it looks like it's a matter of compiler flags applied, which causes the issue.
atftpd runs as daemon from the shell but fails on launch by the systemd socket
Fix line counting in test.sh by redirecting stderr to /dev/null
Declare variable D local in test.sh stop_and_clean()
Well, this is exactly how I found out. The default install on jammy uses systemd sockets and restarts atftpd daemon after each segfault. This might hide the problem for some time.
I cannot confirm the problem either. Requesting non-existent files from atftpd happens very often during pxeboot, so a serious error like this would be detected quickly.
Hm, seems to be Ubuntu specific, still not reproducible here on Debian. I'm going to install jammy and try to find the difference. Ubuntu bug : https://bugs.launchpad.net/ubuntu/+source/atftp/+bug/1989816
Hi, thanks for the report and patch. I have to investigate, as soon I find some available time slot.
Hi, I've found the same behavior on ubuntu jammy. After some debugging I've found this: Strncpy(tftphdr->th_msg, tftp_errmsg[err_code], buffer_size - 4); in line 171 of tftp_io.c The definition of tftphdr in /usr/include/arpa/tftp.h is as follows: struct tftphdr { short th_opcode; /* packet type */ union { char tu_padding[3]; /* sizeof() compat */ struct { union { unsigned short tu_block; /* block # */ short tu_code; /* error code */ } __attribute__ ((__packed__)) th_u3; char tu_data[0]; /* data...
Version 0.8.0 (Feature Release)
Version 0.8.0 (Feature Release)
Hi Martin, many thanks for preparing the release. One minor cosmetic thing is the version string, patch attached. Best regards, Andi
Version 0.8.0 (Feature Release)
Hi Martin, great, many thanks! I checked differences from commit 982ba233fc6eac35 to current Debian sources after applying all patches there: It's all fine! Differences are only white space and typos (Debian sources miss some upstream fixes/cleanup). This is also the case for the PCRE part. So from my point of view, it's good to go! Best regards, Andi
Hi Adni, I merged all your patches and pushed the repositories. Please check is all your fixes and patches arrived properly. Especially the PCD2 implenetation needs a check - I committed the patch from december and you mention a fixed version - but I see no newer patch. PLS let me know. Bye Martin
Fix multicast download.
Fix MTFTP support for atftp
Improve the robustness of the atftp-client in case of package loss or duplication
Fix algorithm in case of packet loss in the last window.
test.sh: add MTFTP tests
Improve upstream test script. Include tests for windowsize option.
*.h: cosmetic and spelling fixes
Fix the 'windowsize' option for write requests
Port to maintained PCRE2 API
Changelog: update the recent changes
Fix/update minor issues in upstream
FAQ INSTALL README.CVS README.PCRE: cosmetic and spelling fixes
configure.ac: more fixes, also for libpcre2 detection
Implement PCRE tests.
… and the test script with some MTFTP tests added.
… and the test script with some MTFTP tests added.
MTFTP support
Hi, I tested (and fixed, see https://sourceforge.net/p/atftp/support-requests/10/ ) multicast and added the routines to the test script. For me, it needs other machines in the network to test and seems not to be possible on the local machine only. (At least without extra effort, but I am happy to learn better ways.) I set up 12 mactap'ed VMs: ===== Test multicast option ... Run atftp on: 'ansible@daily0.lan', 'ansible@daily1.lan', 'ansible@daily2.lan', … , fetching '128K.bin'. Copy log file: 'ansible@daily0.lan'...
The attached patch improves the robustness of the atftp-client in case of package loss or duplication. Updates: patch simplified. add Server part (upload) another patch fix-last-window.patch is needed to handle losses in the last window (apply that after handle-loss-duplication-better.patch) multicast did not work anymore, but is fixed now: fix-multicast.patch . Tested thoroughly with the new test script and wireshark. ToDo: multicast with windowsize != 1
The attached patch improves the robustness of the atftp-client in case of package loss or duplication. Updates: patch simplified. add Server part (upload) another patch fix-last-window.patch is needed to handle losses in the last window (apply that after handle-loss-duplication-better.patch) multicast did not work anymore, but is fixed now: fix-multicast.patch tested thoroughly with the new test script and wireshark. ToDo: multicast with windowsize != 1
The attached patch improves the robustness of the atftp-client in case of package loss or duplication. Updates: patch simplified. add Server part (upload) another patch fix-last-window.patch is needed to handle losses in the last window (apply that after handle-loss-duplication-better.patch)
The attached patch improves the robustness of the atftp-client in case of package loss or duplication. patch simplified. add Server part (upload) another patch is needed to handle losses in the last window (applied last)
The attached patch improves the robustness of the atftp-client in case of package loss or duplication. patch simplified. add Server part (upload) another patch is needed to handle losses in the last window
The attached patch improves the robustness of the atftp-client in case of package loss or duplication. * Patch simplified. * Add Server part (upload)
The attached patch improves the robustness of the atftp-client in case of package loss or duplication. Patch simplified.
I attached a patch improving the handling of lost or duplicated datagrams in https://sourceforge.net/p/atftp/support-requests/10/ . This patch should be applied in case of a release too.
The attached patch improves the robustness of the atftp-client in case of package loss or duplication.
The only part not tested is the timeout, which needs debugging in atftp enabled. I've tested that now, too. Latest test script version attached.
I have mostly rewritten the test script. Functionality is almost the same, but more structured and mostly cosmetics. The only part not tested is the timeout, which needs debugging in atftp enabled. Best regards, Andi
This works fine, as tested with an improved test.sh (attached).
I think it's all fine with the new implementation: atftp$ grep -A5 tftpd_pcre_sub *.c tftpd.c: if (tftpd_pcre_sub(pcre_top, out, MAXLEN, string) < 0) tftpd.c- printf("Substitution: \"%s\" -> \"\"\n", string); tftpd.c- else tftpd.c- printf("Substitution: \"%s\" -> \"%s\"\n", string, out); and /* search for a replacement and return a string after substitution */ /* if no match is found return -1 */ int tftpd_pcre_sub(tftpd_pcre_self_t *self, char *outstr, int outlen, char *str) From man atftpd: Upon...
I think it's all fine with the new implementation: atftp$ grep -A5 tftpd_pcre_sub *.c tftpd.c: if (tftpd_pcre_sub(pcre_top, out, MAXLEN, string) < 0) tftpd.c- printf("Substitution: \"%s\" -> \"\"\n", string); tftpd.c- else tftpd.c- printf("Substitution: \"%s\" -> \"%s\"\n", string, out); and /* search for a replacement and return a string after substitution */ /* if no match is found return -1 */ int tftpd_pcre_sub(tftpd_pcre_self_t *self, char *outstr, int outlen, char *str) From man atftpd: Upon...
One correction: For the new implementation, the replacements are identical in all tested cases. This is true except there is no match at all. Hm. I guess the best is to always return the same like: perl -e '$x = "nomatchatall"; $x =~ s#PATTERN#SUBSTITUTION#; print "$x\n";' So if there is no match, the initial string should be returned.
Hi Martin, thanks for your quick reply! Some time ago I took your PCRE2 patch, adapted the test suite to evaluate PCRE and made some tests with PCRE1 and PCRE2. AFAIR there were at least small differences in behaviour. Very interesting! I tried the test suite with the old implementation, and indeed: Substitution: "ppxelinux.cfg/012345" -> "pxelinux.cfg/default" Substitution: "ppxelinux.cfg/678" -> "pxelinux.cfg/default" Substitution: "ppxelinux.cfg/9ABCDE" -> "pxelinux.cfg/default" -Substitution:...
Hello Andi, at first thanks for you efforts in fixing bugs and developing. Some time ago I took your PCRE2 patch, adapted the test suite to evaluate PCRE and made some tests with PCRE1 and PCRE2. AFAIR there were at least small differences in behaviour. I will test again and post results at the right place. Then we can decide how to continue. PCRE2, maybe with behaviour changes, would qualify for a 8.0. I'll call you back.... Martin
Hi all, attached is the final patch (fix-windowsize-WRQ.patch) addressing this issue. The problem was, that the server did not use the windowsize value sent from the client when receiving a file. A second patch (minor-fixes.patch) addresses an update to RFC1782 → RFC2347 and adds the windowsize option to the client's status output. I updated the test script in patch tests.patch. It checks different windowsize options. It detects and reports the bug fixed nicely now. Finally, test-pcre.patch implements...
Hi all, attached is the final patch (fix-windowsize-WRQ.patch) addressing this issue. The problem was, that the server did not use the windowsize value sent from the client when receiving a file. A second patch (minor-fixes.patch) addresses an update to RFC1782 → RFC2347 and adds the windowsize option to the client's status output. I updated the test script in patch tests.patch. It checks different windowsize options. It detects and reports the bug fixed nicely now. Finally, test-pcre.patch implements...
Hi all, attached is the final patch (fix-windowsize-WRQ.patch) addressing this issue. The problem was, that the server did not use the windowsize value sent from the client when receiving a file. A second patch (minor-fixes.patch) addresses an update to RFC1782 → RFC2347 and adds the windowsize option to the client's status output. I updated the test script in patch tests.patch. It checks different windowsize options. It detects and reports the bug fixed nicely now. Finally, test-pcre.patch implements...
Hi all, attached is the final patch (fix-windowsize-WRQ.patch) addressing this issue. The problem was, that the server did not use the windowsize value sent from the client when receiving a file. A second patch (minor-fixes.patch) addresses an update to RFC1782 → RFC2347 and adds the windowsize option to the client's status output. Finally I updated the test script in patch tests.patch. It checks different windowsize options. It detects and reports the bug fixed nicely now. Best regards, Andi
Time for a new Release?
Hi all, attached is the final patch (fix-windowsize-WRQ.patch) addressing this issue. The problem was, that the server did not use the windowsize value sent from the client when receiving a file. A second patch (minor-fixes.patch) addresses an update to RFC1782 → RFC2347 and adds the windowsize option to the client's status output. Finally I updated the test script in patch tests.patch. It checks different windowsize options. It detects and reports the bug fixed nicely now. Best regards, Andi
Patch is available: https://salsa.debian.org/debian/atftp/-/blob/master/debian/patches/fix-windowsize-WRQ.patch I'll add some tests to test/test.sh soon.
I can confirm this bug. Write requests have not been tested as much as read requests. I'll investigate. Thanks for testing and reporting!
I can confirm this bug. Write requests have not been tested as much as read requests. I'll investigate. Thanks for testing and reporting!
Have you solved the problem? Perhaps investigate with tcpdump like (shown on localhost here, edited to make it more readable): $ sudo tcpdump -i lo -v tcpdump: listening on lo, link-type EN10MB (Ethernet), snapshot length 262144 bytes localhost.55741 > localhost.tftp: TFTP, length 27, RRQ "d-i/n-a/README.txt" octet localhost.43854 > localhost.55741: UDP, length 177 <--- DATA localhost.55741 > localhost.43854: UDP, length 4 <----ACK The ports are probably not the problem: The client chooses a (client-)...
Have you solved the problem? Perhaps investigate with tcpdump like (shown on localhost here): $ sudo tcpdump -i lo -v tcpdump: listening on lo, link-type EN10MB (Ethernet), snapshot length 262144 bytes 12:13:09.647855 IP6 (flowlabel 0x72911, hlim 64, next-header UDP (17) payload length: 35) localhost.55741 > localhost.tftp: [bad udp cksum 0x0036 -> 0x37da!] TFTP, length 27, RRQ "d-i/n-a/README.txt" octet 12:13:09.666530 IP6 (flowlabel 0x46081, hlim 64, next-header UDP (17) payload length: 185) localhost.43854...
Hi, can you please provide a bit more information? Do you use a packaged atftpd, if 'yes', which distribution? How is atftpd started (systemd socket activation or xinetd)? What if you run the server in debug mode? Perhaps directly from the command line? I tested here on Debian testing and anything seems to work fine: $ atftp tftp> connect localhost tftp> get d-i/n-a/bootnetx64.efi Overwrite local file [y/n]? y tftp> get nonexistent tftp: error received from server <file not="" found=""> tftp: aborting...
ATFTPD crashes when requested a non-existent file
Server ignores windowsize option when files are written
My analysis is wrong with respect to using random source port numbers from the server. I still do not know why the Linux client is returning an ICMP destination unreachable error. This happens with both atftp and the other tftp client available in OpenSUSE 15.3.
No route to destination, bad source UDP port number from tftp server
Great, it works here too now. The patch should be up to date now and I uploaded the package, but perhaps your solution is better. Thanks!
Hi, I have managed the detection by autoconf already. I will share my findings here today in the evening (in approx. 5-6 hours) - have no time for this just now. Or better, I open a feature-development-branch in the git here, okay? and commit my stuff, so you can see what I did.
Hi Martin, I am still working on the integration into the autoconf/configure build environment. After cleaning up and building in a minimal chroot environment for packaging, I found that PCRE is not enabled because it seems not be detected correctly. I am still investigating but it looks like I need to learn a bit about autotools and the like first :-) Best regards, Andi
Hello Andi, I read your notice and already started investigating how to convert to PCRE2. After some hours/days I found that my programming knowledge is not high enough to manage this on my own. So I am very pleased and surprised that you did that already and I am really looking forward to test and integrate the results in the next few days!!!! Greetings / Martin
Latest patch is available here: https://salsa.debian.org/debian/atftp/-/blob/master/debian/patches/pcre2.patch
Switch from old PCRE to actively maintained PCRE2
autoconf: modify autogen.sh
atftp.1: add more examples for options
Add windowsize option as described in RFC7440
Do some cosmetic changes
Add a simple congestion control
Hi Martin, so far it seems to work fine. It would be interesting to test in a network with congestion as it looks like we have too much bandwidth and we have no package loss ;-). I tried to simulate that locally with examples from https://wiki.linuxfoundation.org/networking/netem , but it's hard to choose a realistic scenario. I implemented some kind of simple congestion control on top of the other patches: When windows are not completely received because frames got lost, the delay between the datagrams...
Hi, here is an updated/refurbished windowsize patch that applies and integrates into the existing code. I tried to drop white space changes and some cleanup into an extra patch to simplify comparing the modifications. I'll do some more testing in the next days/weeks, also on real hardware. Testers and reviewers are welcome, especially for multicast setups! 2021-10-23: I added the congestion patch here; the post below is waiting for moderation…
Note: An additional congestion control patch is available, see above.
Note: An additional congestion control patch is available.