From: Tony He <hua...@gm...> - 2021-03-31 04:22:41
|
Hi Antonio, As you know I am porting opvn-dco to my router whose kernel is V4.14.76. After solving AF_NETLINK group issue we discussed yesterday. It finally works. But I encounter another issue :-( . When testing the performance with iperf3, disconnection occurs and recovers after a few seconds and then again. 1. powerful x86-64 PC -> weak MIPS64 router(clocked 1GHZ) Accepted connection from 10.8.0.2, port 38930 [ 5] local 10.8.0.1 port 5201 connected to 10.8.0.2 port 38932 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 6.45 MBytes 54.1 Mbits/sec [ 5] 1.00-2.00 sec 7.93 MBytes 66.6 Mbits/sec [ 5] 2.00-3.00 sec 7.66 MBytes 64.3 Mbits/sec [ 5] 3.00-4.00 sec 7.94 MBytes 66.6 Mbits/sec [ 5] 4.00-5.00 sec 7.84 MBytes 65.8 Mbits/sec [ 5] 5.00-6.00 sec 7.83 MBytes 65.7 Mbits/sec [ 5] 6.00-7.00 sec 7.84 MBytes 65.8 Mbits/sec [ 5] 7.00-8.00 sec 9.39 MBytes 78.8 Mbits/sec [ 5] 8.00-9.00 sec 8.13 MBytes 68.2 Mbits/sec [ 5] 9.00-10.00 sec 6.31 MBytes 53.0 Mbits/sec [ 5] 10.00-11.00 sec 6.31 MBytes 53.0 Mbits/sec [ 5] 11.00-12.00 sec 6.24 MBytes 52.3 Mbits/sec [ 5] 12.00-13.00 sec 6.42 MBytes 53.8 Mbits/sec [ 5] 13.00-14.00 sec 7.80 MBytes 65.4 Mbits/sec [ 5] 14.00-15.00 sec 7.75 MBytes 65.0 Mbits/sec [ 5] 15.00-16.00 sec 7.86 MBytes 66.0 Mbits/sec [ 5] 16.00-17.00 sec 5.03 MBytes 42.2 Mbits/sec [ 5] 17.00-18.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 18.00-19.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 19.00-20.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 20.00-21.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 21.00-22.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 22.00-23.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 23.00-24.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 24.00-25.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 25.00-26.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 26.00-27.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 27.00-28.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 28.00-29.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 29.00-30.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 30.00-31.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 31.00-32.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 32.00-33.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 33.00-34.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 34.00-35.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 35.00-36.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 36.00-37.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 37.00-38.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 38.00-39.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 39.00-40.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 40.00-41.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 41.00-42.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 42.00-43.00 sec 0.00 Bytes 0.00 bits/sec [ 5] 43.00-44.00 sec 4.92 MBytes 41.3 Mbits/sec [ 5] 44.00-45.00 sec 7.74 MBytes 64.9 Mbits/sec [ 5] 45.00-46.00 sec 8.01 MBytes 67.2 Mbits/sec [ 5] 46.00-47.00 sec 7.96 MBytes 66.8 Mbits/sec [ 5] 47.00-48.00 sec 7.49 MBytes 62.8 Mbits/sec Log from dco module: [36981.631546] ovpn_udp_encap_recv: cannot handle incoming packet: -28 [36982.692005] ovpn_udp_encap_recv: cannot handle incoming packet: -28 2. weak MIPS64 router -> powerful x86-64 PC. It's OK. Server listening on 5201 ----------------------------------------------------------- Accepted connection from 10.8.0.2, port 39238 [ 5] local 10.8.0.1 port 5201 connected to 10.8.0.2 port 39240 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 11.2 MBytes 94.3 Mbits/sec 0 53.7 KBytes [ 5] 1.00-2.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 2.00-3.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 3.00-4.00 sec 8.89 MBytes 74.5 Mbits/sec 0 48.1 KBytes [ 5] 4.00-5.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 5.00-6.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 6.00-7.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 7.00-8.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 8.00-9.00 sec 8.89 MBytes 74.5 Mbits/sec 0 53.7 KBytes [ 5] 9.00-10.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 10.00-11.00 sec 8.89 MBytes 74.5 Mbits/sec 0 53.7 KBytes [ 5] 11.00-12.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 12.00-13.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 13.00-14.00 sec 9.57 MBytes 80.3 Mbits/sec 0 50.9 KBytes [ 5] 14.00-15.00 sec 9.57 MBytes 80.3 Mbits/sec 0 48.1 KBytes [ 5] 15.00-16.00 sec 9.63 MBytes 80.8 Mbits/sec 0 65.0 KBytes [ 5] 16.00-17.00 sec 11.6 MBytes 97.5 Mbits/sec 0 59.4 KBytes [ 5] 17.00-18.00 sec 8.89 MBytes 74.5 Mbits/sec 0 53.7 KBytes [ 5] 18.00-19.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 19.00-20.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 20.00-21.00 sec 8.89 MBytes 74.5 Mbits/sec 0 53.7 KBytes [ 5] 21.00-22.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 22.00-23.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 23.00-24.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 24.00-25.00 sec 9.57 MBytes 80.3 Mbits/sec 0 48.1 KBytes [ 5] 25.00-26.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 26.00-27.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 27.00-28.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 28.00-29.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 29.00-30.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 30.00-31.00 sec 8.20 MBytes 68.8 Mbits/sec 0 53.7 KBytes [ 5] 31.00-32.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 32.00-33.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 33.00-34.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 34.00-35.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 35.00-36.00 sec 9.57 MBytes 80.3 Mbits/sec 0 59.4 KBytes [ 5] 36.00-37.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 37.00-38.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 38.00-39.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 39.00-40.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 40.00-41.00 sec 8.89 MBytes 74.5 Mbits/sec 0 53.7 KBytes [ 5] 41.00-42.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 42.00-43.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 43.00-44.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 44.00-45.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 45.00-46.00 sec 9.57 MBytes 80.2 Mbits/sec 0 53.7 KBytes [ 5] 46.00-47.00 sec 8.89 MBytes 74.6 Mbits/sec 0 53.7 KBytes [ 5] 47.00-48.00 sec 10.3 MBytes 86.0 Mbits/sec 0 53.7 KBytes [ 5] 48.00-49.00 sec 8.89 MBytes 74.5 Mbits/sec 0 53.7 KBytes [ 5] 49.00-50.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 50.00-51.00 sec 8.89 MBytes 74.6 Mbits/sec 0 5.66 KBytes [ 5] 51.00-52.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 52.00-53.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 53.00-54.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 54.00-55.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 55.00-56.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 56.00-57.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 57.00-58.00 sec 9.57 MBytes 80.3 Mbits/sec 0 48.1 KBytes [ 5] 58.00-59.00 sec 9.57 MBytes 80.3 Mbits/sec 0 53.7 KBytes [ 5] 59.00-60.00 sec 9.57 MBytes 80.3 Mbits/sec 0 48.1 KBytes 3. powerful x86-64 PC -> weak MIPS64 router, but disable dco. Also OK. Server listening on 5201 ----------------------------------------------------------- ^BAccepted connection from 10.8.0.2, port 40324 [ 5] local 10.8.0.1 port 5201 connected to 10.8.0.2 port 40326 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 5.52 MBytes 46.3 Mbits/sec [ 5] 1.00-2.00 sec 5.77 MBytes 48.4 Mbits/sec [ 5] 2.00-3.00 sec 5.68 MBytes 47.6 Mbits/sec [ 5] 3.00-4.00 sec 5.83 MBytes 48.9 Mbits/sec [ 5] 4.00-5.00 sec 5.97 MBytes 50.1 Mbits/sec [ 5] 5.00-6.00 sec 6.42 MBytes 53.9 Mbits/sec [ 5] 6.00-7.00 sec 6.59 MBytes 55.3 Mbits/sec [ 5] 7.00-8.00 sec 6.48 MBytes 54.3 Mbits/sec [ 5] 8.00-9.00 sec 6.53 MBytes 54.8 Mbits/sec [ 5] 9.00-10.00 sec 6.44 MBytes 54.0 Mbits/sec [ 5] 10.00-11.00 sec 6.59 MBytes 55.3 Mbits/sec [ 5] 11.00-12.00 sec 6.53 MBytes 54.8 Mbits/sec [ 5] 12.00-13.00 sec 6.52 MBytes 54.7 Mbits/sec [ 5] 13.00-14.00 sec 6.24 MBytes 52.4 Mbits/sec [ 5] 14.00-15.00 sec 6.38 MBytes 53.5 Mbits/sec [ 5] 15.00-16.00 sec 6.55 MBytes 55.0 Mbits/sec [ 5] 16.00-17.00 sec 6.53 MBytes 54.7 Mbits/sec [ 5] 17.00-18.00 sec 6.46 MBytes 54.1 Mbits/sec [ 5] 18.00-19.00 sec 6.50 MBytes 54.6 Mbits/sec [ 5] 19.00-20.00 sec 6.55 MBytes 55.0 Mbits/sec [ 5] 20.00-21.00 sec 6.51 MBytes 54.6 Mbits/sec [ 5] 21.00-22.00 sec 6.53 MBytes 54.8 Mbits/sec [ 5] 22.00-23.00 sec 6.46 MBytes 54.2 Mbits/sec [ 5] 23.00-24.00 sec 6.47 MBytes 54.3 Mbits/sec [ 5] 24.00-25.00 sec 6.60 MBytes 55.4 Mbits/sec [ 5] 25.00-26.00 sec 6.46 MBytes 54.2 Mbits/sec [ 5] 26.00-27.00 sec 6.46 MBytes 54.2 Mbits/sec [ 5] 27.00-28.00 sec 6.54 MBytes 54.9 Mbits/sec [ 5] 28.00-29.00 sec 6.56 MBytes 55.0 Mbits/sec [ 5] 29.00-30.00 sec 6.49 MBytes 54.5 Mbits/sec [ 5] 30.00-31.00 sec 6.40 MBytes 53.7 Mbits/sec [ 5] 31.00-32.00 sec 6.59 MBytes 55.3 Mbits/sec [ 5] 32.00-33.00 sec 6.44 MBytes 54.0 Mbits/sec [ 5] 33.00-34.00 sec 6.58 MBytes 55.2 Mbits/sec [ 5] 34.00-35.00 sec 6.33 MBytes 53.1 Mbits/sec [ 5] 35.00-36.00 sec 6.45 MBytes 54.1 Mbits/sec [ 5] 36.00-37.00 sec 6.49 MBytes 54.5 Mbits/sec [ 5] 37.00-38.00 sec 6.46 MBytes 54.2 Mbits/sec [ 5] 38.00-39.00 sec 6.53 MBytes 54.8 Mbits/sec [ 5] 39.00-40.00 sec 6.52 MBytes 54.7 Mbits/sec [ 5] 40.00-41.00 sec 6.44 MBytes 54.0 Mbits/sec [ 5] 41.00-42.00 sec 6.48 MBytes 54.3 Mbits/sec [ 5] 42.00-43.00 sec 6.43 MBytes 54.0 Mbits/sec [ 5] 43.00-44.00 sec 6.59 MBytes 55.3 Mbits/sec [ 5] 44.00-45.00 sec 6.46 MBytes 54.2 Mbits/sec [ 5] 45.00-46.00 sec 6.55 MBytes 54.9 Mbits/sec [ 5] 46.00-47.00 sec 6.60 MBytes 55.4 Mbits/sec [ 5] 47.00-48.00 sec 6.06 MBytes 50.8 Mbits/sec [ 5] 48.00-49.00 sec 4.63 MBytes 38.8 Mbits/sec [ 5] 49.00-50.00 sec 4.68 MBytes 39.3 Mbits/sec [ 5] 50.00-51.00 sec 5.26 MBytes 44.1 Mbits/sec [ 5] 51.00-52.00 sec 5.79 MBytes 48.5 Mbits/sec [ 5] 52.00-53.00 sec 5.74 MBytes 48.2 Mbits/sec [ 5] 53.00-54.00 sec 5.71 MBytes 47.9 Mbits/sec [ 5] 54.00-55.00 sec 5.77 MBytes 48.4 Mbits/sec [ 5] 55.00-56.00 sec 5.74 MBytes 48.2 Mbits/sec [ 5] 56.00-57.00 sec 5.79 MBytes 48.5 Mbits/sec [ 5] 57.00-58.00 sec 5.66 MBytes 47.5 Mbits/sec [ 5] 58.00-59.00 sec 5.81 MBytes 48.7 Mbits/sec [ 5] 59.00-60.00 sec 5.74 MBytes 48.2 Mbits/sec [ 5] 60.00-60.00 sec 11.9 KBytes 54.6 Mbits/sec Not sure if it's a problem caused by dco module or my old kernel or hardware. Maybe I should test in more platforms and kernel versions to compare. If you have any idea, please let me know. Thanks a lot. Tony |
From: Simon M. <sim...@in...> - 2021-03-31 05:38:55
|
> Hi Antonio, > > As you know I am porting opvn-dco to my router whose kernel is V4.14.76. > After solving AF_NETLINK group issue we discussed > yesterday. It finally works. But I encounter another issue :-( . When > testing the performance with iperf3, disconnection occurs > and recovers after a few seconds and then again. Just a guess, could it have something to do with missing entropy? Maybe for such tests a passthrough mode in the kernel module without encryption could help to debug. Or, as a second option, a non encryption mode in OpenVPN to test. I think this is or was available. Simon |
From: Antonio Q. <a...@un...> - 2021-03-31 06:31:20
|
Hi, On 31/03/2021 07:21, Simon Matter wrote: >> Hi Antonio, >> >> As you know I am porting opvn-dco to my router whose kernel is V4.14.76. >> After solving AF_NETLINK group issue we discussed >> yesterday. It finally works. But I encounter another issue :-( . When >> testing the performance with iperf3, disconnection occurs >> and recovers after a few seconds and then again. > > Just a guess, could it have something to do with missing entropy? > > Maybe for such tests a passthrough mode in the kernel module without > encryption could help to debug. Or, as a second option, a non encryption > mode in OpenVPN to test. I think this is or was available. There is really not much entropy involved in the encryption/decryption routine. Randomness is mainly used during session key generation (at negotiation time) and that's all. Cheers, -- Antonio Quartulli |
From: Antonio Q. <a...@un...> - 2021-03-31 06:30:16
|
Hi, On 31/03/2021 06:22, Tony He wrote: > [36981.631546] ovpn_udp_encap_recv: cannot handle incoming packet: -28 > [36982.692005] ovpn_udp_encap_recv: cannot handle incoming packet: -28 Are these the only two lines from ovpn-dco? or is this a "summary"? The problem might be related to the router not being able to consume incoming packets as fast as it needs, thus dropping packets that kill the iperf connection. Seems a bit too drastic, but with the little information I have, this is all I can come up. Can you please re-run the test and have a ping from the PC to the router ongoing at the same time? It'd be interesting to see how being behaves when the problems appears. A packet dump of the whole session may also help. Cheers, -- -- Antonio Quartulli |
From: Antonio Q. <a...@un...> - 2021-03-31 06:32:39
|
Hi, On 31/03/2021 08:29, Antonio Quartulli wrote: > A packet dump of the whole session may also help. Before taking the dump, I would switch to encryption "none", as it will help understanding what is going on at all levels. (Assuming the problem will still appear) Cheers, -- Antonio Quartulli |
From: Tony He <hua...@gm...> - 2021-03-31 07:29:55
|
Hi Arne, I'm going to test encryption "none" to narrow down this issue, but I found your dco branch doesn't support this. Can you support? Tony Antonio Quartulli <a...@un...> 于2021年3月31日周三 下午2:32写道: > Hi, > > On 31/03/2021 08:29, Antonio Quartulli wrote: > > A packet dump of the whole session may also help. > > Before taking the dump, I would switch to encryption "none", as it will > help understanding what is going on at all levels. (Assuming the problem > will still appear) > > Cheers, > > > -- > Antonio Quartulli > |
From: Antonio Q. <a...@un...> - 2021-03-31 07:32:55
|
Hi, On 31/03/2021 09:29, Tony He wrote: > Hi Arne, > > I'm going to test encryption "none" to narrow down this issue, but I > found your dco branch doesn't support this. > Can you support? For the sake of this test, could you use the ovpn-cli.c tool in the ovpn-dco/tests folder? Or that's not an option? -- Antonio Quartulli |
From: Tony He <hua...@gm...> - 2021-03-31 07:56:33
|
Antonio Quartulli <a...@un...> 于2021年3月31日周三 下午3:32写道: > Hi, > > On 31/03/2021 09:29, Tony He wrote: > > Hi Arne, > > > > I'm going to test encryption "none" to narrow down this issue, but I > > found your dco branch doesn't support this. > > Can you support? > > For the sake of this test, could you use the ovpn-cli.c tool in the > ovpn-dco/tests folder? Or that's not an option? > Because of cross-compilation, I need to put ovpn-cli.c into the buildroot. This needs some work. But Arne's openvpn dco branch is already in. Anyway, I will have a try. Tony |
From: Arne S. <ar...@rf...> - 2021-03-31 12:11:00
|
Am 31.03.21 um 09:56 schrieb Tony He: > > > Antonio Quartulli <a...@un...> 于2021年3月31日周三 下午3:32写道: > > Hi, > > On 31/03/2021 09:29, Tony He wrote: > > Hi Arne, > > > > I'm going to test encryption "none" to narrow down this issue, but I > > found your dco branch doesn't support this. > > Can you support? > > For the sake of this test, could you use the ovpn-cli.c tool in the > ovpn-dco/tests folder? Or that's not an option? > > Because of cross-compilation, I need to put ovpn-cli.c into the > buildroot. This needs some work. > But Arne's openvpn dco branch is already in. Anyway, I will have a try. I updated the branch to support none but you still need to manually add it to DCO_SUPPORTED_CIPHERS in src/openvpn/networking_linuxdco.h as we do not want to officially support none. Arne |
From: Tony He <hua...@gm...> - 2021-03-31 15:43:00
|
Antonio Quartulli <a...@un...> 于2021年3月31日周三 下午2:32写道: > Hi, > > On 31/03/2021 08:29, Antonio Quartulli wrote: > > A packet dump of the whole session may also help. > > Before taking the dump, I would switch to encryption "none", as it will > help understanding what is going on at all levels. (Assuming the problem > will still appear) > Some issue. Will provide dump or more log when I am free. Tony |
From: Tony He <hua...@gm...> - 2021-04-01 02:38:39
|
Hi Antonio, Arne, According to the dump, this issue is caused by fragment. If I set link-mtu to 1472 in the condition of encryption "none", it's gone. I also can reproduce the fragment in my Linux x86-64 PC and Linux VM . They use kernel 5.4. Fragment affects the performance in the low-end devices. It also consumes more CPU resource in low-end and high-end devices. If I'm not mistaken, we don't need to set link-mtu without dco. Is this a bug? Can you reproduce? Do I still need to upload my dump? If so, maybe I need to provide a link. Tony Tony He <hua...@gm...> 于2021年3月31日周三 下午11:42写道: > > > Antonio Quartulli <a...@un...> 于2021年3月31日周三 下午2:32写道: > >> Hi, >> >> On 31/03/2021 08:29, Antonio Quartulli wrote: >> > A packet dump of the whole session may also help. >> >> Before taking the dump, I would switch to encryption "none", as it will >> help understanding what is going on at all levels. (Assuming the problem >> will still appear) >> > Some issue. Will provide dump or more log when I am free. > > Tony > |
From: Antonio Q. <a...@un...> - 2021-04-01 06:36:07
|
Hi Tony, On 01/04/2021 04:38, Tony He wrote: > Hi Antonio, Arne, > > According to the dump, this issue is caused by fragment. If I set > link-mtu to 1472 in the condition of encryption "none", it's gone. > I also can reproduce the fragment in my Linux x86-64 PC and Linux VM . > They use kernel 5.4. Fragment affects the performance > in the low-end devices. It also consumes more CPU resource in low-end > and high-end devices. If I'm not mistaken, we don't need > to set link-mtu without dco. Is this a bug? Can you reproduce? Do I > still need to upload my dump? If so, maybe I need to provide a link. You told us what you did to fix, but you haven't fully explained what the "broken setup" is. We don't have your configs, so we can't say what is creating the issue in your scenario. What is the MTU on the DCO and on the transport interfaces when the problem shows us? DCO should automatically come up with a working MTU, unless instructed to do differently or unless the transport interface has non standard MTU. Regards, -- Antonio Quartulli |
From: Tony He <hua...@gm...> - 2021-04-01 07:02:27
|
Antonio Quartulli <a...@un...> 于2021年4月1日周四 下午2:35写道: > Hi Tony, > > On 01/04/2021 04:38, Tony He wrote: > > Hi Antonio, Arne, > > > > According to the dump, this issue is caused by fragment. If I set > > link-mtu to 1472 in the condition of encryption "none", it's gone. > > I also can reproduce the fragment in my Linux x86-64 PC and Linux VM . > > They use kernel 5.4. Fragment affects the performance > > in the low-end devices. It also consumes more CPU resource in low-end > > and high-end devices. If I'm not mistaken, we don't need > > to set link-mtu without dco. Is this a bug? Can you reproduce? Do I > > still need to upload my dump? If so, maybe I need to provide a link. > > You told us what you did to fix, but you haven't fully explained what > the "broken setup" is. We don't have your configs, so we can't say what > is creating the issue in your scenario. > server config: root@OpenWrt:/tmp# cat openvpn-sample_server-fragment.conf data-ciphers none auth none topology subnet persist-key persist-tun ca /etc/luci-uploads/cbid.openvpn.sample_server.ca cert /etc/luci-uploads/cbid.openvpn.sample_server.cert dev tun dh /etc/luci-uploads/cbid.openvpn.sample_server.dh ifconfig-pool-persist /tmp/ipp.txt keepalive 10 120 key /etc/luci-uploads/cbid.openvpn.sample_server.key port 1194 proto udp server 10.8.0.0 255.255.255.0 status /tmp/openvpn-status.log verb 3 client config: % cat client-framgment.conf auth none client dev tun proto udp remote 192.168.1.1 1194 resolv-retry infinite nobind persist-key persist-tun ca ca.crt cert client.crt key client.key data-ciphers none verb 2 writepid /var/run/openvpn.pid > What is the MTU on the DCO and on the transport interfaces when the > problem shows us? > % ifconfig ovpn-dco0 ovpn-dco0: flags=81<UP,POINTOPOINT,RUNNING> mtu 1500 inet 10.8.0.2 netmask 255.255.255.0 destination 10.8.0.2 inet6 fe80::3559:b6c1:3fc3:b8cb prefixlen 64 scopeid 0x20<link> unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 1000 (UNSPEC) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1 bytes 134 (134.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 % ifconfig ovpn-dco0 ovpn-dco0: flags=81<UP,POINTOPOINT,RUNNING> mtu 1500 inet 10.8.0.2 netmask 255.255.255.0 destination 10.8.0.2 inet6 fe80::3559:b6c1:3fc3:b8cb prefixlen 64 scopeid 0x20<link> unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen 1000 (UNSPEC) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1 bytes 134 (134.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 log from openvpn client: 2021-04-01 14:57:31 net_iface_mtu_set: mtu 1500 for ovpn-dco0 |
From: Tony He <hua...@gm...> - 2021-04-01 07:03:37
|
sorry, update transport interface. % ifconfig enx00e04c680a44 enx00e04c680a44: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.1.10 netmask 255.255.255.0 broadcast 192.168.1.255 inet6 fe80::ec9b:2258:82ec:3cdb prefixlen 64 scopeid 0x20<link> ether 00:e0:4c:68:0a:44 txqueuelen 1000 (Ethernet) RX packets 10365932 bytes 6963820421 (6.9 GB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 11883693 bytes 11887431595 (11.8 GB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 Tony He <hua...@gm...> 于2021年4月1日周四 下午3:01写道: > > > Antonio Quartulli <a...@un...> 于2021年4月1日周四 下午2:35写道: > >> Hi Tony, >> >> On 01/04/2021 04:38, Tony He wrote: >> > Hi Antonio, Arne, >> > >> > According to the dump, this issue is caused by fragment. If I set >> > link-mtu to 1472 in the condition of encryption "none", it's gone. >> > I also can reproduce the fragment in my Linux x86-64 PC and Linux VM . >> > They use kernel 5.4. Fragment affects the performance >> > in the low-end devices. It also consumes more CPU resource in low-end >> > and high-end devices. If I'm not mistaken, we don't need >> > to set link-mtu without dco. Is this a bug? Can you reproduce? Do I >> > still need to upload my dump? If so, maybe I need to provide a link. >> >> You told us what you did to fix, but you haven't fully explained what >> the "broken setup" is. We don't have your configs, so we can't say what >> is creating the issue in your scenario. >> > server config: > root@OpenWrt:/tmp# cat openvpn-sample_server-fragment.conf > data-ciphers none > auth none > topology subnet > persist-key > persist-tun > ca /etc/luci-uploads/cbid.openvpn.sample_server.ca > cert /etc/luci-uploads/cbid.openvpn.sample_server.cert > dev tun > dh /etc/luci-uploads/cbid.openvpn.sample_server.dh > ifconfig-pool-persist /tmp/ipp.txt > keepalive 10 120 > key /etc/luci-uploads/cbid.openvpn.sample_server.key > port 1194 > proto udp > server 10.8.0.0 255.255.255.0 > status /tmp/openvpn-status.log > verb 3 > > client config: > % cat client-framgment.conf > auth none > client > dev tun > proto udp > remote 192.168.1.1 1194 > resolv-retry infinite > nobind > persist-key > persist-tun > ca ca.crt > cert client.crt > key client.key > data-ciphers none > verb 2 > writepid /var/run/openvpn.pid > > >> What is the MTU on the DCO and on the transport interfaces when the >> problem shows us? >> > % ifconfig ovpn-dco0 > ovpn-dco0: flags=81<UP,POINTOPOINT,RUNNING> mtu 1500 > inet 10.8.0.2 netmask 255.255.255.0 destination 10.8.0.2 > inet6 fe80::3559:b6c1:3fc3:b8cb prefixlen 64 scopeid 0x20<link> > unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen > 1000 (UNSPEC) > RX packets 0 bytes 0 (0.0 B) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 1 bytes 134 (134.0 B) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > % ifconfig ovpn-dco0 > ovpn-dco0: flags=81<UP,POINTOPOINT,RUNNING> mtu 1500 > inet 10.8.0.2 netmask 255.255.255.0 destination 10.8.0.2 > inet6 fe80::3559:b6c1:3fc3:b8cb prefixlen 64 scopeid 0x20<link> > unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 txqueuelen > 1000 (UNSPEC) > RX packets 0 bytes 0 (0.0 B) > RX errors 0 dropped 0 overruns 0 frame 0 > TX packets 1 bytes 134 (134.0 B) > TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 > log from openvpn client: > 2021-04-01 14:57:31 net_iface_mtu_set: mtu 1500 for ovpn-dco0 > |
From: Antonio Q. <a...@un...> - 2021-04-01 07:07:45
|
Hi, Thanks for providing the info. On 01/04/2021 09:01, Tony He wrote: > % ifconfig ovpn-dco0 > ovpn-dco0: flags=81<UP,POINTOPOINT,RUNNING> mtu 1500 > % ifconfig ovpn-dco0 > ovpn-dco0: flags=81<UP,POINTOPOINT,RUNNING> mtu 1500 These values are not what ovpn-dco would set by default. > log from openvpn client: > 2021-04-01 14:57:31 net_iface_mtu_set: mtu 1500 for ovpn-dco0 Mhh, I presume the openvpn2 client still sets 1500 MTU as default, while it should just leave the MTU value alone when no link-mtu was specified by the user. Arne, I presume this is a change that could be applied to your dco branch? Tony, in the meantime, setting the link-mtu to something lower than 1500 is the proper workaround. Thanks. Regards, -- Antonio Quartulli |
From: Antonio Q. <a...@un...> - 2021-04-01 07:29:48
|
On 01/04/2021 09:07, Antonio Quartulli wrote: > Tony, in the meantime, setting the link-mtu to something lower than 1500 > is the proper workaround. Sorry I take this last sentence back. What I was talking all time long was the "tun-mtu". To fix the MTU issue when there is encapsulation, it's the tun-mtu that should be reduced (so that it can fit in the link-mtu after adding the protocol overhead). So the proper workaround should be to reduce the tun-mtu and avoid the default of 1500. Regards, -- Antonio Quartulli |
From: Arne S. <ar...@rf...> - 2021-04-01 10:48:03
|
Am 01.04.21 um 04:38 schrieb Tony He: > Hi Antonio, Arne, > > According to the dump, this issue is caused by fragment. If I set > link-mtu to 1472 in the condition of encryption "none", it's gone. > I also can reproduce the fragment in my Linux x86-64 PC and Linux VM . > They use kernel 5.4. Fragment affects the performance > in the low-end devices. It also consumes more CPU resource in low-end > and high-end devices. If I'm not mistaken, we don't need > to set link-mtu without dco. Is this a bug? Can you reproduce? Do I > still need to upload my dump? If so, maybe I need to provide a link. Yes that is expected. Use tun-mtu 1400. The mtu 1500 leading to fragmentation is a longstanding issue but ist almost entirely masked by mssfix being enabled by default. In your dco to dco setup you probably don't have mssfix on either side unless you explicitly added iptables rules for that. Arne |
From: Gert D. <ge...@gr...> - 2021-04-01 12:10:28
Attachments:
signature.asc
|
Hi, On Thu, Apr 01, 2021 at 12:47:46PM +0200, Arne Schwabe wrote: > In your dco to dco setup you probably > don't have mssfix on either side unless you explicitly added iptables > rules for that. Ah, this is interesting and needs to be documented - "if you want feature <a>, <b>, <c> together with DCO, it needs to be done in iptables". (Of course it makes lots of sense to defer this to iptables etc. on all platforms that have DCO *and* a reasonable firewall layer... dco-win will be interesting) gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany ge...@gr... |
From: Antonio Q. <a...@un...> - 2021-04-01 12:16:42
|
Hi, On 01/04/2021 14:10, Gert Doering wrote: > Hi, > > On Thu, Apr 01, 2021 at 12:47:46PM +0200, Arne Schwabe wrote: >> In your dco to dco setup you probably >> don't have mssfix on either side unless you explicitly added iptables >> rules for that. > > Ah, this is interesting and needs to be documented - "if you want feature > <a>, <b>, <c> together with DCO, it needs to be done in iptables". > > (Of course it makes lots of sense to defer this to iptables etc. on > all platforms that have DCO *and* a reasonable firewall layer... dco-win > will be interesting) Features that are not compatible with DCO are being documented in the code as we go. If you try to explicitly use one of those with DCO, openvpn2 will log a big warning and will tell you that it is switching to non-DCO mode to make sure your connection can work. However, we will definitely make some larger doc explaining what we decided that DCO should not do and what are possible alternatives. Regards, -- Antonio Quartulli |
From: Gert D. <ge...@gr...> - 2021-04-01 12:37:15
Attachments:
signature.asc
|
Hi, On Thu, Apr 01, 2021 at 02:16:25PM +0200, Antonio Quartulli wrote: > > (Of course it makes lots of sense to defer this to iptables etc. on > > all platforms that have DCO *and* a reasonable firewall layer... dco-win > > will be interesting) > > Features that are not compatible with DCO are being documented in the > code as we go. > > If you try to explicitly use one of those with DCO, openvpn2 will log a > big warning and will tell you that it is switching to non-DCO mode to > make sure your connection can work. Which is actually interesting for mssfix, as that is "on by default", so "all configs are incompatible with DCO", by that definition :-) > However, we will definitely make some larger doc explaining what we > decided that DCO should not do and what are possible alternatives. *like* gert -- "If was one thing all people took for granted, was conviction that if you feed honest figures into a computer, honest figures come out. Never doubted it myself till I met a computer with a sense of humor." Robert A. Heinlein, The Moon is a Harsh Mistress Gert Doering - Munich, Germany ge...@gr... |
From: Antonio Q. <a...@un...> - 2021-04-01 12:42:08
|
Hi, On 01/04/2021 14:37, Gert Doering wrote: > Which is actually interesting for mssfix, as that is "on by default", > so "all configs are incompatible with DCO", by that definition :-) hehe Arne can confirm, but I think defaults are adapted to what DCO expects, so mssfix is just disabled by default when DCO is on. I can already hear the arguments that this will create some confusion, but it will be documented :-) It's our chance to get rid of some "legacy". > >> However, we will definitely make some larger doc explaining what we >> decided that DCO should not do and what are possible alternatives. > > *like* > > gert > Regards, -- Antonio Quartulli |
From: Arne S. <ar...@rf...> - 2021-04-01 12:46:06
|
Am 01.04.21 um 14:37 schrieb Gert Doering: > Hi, > > On Thu, Apr 01, 2021 at 02:16:25PM +0200, Antonio Quartulli wrote: >>> (Of course it makes lots of sense to defer this to iptables etc. on >>> all platforms that have DCO *and* a reasonable firewall layer... dco-win >>> will be interesting) >> >> Features that are not compatible with DCO are being documented in the >> code as we go. >> >> If you try to explicitly use one of those with DCO, openvpn2 will log a >> big warning and will tell you that it is switching to non-DCO mode to >> make sure your connection can work. > > Which is actually interesting for mssfix, as that is "on by default", > so "all configs are incompatible with DCO", by that definition :-) Yes. With DCO we currently discovering that the OpenVPN way of doing things with MTU set to 1500, doing mssfix and hoping that this will clamp everything to 1450-overhead, while working quite well has a lot of problems compared to just setting a proper MTU in the first case. Currently setting tun-mtu 1400 manually workaround these problems but we need to figure out a good way to solve this MTU mess. The current working idea is to allow MTU 1500 on receive but use a correct MTU on receive and on the interface (at least on platforms where you can receive larger packets than MTU). But currently we are just on the "we are aware of it" stage. And too busy with other things to actually tackle that too right now. Arne |