bsdrp-users Mailing List for BSD Router Project (Page 3)
Router distribution based on FreeBSD with FFRouting and Bird
Brought to you by:
cochard
You can subscribe to this list here.
2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(5) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2010 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(2) |
Jun
(12) |
Jul
(13) |
Aug
(1) |
Sep
|
Oct
|
Nov
(4) |
Dec
(1) |
2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(7) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(3) |
2012 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(10) |
Sep
(2) |
Oct
(1) |
Nov
(3) |
Dec
(2) |
2013 |
Jan
(3) |
Feb
(6) |
Mar
(12) |
Apr
(10) |
May
(2) |
Jun
|
Jul
(2) |
Aug
(5) |
Sep
|
Oct
|
Nov
|
Dec
(4) |
2014 |
Jan
(3) |
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
(9) |
Jul
(10) |
Aug
|
Sep
|
Oct
(2) |
Nov
(1) |
Dec
(1) |
2015 |
Jan
|
Feb
|
Mar
|
Apr
(3) |
May
(1) |
Jun
(1) |
Jul
(4) |
Aug
|
Sep
|
Oct
(6) |
Nov
|
Dec
|
2016 |
Jan
|
Feb
|
Mar
(2) |
Apr
|
May
(3) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
(2) |
Dec
|
2017 |
Jan
|
Feb
(3) |
Mar
(4) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2018 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
(2) |
Jul
|
Aug
(1) |
Sep
(3) |
Oct
|
Nov
|
Dec
|
2019 |
Jan
|
Feb
(1) |
Mar
(2) |
Apr
(10) |
May
(16) |
Jun
(5) |
Jul
|
Aug
(3) |
Sep
(3) |
Oct
|
Nov
|
Dec
|
2020 |
Jan
(4) |
Feb
(5) |
Mar
(3) |
Apr
(7) |
May
(4) |
Jun
(5) |
Jul
(1) |
Aug
|
Sep
(3) |
Oct
(3) |
Nov
(11) |
Dec
|
2021 |
Jan
(6) |
Feb
(20) |
Mar
(12) |
Apr
|
May
|
Jun
(12) |
Jul
|
Aug
|
Sep
|
Oct
(1) |
Nov
(1) |
Dec
(2) |
2022 |
Jan
(1) |
Feb
|
Mar
(4) |
Apr
(2) |
May
|
Jun
|
Jul
(8) |
Aug
(1) |
Sep
(5) |
Oct
|
Nov
|
Dec
|
2023 |
Jan
|
Feb
(1) |
Mar
(4) |
Apr
(1) |
May
(1) |
Jun
|
Jul
(3) |
Aug
|
Sep
|
Oct
(1) |
Nov
(2) |
Dec
|
2024 |
Jan
|
Feb
(6) |
Mar
(3) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Clemens Z. <cz...@at...> - 2021-06-23 18:50:58
|
On Wed, 2021-06-23 at 20:07 +0200, Clemens Zauner wrote: > > And what about the output of : > > sysctl net.route.algo > > [root@amarklor]/etc# sysctl net.route.algo > net.route.algo.debug_level: 5 > net.route.algo.inet.algo: dpdk_lpm4 > net.route.algo.inet.algo_list: dpdk_lpm4, bsearch4, radix4_lockless, > radix4 Ah I see; thats a new forwarding scheme. Is it possible to switch back to radix4(_lockless) in a more or less hitless fashion (this is a Boarder-router, full table, forwarding 'real' traffic)? If yes, I give It a try - not during main-time, but in the evenings. If I require a reboot ore something disruptive , I'm willing to give It a try; but I have to engineer an excuse for a short-notice-troube- ticket ('Increased sunspot activity', or something along that way). And, as I'm on my 'asking stupid questions' - train: what is the difference between radix4 and radix4_lockless? I got the memo, that bsearch(4) is *not* the way to go for large routing-tables, but I'm somwhat lost regarding the difference between the 'normal' and '_lockless' one. Yes, I saw that radix_* is slower, especially also due to high cahce- usage. But: better slow and functional. Besides: Epyc System here; slightly larger L3-Cache than the Core-i7 which was used for the benchmarks I found online. Multipath-routing is not a requirement for me. I can handle my flows without residing to multipathing (assuming that means multiple active gateways for the same destinations). As this seems to be turned on here: [root@amarklor]/etc# sysctl net.route.multipath net.route.multipath: 1 cu Clemens. |
From: Clemens Z. <cz...@at...> - 2021-06-23 18:07:44
|
On Wed, 2021-06-23 at 19:32 +0200, Olivier Cochard-Labbé wrote: > On Wed, Jun 23, 2021 at 6:38 PM Clemens Zauner <cz...@at...> > wrote: > > [root@amarklor]/etc# netstat -4rnW | grep 51.15.0.0 > > 51.15.0.0/17 149.6.174.241 UG1 8 1500 cxl0 > > 51.15.0.0/16 149.6.174.241 UG1 8 1500 cxl0 > > > > [root@amarklor]/etc# netstat -4onW | grep 51.15.0.0 > > > > > And what about the output of : > sysctl net.route.algo [root@amarklor]/etc# sysctl net.route.algo net.route.algo.debug_level: 5 net.route.algo.inet.algo: dpdk_lpm4 net.route.algo.inet.algo_list: dpdk_lpm4, bsearch4, radix4_lockless, radix4 net.route.algo.inet6.algo: dpdk_lpm6 net.route.algo.inet6.algo_list: dpdk_lpm6, radix6_lockless, radix6 net.route.algo.fib_max_sync_delay_ms: 1000 net.route.algo.bucket_change_threshold_rate: 500 net.route.algo.bucket_time_ms: 50 [root@amarklor]/etc# I remember, when the machine boots, there is a message about changing to dpdk_lpm 4 or 6 or something like that. A message I've never seen before, And that I should definitively google what that means. I assume, 'lpm' means 'longest prefix match', correct? cu Clemens. |
From: Olivier Cochard-L. <ol...@co...> - 2021-06-23 17:33:06
|
On Wed, Jun 23, 2021 at 6:38 PM Clemens Zauner <cz...@at...> wrote: > > > [root@amarklor]/etc# netstat -4rnW | grep 51.15.0.0 > 51.15.0.0/17 149.6.174.241 UG1 8 1500 cxl0 > 51.15.0.0/16 149.6.174.241 UG1 8 1500 cxl0 > > [root@amarklor]/etc# netstat -4onW | grep 51.15.0.0 > > And what about the output of : sysctl net.route.algo Thanks |
From: Clemens Z. <cz...@at...> - 2021-06-23 16:37:35
|
On Tue, 2021-06-22 at 19:02 +0200, Olivier Cochard-Labbé wrote: > On Tue, Jun 22, 2021 at 5:17 PM Clemens Zauner <cz...@at...> > wrote: > > Hi Clemens, > indeed there is a problem somewhere, like a not synchronized RIB and > FIB. > > Could you display the output of those commands: > sysctl net.route.algo > netstat -4rnW | grep 51.15.0.0 > netstat -4onW | grep 51.15.0.0 > [root@amarklor]/etc# netstat -4rnW | grep 51.15.0.0 51.15.0.0/17 149.6.174.241 UG1 8 1500 cxl0 51.15.0.0/16 149.6.174.241 UG1 8 1500 cxl0 [root@amarklor]/etc# netstat -4onW | grep 51.15.0.0 [root@amarklor]/etc# Trace takes the 'wrong direction': [root@amarklor]/etc# traceroute 51.15.183.144 traceroute to 51.15.183.144 (51.15.183.144), 64 hops max, 40 byte packets 1 em2-910.panem.atnoc.net (193.239.188.36) 0.526 ms 0.255 ms 0.193 ms I'm somehow lost, how the machine does that (btw: 193.239.188.36 is not reachable via the chelsio, but via igb0, so there is not rewriting or switching going on the the chelsio-T5. BGP/FRR reflects the state of the kernel routing table: amarklor.atnoc.net# sh ip route 51.15.183.144 Routing entry for 51.15.0.0/16 Known via "bgp", distance 20, metric 25061, best Last update 01w1d22h ago * 149.6.174.241, via cxl0, weight 1 amarklor.atnoc.net# sh ip bgp 51.15.0.0/16 BGP routing table entry for 51.15.0.0/16 Paths: (1 available, best #1, table default) Advertised to non peer-group peers: 193.239.188.34 193.239.188.36 193.239.188.214 193.239.188.216 174 12876 149.6.174.241 (metric 1) from 149.6.174.241 (154.26.32.25) Origin IGP, metric 25061, valid, external, best (First path received) Community: 174:21101 174:22008 Last update: Mon Jun 14 19:07:00 2021 amarklor.atnoc.net# For what It's worth, for all prefixes displaying the behaviour, there ist alsways a longer prefix for the same network in the routing-table. As seen above: 51.15.0.0/16 and 51.15.0.0/17. But not all prefixes with a more specific show this behaviour. for instance: [root@amarklor]/etc# netstat -4rnW | grep 88.10..0.0 88.100.0.0/15 149.6.174.241 UG1 8 1500 cxl0 88.100.0.0/14 149.6.174.241 UG1 8 1500 cxl0 88.102.0.0/16 149.6.174.241 UG1 8 1500 cxl0 88.104.0.0/14 149.6.174.241 UG1 8 1500 cxl0 88.104.0.0/13 149.6.174.241 UG1 8 1500 cxl0 88.108.0.0/14 149.6.174.241 UG1 8 1500 cxl0 or: [root@amarklor]/etc# netstat -4rnW | grep 149.12.0.0 149.12.0.0/20 149.6.174.241 UG1 8 1500 cxl0 149.12.0.0/15 149.6.174.241 UG1 8 1500 cxl0 The 'netstat -4onW' is rather compact, I post the whole, for reference: [root@amarklor]/etc# netstat -4onW Nexthop data Internet: Idx Type IFA Gateway Flags Use Mtu Netif Addrif Refcnt Prepend 1 v4/resolve 127.0.0.1 lo0/resolve H 76508 16384 lo0 2 2 v4/resolve 193.239.188.197 lo1/resolve H 0 16384 lo1 2 3 v4/resolve 149.6.174.242 cxl0/resolve 192653 1500 cxl0 3 4 v4/resolve 127.0.0.1 lo0/resolve HS 0 16384 lo0 cxl0 2 5 v4/gw 127.0.0.1 127.0.0.1 G1B 3546 16384 lo0 7 6 v4/resolve 193.239.188.38 igb0/resolve 2893089 1500 igb0 5 7 v4/resolve 127.0.0.1 lo0/resolve HS 0 16384 lo0 igb0 2 8 v4/gw 149.6.174.242 149.6.174.241 G1 176047 1500 cxl0 598127 9 v4/resolve 193.239.188.217 igb2/resolve 709843 9216 igb2 3 10 v4/resolve 127.0.0.1 lo0/resolve HS 0 16384 lo0 igb2 2 11 v4/resolve 193.239.188.215 igb3/resolve 708796 9216 igb3 3 12 v4/resolve 127.0.0.1 lo0/resolve HS 0 16384 lo0 igb3 2 13 v4/resolve 95.129.200.114 vlan960/resolve 2450 1500 vlan960 2 14 v4/resolve 127.0.0.1 lo0/resolve HS 0 16384 lo0 vlan960 2 15 v4/gw 95.129.200.114 95.129.200.123 G1 0 1500 vlan960 4 16 v4/gw 95.129.200.114 95.129.200.118 G1 0 1500 vlan960 5 17 v4/gw 95.129.200.114 95.129.200.124 G1 32 1500 vlan960 3 18 v4/gw 193.239.188.215 193.239.188.214 G1 54272925 9216 igb3 7 19 v4/gw 95.129.200.114 95.129.200.113 GH1 3 1500 vlan960 2 20 v4/gw 95.129.200.114 95.129.200.117 G1 95310 1500 vlan960 14 21 v4/gw 95.129.200.114 95.129.200.120 G1 0 1500 vlan960 8 22 v4/gw 95.129.200.114 95.129.200.117 GH1 0 1500 vlan960 7 23 v4/gw 95.129.200.114 95.129.200.125 G1 3613 1500 vlan960 35 24 v4/gw 95.129.200.114 95.129.200.119 G1 0 1500 vlan960 2 25 v4/gw 193.239.188.215 193.239.188.214 GH1 2662 9216 igb3 4 26 v4/gw 193.239.188.217 193.239.188.216 G1 98212 9216 igb2 4 27 v4/gw 193.239.188.217 193.239.188.216 GH1 1895 9216 igb2 5 28 v4/gw 193.239.188.38 193.239.188.34 G1 31251 1500 igb0 154517 29 v4/gw 193.239.188.38 193.239.188.36 G1 16214 1500 igb0 79933 30 v4/gw 149.6.174.242 149.6.174.241 G1 177 1500 cxl0 2 HTH Clemens. |
From: Olivier Cochard-L. <ol...@co...> - 2021-06-22 17:03:20
|
On Tue, Jun 22, 2021 at 5:17 PM Clemens Zauner <cz...@at...> wrote: Hi Clemens, indeed there is a problem somewhere, like a not synchronized RIB and FIB. Could you display the output of those commands: sysctl net.route.algo netstat -4rnW | grep 51.15.0.0 netstat -4onW | grep 51.15.0.0 Thanks, Olivier |
From: Clemens Z. <cz...@at...> - 2021-06-22 15:17:09
|
On Tue, 2021-06-22 at 12:21 +0300, Lyubomir Yotov wrote: > Hi Clemens, > What version of BSDRP are you using? One of the latest, an uname -a gives me: FreeBSD amarklor.atnoc.net 14.0-CURRENT FreeBSD 14.0-CURRENT BSDRP- AMD64 amd64 > I would suggest that you check your configuration for some errors in > peers definitions in the first place. A help from a colleague of > yours would help a lot (I have had some stupid typos like using /29 > instead of /30 and the sort, also metrics here and there in route- > maps could be good place to look for errors) These are fine, I'm doing BGP on ISP Level for >20 years; I' seeing the same Problem on other prefixes too. For that matter, I just got a 'looping' -Complaint right now - I spare the BGP Info as a first. Im seeing the following information in the BSD kernel-routing table (for the IP I've got the compaint for): [root@amarklor]/etc# route -n get 149.13.75.104 route to: 149.13.75.104 destination: 149.12.0.0 mask: 255.254.0.0 gateway: 149.6.174.241 fib: 0 interface: cxl0 flags: <UP,GATEWAY,DONE,PROTO1> recvpipe sendpipe ssthresh rtt,msec mtu weight expire 0 0 0 0 1500 1 0 Which clearly indicates a next-Hop at 149.6.174.241 - the router of our upstream. But If I 'traceroute' the IP: [root@amarklor]/etc# traceroute -n 149.13.75.104 traceroute to 149.13.75.104 (149.13.75.104), 64 hops max, 40 byte packets 1 193.239.188.36 0.256 ms 0.333 ms 0.368 ms 2 193.239.188.38 0.270 ms 0.096 ms 0.118 ms The BSD-Kernel sends the Packets towards 193.239.188.36. Which is completely wrong. And not what the Kernel FIB klaims what will happen to the packets. I spare you with the BGP-related entries right now; I checked thema, The BGP-Paths seems correct, and all the routers in question produce correct paths (If I look at the BGP routes, everythin is correct, If I look at the kernel-routing-tables, as well the RIBs on the Ciscos: They reflect the BGP Information in a consitent manner) But for what Its worth: The BSD-kernel seems to forward the Packet towards another router. If I 'grep' all routes (from a netstat -rn) for the range in quesion one gets the following: 149.12.0.0/20 149.6.174.241 UG1 cxl0 149.12.0.0/15 149.6.174.241 UG1 cxl0 149.12.17.0/24 149.6.174.241 UG1 cxl0 149.12.70.0/24 149.6.174.241 UG1 cxl0 149.12.128.0/24 149.6.174.241 UG1 cxl0 149.12.128.0/22 149.6.174.241 UG1 cxl0 149.12.129.0/24 149.6.174.241 UG1 cxl0 149.12.130.0/24 149.6.174.241 UG1 cxl0 149.12.131.0/24 149.6.174.241 UG1 cxl0 149.12.183.0/24 149.6.174.241 UG1 cxl0 149.12.184.0/22 149.6.174.241 UG1 cxl0 149.12.200.0/24 149.6.174.241 UG1 cxl0 149.12.201.0/24 149.6.174.241 UG1 cxl0 149.12.224.0/24 149.6.174.241 UG1 cxl0 149.12.227.0/24 149.6.174.241 UG1 cxl0 149.12.241.0/24 193.239.188.34 UG1 igb0 149.12.242.0/23 193.239.188.34 UG1 igb0 149.12.244.0/24 149.6.174.241 UG1 cxl0 149.13.0.0/24 149.6.174.241 UG1 cxl0 149.13.2.0/24 149.6.174.241 UG1 cxl0 149.13.3.0/24 149.6.174.241 UG1 cxl0 149.13.16.0/24 149.6.174.241 UG1 cxl0 149.13.18.0/23 149.6.174.241 UG1 cxl0 149.13.24.0/23 149.6.174.241 UG1 cxl0 149.13.24.0/22 149.6.174.241 UG1 cxl0 149.13.25.0/24 149.6.174.241 UG1 cxl0 149.13.26.0/23 149.6.174.241 UG1 cxl0 149.13.32.0/24 193.239.188.34 UG1 igb0 149.13.65.0/24 149.6.174.241 UG1 cxl0 149.13.69.0/24 149.6.174.241 UG1 cxl0 149.13.70.0/24 149.6.174.241 UG1 cxl0 149.13.72.0/24 149.6.174.241 UG1 cxl0 149.13.78.0/23 193.239.188.34 UG1 igb0 149.13.80.0/23 193.239.188.34 UG1 igb0 149.13.82.0/24 149.6.174.241 UG1 cxl0 149.13.83.0/24 193.239.188.34 UG1 igb0 149.13.90.0/24 149.6.174.241 UG1 cxl0 149.13.94.0/24 149.6.174.241 UG1 cxl0 149.13.113.0/24 149.6.174.241 UG1 cxl0 149.13.115.0/24 149.6.174.241 UG1 cxl0 149.13.145.0/24 149.6.174.241 UG1 cxl0 149.13.146.0/24 149.6.174.241 UG1 cxl0 149.13.147.0/24 149.6.174.241 UG1 cxl0 149.13.148.0/22 149.6.174.241 UG1 cxl0 149.13.152.0/22 149.6.174.241 UG1 cxl0 149.13.156.0/22 149.6.174.241 UG1 cxl0 149.13.176.0/24 149.6.174.241 UG1 cxl0 There ar in fact some more-spefics within 149.12.0.0/15, but none for 149.13.75.104. So 149.13.75.104 is handled by 149.12.0.0/15 (compare route -n get 149.13.75.104) - but despite the *korrekt* information in the FIB, the Packets get forwardet via 193.239.188.36 (even though this Border is not referenced even once in the FIB for 149.12.0.0/15, and more specific prefixes within this network. > Another thing you could try is to reconfigure the peers one by one > but not copy and paste. Take some time to make the configurations > again. > Most probably it is a matter of configuration. > The Configuration seems pretty solid, to me; Which part could I've missed leding to packet-forwarding not matching the kernels fib? I do have *some* ipfw rules, but no redirects in there (filtering of RFC1918 (rules numered 3000), denying AS35339 SRC-Address Packets in via Upstream-Interfaces (rules 2000), such stuff. Bog-Standard basic network security rules, Required by some BCPs. And Im Blocking TR69-Syns towards our dialin-ranges (rules 04001 und 04002). Ah yes, and I'v enabled ipfw_netflow. Redirects are disabled: net.inet.ip.redirect: 0 complete ipfw show output as follows: 00100 666753 70419169 allow ip from any to any via lo0 00200 0 0 deny ip from any to 127.0.0.0/8 00300 8 4768 deny ip from 127.0.0.0/8 to any 00400 0 0 deny ip from any to ::1 00500 0 0 deny ip from ::1 to any 00600 0 0 allow ipv6-icmp from :: to ff02::/16 00700 1264589 84841128 allow ipv6-icmp from fe80::/10 to fe80::/10 00800 704954 52076976 allow ipv6-icmp from fe80::/10 to ff02::/16 00900 16562562 2242736622 allow ipv6-icmp from any to any icmp6types 1 01000 1937949 1330602388 allow ipv6-icmp from any to any icmp6types 2,135,136 01000 70475719 73830970321 allow ip from me to any 01001 91537 13532464 allow ip from 95.129.200.0/21 to me 01002 3829904 334555883 allow ip from 193.239.188.0/23 to me 02000 1481 142567 deny ip from 95.129.200.0/21 to any in via cxl0 02001 257 22873 deny ip from 193.239.188.0/23 to any in via cxl0 02002 552 49209 deny ip from 185.65.232.0/22 to any in via cxl0 02003 0 0 deny ip6 from 2a02:b18::/32 to any in via cxl0 03001 552891 51238709 reject ip from any to 10.0.0.0/8 03002 4753 431799 reject ip from any to 172.16.0.0/12 03003 5018090 381259117 reject ip from any to 192.168.0.0/16 04001 833432 33475188 unreach port tcp from any to 95.129.200.0/21 7547 in via cxl0 setup 04002 33411 1409920 unreach port tcp from any to 185.65.232.0/22 7547 in via cxl0 setup 10000 73384539658 34867129902846 ngtee 9995 ip from any to any 65000 73552389878 34940200731572 allow ip from any to any I'm a decades user of *BSDs, I've never seen such a forwarding behaviour in my life. Is there something 'new' in 13 / or 14 Releases, I'm not aware of? some kind of caching, or new route-lookup-mechanic (going haywire, here, obviously)? If yes: Can that be disabled? > Regards, > > Lyubo > > cu Clemens. PS: I've a Hard time playing with this spefic Box, as she handles multiple GBits/s of Upstream, but I have another one, which I can setup for testing. I did intend to relayce the other upstream too (currently Cisco), but I'm currently refraining from that, until I know whats going on here. PPS: I can provide more-or-less complete configurations, If required; I do not have really sensitive information in those. |
From: Lyubomir Y. <l....@gm...> - 2021-06-22 09:22:15
|
Hi Clemens, What version of BSDRP are you using? I would suggest that you check your configuration for some errors in peers definitions in the first place. A help from a colleague of yours would help a lot (I have had some stupid typos like using /29 instead of /30 and the sort, also metrics here and there in route-maps could be good place to look for errors) Another thing you could try is to reconfigure the peers one by one but not copy and paste. Take some time to make the configurations again. Most probably it is a matter of configuration. Regards, Lyubo On Fri, 18 Jun 2021 at 16:05, Clemens Zauner <cz...@at...> wrote: > > We changed a Borderrouter to an BSDRP, 2 Weeks ago. I have some strange > issues. There ist one BSDRP/FRR and one Cisco involved, on this part of > the Network: > > We are AS35339, One Upstream is AS179 the oder is AS1299 > > For Instance, now I'm having a 'looping' towards 51.15.183.144 > (www.lymo.fr). My core finds a valid path to the BSDR (facing AS179), > routing packets towards this machine. > > On the BSDRP-Box: > > amarklor.atnoc.net# sh ip route 51.15.183.144 > Routing entry for 51.15.0.0/16 > Known via "bgp", distance 20, metric 25061, best > Last update 3d17h44m ago > * 149.6.174.241, via cxl0, weight 1 > > amarklor.atnoc.net# sh ip bgp 51.15.0.0/16 > BGP routing table entry for 51.15.0.0/16 > Paths: (1 available, best #1, table default) > Advertised to non peer-group peers: > 193.239.188.34 193.239.188.36 193.239.188.214 193.239.188.216 > 174 12876 > 149.6.174.241 (metric 1) from 149.6.174.241 (154.26.32.25) > Origin IGP, metric 25061, valid, external, best (First path > received) > Community: 174:21101 174:22008 > Last update: Mon Jun 14 19:07:01 2021 > amarklor.atnoc.net# > > > Which is also reflected in the kernel routing-table: > [root@amarklor]/etc# route -n get 51.15.183.144 > route to: 51.15.183.144 > destination: 51.15.0.0 > mask: 255.255.0.0 > gateway: 149.6.174.241 > fib: 0 > interface: cxl0 > flags: <UP,GATEWAY,DONE,PROTO1> > recvpipe sendpipe ssthresh rtt,msec mtu weight expire > 0 0 0 0 1500 1 0 > [root@amarklor]/etc# > > > *BUT*: > [root@amarklor]/etc# traceroute -n 51.15.183.144 > traceroute to 51.15.183.144 (51.15.183.144), 64 hops max, 40 byte > packets > 1 193.239.188.36 0.486 ms 0.297 ms 0.271 ms > 2 193.239.188.38 0.341 ms 0.154 ms 0.094 ms > 3 193.239.188.36 0.312 ms 0.382 ms 0.300 ms > 4 193.239.188.38 0.295 ms 0.260 ms 0.254 ms > 5 193.239.188.36 0.563 ms 0.512 ms 0.558 ms > > With 193.239.188.36 beeing the Cisco towards AS1299; and 193.239.188.38 > beeing the BSDRP. Why does BSDRP send the packets to 193.239.188.36, > even if an 'route get' clearly shows the correct nex-hop 149.6.174.241 > within AS179? > > > > The routing-Inforation on the Cisco reflects the Cisco's FIB: > gant#sh ip route 51.15.183.144 > Routing entry for 51.15.0.0/16 > Known via "bgp 35339", distance 200, metric 25061 > Tag 174, type internal > Last update from 149.6.174.241 3d18h ago > Routing Descriptor Blocks: > * 149.6.174.241, from 193.239.188.38, 3d18h ago > Route metric is 25061, traffic share count is 1 > AS Hops 2 > Route tag 174 > MPLS label: none > gant#sh ip bgp 51.15.0.0/16 > BGP routing table entry for 51.15.0.0/16, version 66104870 > Paths: (2 available, best #1, table default) > Not advertised to any peer > Refresh Epoch 1 > 174 12876 > 149.6.174.241 (metric 20) from 193.239.188.38 (193.239.188.197) > Origin IGP, metric 25061, localpref 100, valid, internal, best > Community: 174:21101 174:22008 > rx pathid: 0, tx pathid: 0x0 > Refresh Epoch 1 > 1299 3356 12876 > 213.248.76.28 from 213.248.76.28 (2.255.254.134) > Origin IGP, localpref 100, valid, external > Community: 1299:20000 > rx pathid: 0, tx pathid: 0 > gant# > > Am I missing something? > Clemens. > > _______________________________________________ > Bsdrp-users mailing list > Bsd...@li... > https://lists.sourceforge.net/lists/listinfo/bsdrp-users > |
From: Clemens Z. <cz...@at...> - 2021-06-18 13:05:26
|
We changed a Borderrouter to an BSDRP, 2 Weeks ago. I have some strange issues. There ist one BSDRP/FRR and one Cisco involved, on this part of the Network: We are AS35339, One Upstream is AS179 the oder is AS1299 For Instance, now I'm having a 'looping' towards 51.15.183.144 (www.lymo.fr). My core finds a valid path to the BSDR (facing AS179), routing packets towards this machine. On the BSDRP-Box: amarklor.atnoc.net# sh ip route 51.15.183.144 Routing entry for 51.15.0.0/16 Known via "bgp", distance 20, metric 25061, best Last update 3d17h44m ago * 149.6.174.241, via cxl0, weight 1 amarklor.atnoc.net# sh ip bgp 51.15.0.0/16 BGP routing table entry for 51.15.0.0/16 Paths: (1 available, best #1, table default) Advertised to non peer-group peers: 193.239.188.34 193.239.188.36 193.239.188.214 193.239.188.216 174 12876 149.6.174.241 (metric 1) from 149.6.174.241 (154.26.32.25) Origin IGP, metric 25061, valid, external, best (First path received) Community: 174:21101 174:22008 Last update: Mon Jun 14 19:07:01 2021 amarklor.atnoc.net# Which is also reflected in the kernel routing-table: [root@amarklor]/etc# route -n get 51.15.183.144 route to: 51.15.183.144 destination: 51.15.0.0 mask: 255.255.0.0 gateway: 149.6.174.241 fib: 0 interface: cxl0 flags: <UP,GATEWAY,DONE,PROTO1> recvpipe sendpipe ssthresh rtt,msec mtu weight expire 0 0 0 0 1500 1 0 [root@amarklor]/etc# *BUT*: [root@amarklor]/etc# traceroute -n 51.15.183.144 traceroute to 51.15.183.144 (51.15.183.144), 64 hops max, 40 byte packets 1 193.239.188.36 0.486 ms 0.297 ms 0.271 ms 2 193.239.188.38 0.341 ms 0.154 ms 0.094 ms 3 193.239.188.36 0.312 ms 0.382 ms 0.300 ms 4 193.239.188.38 0.295 ms 0.260 ms 0.254 ms 5 193.239.188.36 0.563 ms 0.512 ms 0.558 ms With 193.239.188.36 beeing the Cisco towards AS1299; and 193.239.188.38 beeing the BSDRP. Why does BSDRP send the packets to 193.239.188.36, even if an 'route get' clearly shows the correct nex-hop 149.6.174.241 within AS179? The routing-Inforation on the Cisco reflects the Cisco's FIB: gant#sh ip route 51.15.183.144 Routing entry for 51.15.0.0/16 Known via "bgp 35339", distance 200, metric 25061 Tag 174, type internal Last update from 149.6.174.241 3d18h ago Routing Descriptor Blocks: * 149.6.174.241, from 193.239.188.38, 3d18h ago Route metric is 25061, traffic share count is 1 AS Hops 2 Route tag 174 MPLS label: none gant#sh ip bgp 51.15.0.0/16 BGP routing table entry for 51.15.0.0/16, version 66104870 Paths: (2 available, best #1, table default) Not advertised to any peer Refresh Epoch 1 174 12876 149.6.174.241 (metric 20) from 193.239.188.38 (193.239.188.197) Origin IGP, metric 25061, localpref 100, valid, internal, best Community: 174:21101 174:22008 rx pathid: 0, tx pathid: 0x0 Refresh Epoch 1 1299 3356 12876 213.248.76.28 from 213.248.76.28 (2.255.254.134) Origin IGP, localpref 100, valid, external Community: 1299:20000 rx pathid: 0, tx pathid: 0 gant# Am I missing something? Clemens. |
From: Paulo F. <pa...@nl...> - 2021-06-04 16:48:19
|
Hi guys. Could you please include lcdrpoc port in BSDRP? LCDproc is very useful to who is using appliance like Netgate, Checkpoint, ServerU and others. LCDproc is available in PFsense and in brand new OPNsense. The LCDproc port for FreeBSd is locate in ports tree: /usr/ports/sysutils/lcdproc and can be easily configurable for those appliance in that way: # LCDd: (Driver=sdeclcd) cd /usr/local/etc cp LCDd.conf.sample LCDd.conf sed -i '.orig' 's|Driver=curses|Driver=sdeclcd|' LCDd.conf sed -i '' 's|#GoodBye="Thanks for using"|GoodBye="The Power to Serve"|' LCDd.conf sed -i '' 's|#GoodBye=" LCDproc!"|GoodBye=" FreeBSD!"|' LCDd.conf cp lcdexec.conf.sample lcdexec.conf cp lcdproc.conf.sample lcdproc.conf sysrc LCDd_enable="YES" sysrc lcdproc_enable="YES" sysrc lcdexec_enable="YES" service LCDd start service lcdexec start service lcdproc start Thanks, Paulo. -- “Because there is no army that can hold back an economic principle whose time has come.” John Donovan, AT&T Skype: paulo.dcap |
From: Olivier Cochard-L. <ol...@co...> - 2021-03-29 22:19:38
|
On Sun, Mar 28, 2021 at 6:46 PM Paulo Fragoso <pa...@nl...> wrote: > Hi guys. > > Is possible to include nc (the arbitrary TCP and UDP connections and > listens) command in BSDRP? It is in /usr/bin/nc in original FreeBSD > installation. > > Hi, done. the nightly version includes it: https://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/nightly/2021-03-30/amd64/ Regards, Olivier |
From: Paulo F. <pa...@nl...> - 2021-03-28 16:45:33
|
Hi guys. Is possible to include nc (the arbitrary TCP and UDP connections and listens) command in BSDRP? It is in /usr/bin/nc in original FreeBSD installation. Recently was included ncat command which is pretty good. Thanks, Paulo. -- “Because there is no army that can hold back an economic principle whose time has come.” John Donovan, AT&T Skype: paulo.dcap |
From: Lyubomir Y. <l....@gm...> - 2021-03-17 10:42:20
|
Hi again, No one here to share ideas? Lyubomir On Tue, 9 Mar 2021 at 19:03, Lyubomir Yotov <l....@gm...> wrote: > Hi, > I would like to know if dummynet is a good solution for traffic shaper. > The total volumes on a router can go from 500Mbps to 3-4 Gbps and the pps > can go up to several millions (could reach ~1.2Mpps @700Mbps) . Do you have > any optimization that could improve the performance of dummynet. > My tests with dummynet so far are with different pipes per different > segments (regardless they have the same or different capacity) and then > using queues. An example is: > ipfw pipe 100 config bw 90Mbit/s > ipfw pipe 101 config bw 90Mbit/s > ipfw pipe 102 config bw 80Mbit/s > ipfw pipe 103 config bw 80Mbit/s > ipfw queue 100 config pipe 100 > ipfw queue 101 config pipe 101 > ipfw queue 102 config pipe 100 > ipfw queue 103 config pipe 101 > ipfw add 100 queue 100 ip from any to any xmit bce1.100 out > ipfw add 101 queue 101 ip from any to any recv bce1.100 in > ipfw add 102 queue 102 ip from any to any xmit bce1.101 out > ipfw add 103 queue 103 ip from any to any recv bce1.101 in > Do you have a good way to plot the state of the shaper? > If dummynet is not suitable I would appreciate any advice on other options. > I have Chelsio card on the router and I read somewhere that it might be > possible to do shaping with them but I still have to figure out how to do > it. Any information will be great. > > Regards, > Lyubomir > |
From: Lyubomir Y. <l....@gm...> - 2021-03-09 17:03:57
|
Hi, I would like to know if dummynet is a good solution for traffic shaper. The total volumes on a router can go from 500Mbps to 3-4 Gbps and the pps can go up to several millions (could reach ~1.2Mpps @700Mbps) . Do you have any optimization that could improve the performance of dummynet. My tests with dummynet so far are with different pipes per different segments (regardless they have the same or different capacity) and then using queues. An example is: ipfw pipe 100 config bw 90Mbit/s ipfw pipe 101 config bw 90Mbit/s ipfw pipe 102 config bw 80Mbit/s ipfw pipe 103 config bw 80Mbit/s ipfw queue 100 config pipe 100 ipfw queue 101 config pipe 101 ipfw queue 102 config pipe 100 ipfw queue 103 config pipe 101 ipfw add 100 queue 100 ip from any to any xmit bce1.100 out ipfw add 101 queue 101 ip from any to any recv bce1.100 in ipfw add 102 queue 102 ip from any to any xmit bce1.101 out ipfw add 103 queue 103 ip from any to any recv bce1.101 in Do you have a good way to plot the state of the shaper? If dummynet is not suitable I would appreciate any advice on other options. I have Chelsio card on the router and I read somewhere that it might be possible to do shaping with them but I still have to figure out how to do it. Any information will be great. Regards, Lyubomir |
From: Lyubomir Y. <l....@gm...> - 2021-03-09 16:52:11
|
Thanks Mike, I will try with the sysctl settings on my next reboot (hopefully in the next month or so). About the shaping I will post another mail, just to keep the topics separate. Thanks once again. Regards, Lyubomir On Tue, 9 Mar 2021 at 15:24, mike tancsa <mi...@se...> wrote: > Hi, > > I havent tried shaping. If you are getting both blocks on the NIC > firewall and ipfw its possible the packets ipfw are stopping are > fragments or packets with bad checksums. You can have the NIC drop the > bad checksum traffic too with that sysctl setting in loader.conf > > ---Mike > > On 3/9/2021 8:12 AM, Lyubomir Yotov wrote: > > Hi Mike, > > I tried with the VLAN option but it appears to be valid only for the > > switch action: > > #cxgbetool t5nex0 filter 10 iport 0 action pass dip 192.168.1.122 > > dport 80 vlan "=100" > > cxgbetool: port, dmac, smac, vlan, and nat only make sense with > > "action switch" > > Anyway I have implemented this and it works fine. I noticed that there > > are some hits on ipfw rule that used to take care of it. Do you have > > any idea why? > > Have tried to do some sort of shaping in the card? > > > > Regards, > > > > Lyubo > > > > On Mon, 8 Mar 2021 at 13:52, Lyubomir Yotov <l....@gm... > > <mailto:l....@gm...>> wrote: > > > > Hi Mike, > > Thanks a lot! > > > > Regards, > > > > Lyubomir > > > > On Mon, 8 Mar 2021 at 12:26, mike tancsa <mi...@se... > > <mailto:mi...@se...>> wrote: > > > > On 3/8/2021 3:17 AM, Lyubomir Yotov wrote: > > > Hi Mike, > > > Thanks for the quick response and provided information. I > > currently > > > have only one interface (in and out). I will try to use the > > vlan > > > option as well to be more precise. My rule might look like: > > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip > > 192.168.1.122 > > > dport 23 vlan 100 > > > Just to be on the safe side, if I add only the above drop > > rule in the > > > firewall I won't need explicit "allow all" in the end? > > > > Hi Lyubomir, > > > > Correct, its *not* like pf where its default block. You > > can then > > use cxgbetool t5nex0 filter list to see what hits. Actually, > > maybe to > > be on the safe side at first, instead of action drop add in > > action pass > > and then hit the rule to see if its being hit or not the way > > you expect. > > You will see the counter go up. then delete it and add it back > > as action > > drop when you are confident its going to do just what you want > > it to do. > > > > > > > > I don't have "hw.cxgbe.attack_filter" and > > > "hw.cxgbe.drop_pkts_with_l3_errors". These will either > > appear after I > > > add a rule or if a change the firmware. I will check after > > adding a rule. > > > > These are /boot/loader.conf values. See man cxl to see what > > they do. > > > > ---Mike > > > > > > > > > > Regards, > > > Lyubomir > > > > > > > > > On Sun, 7 Mar 2021 at 19:15, mike tancsa <mi...@se... > > <mailto:mi...@se...> > > > <mailto:mi...@se... <mailto:mi...@se...>>> wrote: > > > > > > Hi, > > > I am using the T5 firewall features on FreeBSD 11 > > and 12 in > > > production and it works great! > > > > > > On 3/7/2021 10:41 AM, Lyubomir Yotov wrote: > > > > - is it safe to add rules on the fly in BSDRP? > > > I add and remove rules on the fly all the time. > > > > - is it safe to implement drop only rules on a > > production server > > > > without breaking the other traffic (should I have an > > allow-all > > > in the > > > > end)? > > > > I would like to test dropping all packets incoming on > > cxl0 from any > > > > host to host 192.168.1.122 with destination port 23. I > > suppose a > > > rule > > > > like the following will do the job: > > > > > > > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip > > 192.168.1.122 > > > > dport 23 > > > > > > > Careful of the orientation. If you have 2 ports, the > > iport makes a > > > difference as to whether the rule gets hit or not. > > > > > > > > > > If I want this persistent I should create a script > > probably and > > > start > > > > it with the system boot? > > > Yes. I have yet to come up with a nice interface to do > > this. For some > > > strange reason, cxgbetool displays IP addresses in HEX ?!? > > > > How many rules can I plug in? > > > > > > I am not sure, but I think > > > > > > dev.t5nex.0.nfilters: number of filters > > > > > > shows the limit ? I have 20 on one box that handles > > about 1Gb/s of > > > packet forwarding. Under DDoS it sees 5-8 and nicely > > drops those > > > packets > > > and normal traffic flows unhindered. > > > > > > I also have > > > > > > hw.cxgbe.attack_filter="1" > > > hw.cxgbe.drop_pkts_with_l3_errors="1" > > > > > > as I often see corrupted packets as part of the DDoS, so > > I just drop > > > those anyways. The packets do show up in the NIC > > counters, so if you > > > are using graphana/cacti to monitor bandwidth, you will > > see them > > > as part > > > of the traffic counts. > > > > > > I run this every 5min and then graph it on cacti to keep > > track of how > > > much is dropped on the box. Its kinda depressing how > > much RFC1918 and > > > Bogon traffic gets dropped :( > > > > > > /usr/sbin/cxgbetool t5nex0 filter list | /usr/bin/grep > > -v Hits | > > > /usr/bin/awk '{ sum+=$2;} END{print sum;}' > > > /var/run/filter-stats.log > > > > > > > > > ---Mike > > > > > > > > > > > > > > Regards, > > > > Lyubomir > > > > > > > > > > > > _______________________________________________ > > > > Bsdrp-users mailing list > > > > Bsd...@li... > > <mailto:Bsd...@li...> > > > <mailto:Bsd...@li... > > <mailto:Bsd...@li...>> > > > > > > https://lists.sourceforge.net/lists/listinfo/bsdrp-users > > <https://lists.sourceforge.net/lists/listinfo/bsdrp-users> > > > > > <https://lists.sourceforge.net/lists/listinfo/bsdrp-users > > <https://lists.sourceforge.net/lists/listinfo/bsdrp-users>> > > > > > > |
From: mike t. <mi...@se...> - 2021-03-09 13:25:02
|
Hi, I havent tried shaping. If you are getting both blocks on the NIC firewall and ipfw its possible the packets ipfw are stopping are fragments or packets with bad checksums. You can have the NIC drop the bad checksum traffic too with that sysctl setting in loader.conf ---Mike On 3/9/2021 8:12 AM, Lyubomir Yotov wrote: > Hi Mike, > I tried with the VLAN option but it appears to be valid only for the > switch action: > #cxgbetool t5nex0 filter 10 iport 0 action pass dip 192.168.1.122 > dport 80 vlan "=100" > cxgbetool: port, dmac, smac, vlan, and nat only make sense with > "action switch" > Anyway I have implemented this and it works fine. I noticed that there > are some hits on ipfw rule that used to take care of it. Do you have > any idea why? > Have tried to do some sort of shaping in the card? > > Regards, > > Lyubo > > On Mon, 8 Mar 2021 at 13:52, Lyubomir Yotov <l....@gm... > <mailto:l....@gm...>> wrote: > > Hi Mike, > Thanks a lot! > > Regards, > > Lyubomir > > On Mon, 8 Mar 2021 at 12:26, mike tancsa <mi...@se... > <mailto:mi...@se...>> wrote: > > On 3/8/2021 3:17 AM, Lyubomir Yotov wrote: > > Hi Mike, > > Thanks for the quick response and provided information. I > currently > > have only one interface (in and out). I will try to use the > vlan > > option as well to be more precise. My rule might look like: > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip > 192.168.1.122 > > dport 23 vlan 100 > > Just to be on the safe side, if I add only the above drop > rule in the > > firewall I won't need explicit "allow all" in the end? > > Hi Lyubomir, > > Correct, its *not* like pf where its default block. You > can then > use cxgbetool t5nex0 filter list to see what hits. Actually, > maybe to > be on the safe side at first, instead of action drop add in > action pass > and then hit the rule to see if its being hit or not the way > you expect. > You will see the counter go up. then delete it and add it back > as action > drop when you are confident its going to do just what you want > it to do. > > > > > I don't have "hw.cxgbe.attack_filter" and > > "hw.cxgbe.drop_pkts_with_l3_errors". These will either > appear after I > > add a rule or if a change the firmware. I will check after > adding a rule. > > These are /boot/loader.conf values. See man cxl to see what > they do. > > ---Mike > > > > > > Regards, > > Lyubomir > > > > > > On Sun, 7 Mar 2021 at 19:15, mike tancsa <mi...@se... > <mailto:mi...@se...> > > <mailto:mi...@se... <mailto:mi...@se...>>> wrote: > > > > Hi, > > I am using the T5 firewall features on FreeBSD 11 > and 12 in > > production and it works great! > > > > On 3/7/2021 10:41 AM, Lyubomir Yotov wrote: > > > - is it safe to add rules on the fly in BSDRP? > > I add and remove rules on the fly all the time. > > > - is it safe to implement drop only rules on a > production server > > > without breaking the other traffic (should I have an > allow-all > > in the > > > end)? > > > I would like to test dropping all packets incoming on > cxl0 from any > > > host to host 192.168.1.122 with destination port 23. I > suppose a > > rule > > > like the following will do the job: > > > > > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip > 192.168.1.122 > > > dport 23 > > > > > Careful of the orientation. If you have 2 ports, the > iport makes a > > difference as to whether the rule gets hit or not. > > > > > > > If I want this persistent I should create a script > probably and > > start > > > it with the system boot? > > Yes. I have yet to come up with a nice interface to do > this. For some > > strange reason, cxgbetool displays IP addresses in HEX ?!? > > > How many rules can I plug in? > > > > I am not sure, but I think > > > > dev.t5nex.0.nfilters: number of filters > > > > shows the limit ? I have 20 on one box that handles > about 1Gb/s of > > packet forwarding. Under DDoS it sees 5-8 and nicely > drops those > > packets > > and normal traffic flows unhindered. > > > > I also have > > > > hw.cxgbe.attack_filter="1" > > hw.cxgbe.drop_pkts_with_l3_errors="1" > > > > as I often see corrupted packets as part of the DDoS, so > I just drop > > those anyways. The packets do show up in the NIC > counters, so if you > > are using graphana/cacti to monitor bandwidth, you will > see them > > as part > > of the traffic counts. > > > > I run this every 5min and then graph it on cacti to keep > track of how > > much is dropped on the box. Its kinda depressing how > much RFC1918 and > > Bogon traffic gets dropped :( > > > > /usr/sbin/cxgbetool t5nex0 filter list | /usr/bin/grep > -v Hits | > > /usr/bin/awk '{ sum+=$2;} END{print sum;}' > > /var/run/filter-stats.log > > > > > > ---Mike > > > > > > > > > > Regards, > > > Lyubomir > > > > > > > > > _______________________________________________ > > > Bsdrp-users mailing list > > > Bsd...@li... > <mailto:Bsd...@li...> > > <mailto:Bsd...@li... > <mailto:Bsd...@li...>> > > > > https://lists.sourceforge.net/lists/listinfo/bsdrp-users > <https://lists.sourceforge.net/lists/listinfo/bsdrp-users> > > > <https://lists.sourceforge.net/lists/listinfo/bsdrp-users > <https://lists.sourceforge.net/lists/listinfo/bsdrp-users>> > > > |
From: Lyubomir Y. <l....@gm...> - 2021-03-09 13:12:32
|
Hi Mike, I tried with the VLAN option but it appears to be valid only for the switch action: #cxgbetool t5nex0 filter 10 iport 0 action pass dip 192.168.1.122 dport 80 vlan "=100" cxgbetool: port, dmac, smac, vlan, and nat only make sense with "action switch" Anyway I have implemented this and it works fine. I noticed that there are some hits on ipfw rule that used to take care of it. Do you have any idea why? Have tried to do some sort of shaping in the card? Regards, Lyubo On Mon, 8 Mar 2021 at 13:52, Lyubomir Yotov <l....@gm...> wrote: > Hi Mike, > Thanks a lot! > > Regards, > > Lyubomir > > On Mon, 8 Mar 2021 at 12:26, mike tancsa <mi...@se...> wrote: > >> On 3/8/2021 3:17 AM, Lyubomir Yotov wrote: >> > Hi Mike, >> > Thanks for the quick response and provided information. I currently >> > have only one interface (in and out). I will try to use the vlan >> > option as well to be more precise. My rule might look like: >> > #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 >> > dport 23 vlan 100 >> > Just to be on the safe side, if I add only the above drop rule in the >> > firewall I won't need explicit "allow all" in the end? >> >> Hi Lyubomir, >> >> Correct, its *not* like pf where its default block. You can then >> use cxgbetool t5nex0 filter list to see what hits. Actually, maybe to >> be on the safe side at first, instead of action drop add in action pass >> and then hit the rule to see if its being hit or not the way you expect. >> You will see the counter go up. then delete it and add it back as action >> drop when you are confident its going to do just what you want it to do. >> >> > >> > I don't have "hw.cxgbe.attack_filter" and >> > "hw.cxgbe.drop_pkts_with_l3_errors". These will either appear after I >> > add a rule or if a change the firmware. I will check after adding a >> rule. >> >> These are /boot/loader.conf values. See man cxl to see what they do. >> >> ---Mike >> >> >> > >> > Regards, >> > Lyubomir >> > >> > >> > On Sun, 7 Mar 2021 at 19:15, mike tancsa <mi...@se... >> > <mailto:mi...@se...>> wrote: >> > >> > Hi, >> > I am using the T5 firewall features on FreeBSD 11 and 12 in >> > production and it works great! >> > >> > On 3/7/2021 10:41 AM, Lyubomir Yotov wrote: >> > > - is it safe to add rules on the fly in BSDRP? >> > I add and remove rules on the fly all the time. >> > > - is it safe to implement drop only rules on a production server >> > > without breaking the other traffic (should I have an allow-all >> > in the >> > > end)? >> > > I would like to test dropping all packets incoming on cxl0 from >> any >> > > host to host 192.168.1.122 with destination port 23. I suppose a >> > rule >> > > like the following will do the job: >> > > >> > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 >> > > dport 23 >> > > >> > Careful of the orientation. If you have 2 ports, the iport makes a >> > difference as to whether the rule gets hit or not. >> > >> > >> > > If I want this persistent I should create a script probably and >> > start >> > > it with the system boot? >> > Yes. I have yet to come up with a nice interface to do this. For >> some >> > strange reason, cxgbetool displays IP addresses in HEX ?!? >> > > How many rules can I plug in? >> > >> > I am not sure, but I think >> > >> > dev.t5nex.0.nfilters: number of filters >> > >> > shows the limit ? I have 20 on one box that handles about 1Gb/s of >> > packet forwarding. Under DDoS it sees 5-8 and nicely drops those >> > packets >> > and normal traffic flows unhindered. >> > >> > I also have >> > >> > hw.cxgbe.attack_filter="1" >> > hw.cxgbe.drop_pkts_with_l3_errors="1" >> > >> > as I often see corrupted packets as part of the DDoS, so I just drop >> > those anyways. The packets do show up in the NIC counters, so if >> you >> > are using graphana/cacti to monitor bandwidth, you will see them >> > as part >> > of the traffic counts. >> > >> > I run this every 5min and then graph it on cacti to keep track of >> how >> > much is dropped on the box. Its kinda depressing how much RFC1918 >> and >> > Bogon traffic gets dropped :( >> > >> > /usr/sbin/cxgbetool t5nex0 filter list | /usr/bin/grep -v Hits | >> > /usr/bin/awk '{ sum+=$2;} END{print sum;}' > >> /var/run/filter-stats.log >> > >> > >> > ---Mike >> > >> > >> > > >> > > Regards, >> > > Lyubomir >> > > >> > > >> > > _______________________________________________ >> > > Bsdrp-users mailing list >> > > Bsd...@li... >> > <mailto:Bsd...@li...> >> > > https://lists.sourceforge.net/lists/listinfo/bsdrp-users >> > <https://lists.sourceforge.net/lists/listinfo/bsdrp-users> >> > >> > |
From: Lyubomir Y. <l....@gm...> - 2021-03-08 11:52:30
|
Hi Mike, Thanks a lot! Regards, Lyubomir On Mon, 8 Mar 2021 at 12:26, mike tancsa <mi...@se...> wrote: > On 3/8/2021 3:17 AM, Lyubomir Yotov wrote: > > Hi Mike, > > Thanks for the quick response and provided information. I currently > > have only one interface (in and out). I will try to use the vlan > > option as well to be more precise. My rule might look like: > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 > > dport 23 vlan 100 > > Just to be on the safe side, if I add only the above drop rule in the > > firewall I won't need explicit "allow all" in the end? > > Hi Lyubomir, > > Correct, its *not* like pf where its default block. You can then > use cxgbetool t5nex0 filter list to see what hits. Actually, maybe to > be on the safe side at first, instead of action drop add in action pass > and then hit the rule to see if its being hit or not the way you expect. > You will see the counter go up. then delete it and add it back as action > drop when you are confident its going to do just what you want it to do. > > > > > I don't have "hw.cxgbe.attack_filter" and > > "hw.cxgbe.drop_pkts_with_l3_errors". These will either appear after I > > add a rule or if a change the firmware. I will check after adding a rule. > > These are /boot/loader.conf values. See man cxl to see what they do. > > ---Mike > > > > > > Regards, > > Lyubomir > > > > > > On Sun, 7 Mar 2021 at 19:15, mike tancsa <mi...@se... > > <mailto:mi...@se...>> wrote: > > > > Hi, > > I am using the T5 firewall features on FreeBSD 11 and 12 in > > production and it works great! > > > > On 3/7/2021 10:41 AM, Lyubomir Yotov wrote: > > > - is it safe to add rules on the fly in BSDRP? > > I add and remove rules on the fly all the time. > > > - is it safe to implement drop only rules on a production server > > > without breaking the other traffic (should I have an allow-all > > in the > > > end)? > > > I would like to test dropping all packets incoming on cxl0 from any > > > host to host 192.168.1.122 with destination port 23. I suppose a > > rule > > > like the following will do the job: > > > > > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 > > > dport 23 > > > > > Careful of the orientation. If you have 2 ports, the iport makes a > > difference as to whether the rule gets hit or not. > > > > > > > If I want this persistent I should create a script probably and > > start > > > it with the system boot? > > Yes. I have yet to come up with a nice interface to do this. For some > > strange reason, cxgbetool displays IP addresses in HEX ?!? > > > How many rules can I plug in? > > > > I am not sure, but I think > > > > dev.t5nex.0.nfilters: number of filters > > > > shows the limit ? I have 20 on one box that handles about 1Gb/s of > > packet forwarding. Under DDoS it sees 5-8 and nicely drops those > > packets > > and normal traffic flows unhindered. > > > > I also have > > > > hw.cxgbe.attack_filter="1" > > hw.cxgbe.drop_pkts_with_l3_errors="1" > > > > as I often see corrupted packets as part of the DDoS, so I just drop > > those anyways. The packets do show up in the NIC counters, so if you > > are using graphana/cacti to monitor bandwidth, you will see them > > as part > > of the traffic counts. > > > > I run this every 5min and then graph it on cacti to keep track of how > > much is dropped on the box. Its kinda depressing how much RFC1918 and > > Bogon traffic gets dropped :( > > > > /usr/sbin/cxgbetool t5nex0 filter list | /usr/bin/grep -v Hits | > > /usr/bin/awk '{ sum+=$2;} END{print sum;}' > > /var/run/filter-stats.log > > > > > > ---Mike > > > > > > > > > > Regards, > > > Lyubomir > > > > > > > > > _______________________________________________ > > > Bsdrp-users mailing list > > > Bsd...@li... > > <mailto:Bsd...@li...> > > > https://lists.sourceforge.net/lists/listinfo/bsdrp-users > > <https://lists.sourceforge.net/lists/listinfo/bsdrp-users> > > > |
From: mike t. <mi...@se...> - 2021-03-08 10:27:00
|
On 3/8/2021 3:17 AM, Lyubomir Yotov wrote: > Hi Mike, > Thanks for the quick response and provided information. I currently > have only one interface (in and out). I will try to use the vlan > option as well to be more precise. My rule might look like: > #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 > dport 23 vlan 100 > Just to be on the safe side, if I add only the above drop rule in the > firewall I won't need explicit "allow all" in the end? Hi Lyubomir, Correct, its *not* like pf where its default block. You can then use cxgbetool t5nex0 filter list to see what hits. Actually, maybe to be on the safe side at first, instead of action drop add in action pass and then hit the rule to see if its being hit or not the way you expect. You will see the counter go up. then delete it and add it back as action drop when you are confident its going to do just what you want it to do. > > I don't have "hw.cxgbe.attack_filter" and > "hw.cxgbe.drop_pkts_with_l3_errors". These will either appear after I > add a rule or if a change the firmware. I will check after adding a rule. These are /boot/loader.conf values. See man cxl to see what they do. ---Mike > > Regards, > Lyubomir > > > On Sun, 7 Mar 2021 at 19:15, mike tancsa <mi...@se... > <mailto:mi...@se...>> wrote: > > Hi, > I am using the T5 firewall features on FreeBSD 11 and 12 in > production and it works great! > > On 3/7/2021 10:41 AM, Lyubomir Yotov wrote: > > - is it safe to add rules on the fly in BSDRP? > I add and remove rules on the fly all the time. > > - is it safe to implement drop only rules on a production server > > without breaking the other traffic (should I have an allow-all > in the > > end)? > > I would like to test dropping all packets incoming on cxl0 from any > > host to host 192.168.1.122 with destination port 23. I suppose a > rule > > like the following will do the job: > > > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 > > dport 23 > > > Careful of the orientation. If you have 2 ports, the iport makes a > difference as to whether the rule gets hit or not. > > > > If I want this persistent I should create a script probably and > start > > it with the system boot? > Yes. I have yet to come up with a nice interface to do this. For some > strange reason, cxgbetool displays IP addresses in HEX ?!? > > How many rules can I plug in? > > I am not sure, but I think > > dev.t5nex.0.nfilters: number of filters > > shows the limit ? I have 20 on one box that handles about 1Gb/s of > packet forwarding. Under DDoS it sees 5-8 and nicely drops those > packets > and normal traffic flows unhindered. > > I also have > > hw.cxgbe.attack_filter="1" > hw.cxgbe.drop_pkts_with_l3_errors="1" > > as I often see corrupted packets as part of the DDoS, so I just drop > those anyways. The packets do show up in the NIC counters, so if you > are using graphana/cacti to monitor bandwidth, you will see them > as part > of the traffic counts. > > I run this every 5min and then graph it on cacti to keep track of how > much is dropped on the box. Its kinda depressing how much RFC1918 and > Bogon traffic gets dropped :( > > /usr/sbin/cxgbetool t5nex0 filter list | /usr/bin/grep -v Hits | > /usr/bin/awk '{ sum+=$2;} END{print sum;}' > /var/run/filter-stats.log > > > ---Mike > > > > > > Regards, > > Lyubomir > > > > > > _______________________________________________ > > Bsdrp-users mailing list > > Bsd...@li... > <mailto:Bsd...@li...> > > https://lists.sourceforge.net/lists/listinfo/bsdrp-users > <https://lists.sourceforge.net/lists/listinfo/bsdrp-users> > |
From: Lyubomir Y. <l....@gm...> - 2021-03-08 08:17:47
|
Hi Mike, Thanks for the quick response and provided information. I currently have only one interface (in and out). I will try to use the vlan option as well to be more precise. My rule might look like: #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 dport 23 vlan 100 Just to be on the safe side, if I add only the above drop rule in the firewall I won't need explicit "allow all" in the end? I don't have "hw.cxgbe.attack_filter" and "hw.cxgbe.drop_pkts_with_l3_errors". These will either appear after I add a rule or if a change the firmware. I will check after adding a rule. Regards, Lyubomir On Sun, 7 Mar 2021 at 19:15, mike tancsa <mi...@se...> wrote: > Hi, > I am using the T5 firewall features on FreeBSD 11 and 12 in > production and it works great! > > On 3/7/2021 10:41 AM, Lyubomir Yotov wrote: > > - is it safe to add rules on the fly in BSDRP? > I add and remove rules on the fly all the time. > > - is it safe to implement drop only rules on a production server > > without breaking the other traffic (should I have an allow-all in the > > end)? > > I would like to test dropping all packets incoming on cxl0 from any > > host to host 192.168.1.122 with destination port 23. I suppose a rule > > like the following will do the job: > > > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 > > dport 23 > > > Careful of the orientation. If you have 2 ports, the iport makes a > difference as to whether the rule gets hit or not. > > > > If I want this persistent I should create a script probably and start > > it with the system boot? > Yes. I have yet to come up with a nice interface to do this. For some > strange reason, cxgbetool displays IP addresses in HEX ?!? > > How many rules can I plug in? > > I am not sure, but I think > > dev.t5nex.0.nfilters: number of filters > > shows the limit ? I have 20 on one box that handles about 1Gb/s of > packet forwarding. Under DDoS it sees 5-8 and nicely drops those packets > and normal traffic flows unhindered. > > I also have > > hw.cxgbe.attack_filter="1" > hw.cxgbe.drop_pkts_with_l3_errors="1" > > as I often see corrupted packets as part of the DDoS, so I just drop > those anyways. The packets do show up in the NIC counters, so if you > are using graphana/cacti to monitor bandwidth, you will see them as part > of the traffic counts. > > I run this every 5min and then graph it on cacti to keep track of how > much is dropped on the box. Its kinda depressing how much RFC1918 and > Bogon traffic gets dropped :( > > /usr/sbin/cxgbetool t5nex0 filter list | /usr/bin/grep -v Hits | > /usr/bin/awk '{ sum+=$2;} END{print sum;}' > /var/run/filter-stats.log > > > ---Mike > > > > > > Regards, > > Lyubomir > > > > > > _______________________________________________ > > Bsdrp-users mailing list > > Bsd...@li... > > https://lists.sourceforge.net/lists/listinfo/bsdrp-users > > |
From: mike t. <mi...@se...> - 2021-03-07 17:15:57
|
Hi, I am using the T5 firewall features on FreeBSD 11 and 12 in production and it works great! On 3/7/2021 10:41 AM, Lyubomir Yotov wrote: > - is it safe to add rules on the fly in BSDRP? I add and remove rules on the fly all the time. > - is it safe to implement drop only rules on a production server > without breaking the other traffic (should I have an allow-all in the > end)? > I would like to test dropping all packets incoming on cxl0 from any > host to host 192.168.1.122 with destination port 23. I suppose a rule > like the following will do the job: > > #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 > dport 23 > Careful of the orientation. If you have 2 ports, the iport makes a difference as to whether the rule gets hit or not. > If I want this persistent I should create a script probably and start > it with the system boot? Yes. I have yet to come up with a nice interface to do this. For some strange reason, cxgbetool displays IP addresses in HEX ?!? > How many rules can I plug in? I am not sure, but I think dev.t5nex.0.nfilters: number of filters shows the limit ? I have 20 on one box that handles about 1Gb/s of packet forwarding. Under DDoS it sees 5-8 and nicely drops those packets and normal traffic flows unhindered. I also have hw.cxgbe.attack_filter="1" hw.cxgbe.drop_pkts_with_l3_errors="1" as I often see corrupted packets as part of the DDoS, so I just drop those anyways. The packets do show up in the NIC counters, so if you are using graphana/cacti to monitor bandwidth, you will see them as part of the traffic counts. I run this every 5min and then graph it on cacti to keep track of how much is dropped on the box. Its kinda depressing how much RFC1918 and Bogon traffic gets dropped :( /usr/sbin/cxgbetool t5nex0 filter list | /usr/bin/grep -v Hits | /usr/bin/awk '{ sum+=$2;} END{print sum;}' > /var/run/filter-stats.log ---Mike > > Regards, > Lyubomir > > > _______________________________________________ > Bsdrp-users mailing list > Bsd...@li... > https://lists.sourceforge.net/lists/listinfo/bsdrp-users |
From: Lyubomir Y. <l....@gm...> - 2021-03-07 15:42:09
|
Hi Olivier, Reading through "Dropping packets at high rate" on bsdrp.org I found a good solution for a project. Unfortunately I could not be 100% sure if the ACL in Chelsio cards is drop-all by default or allow-all by default. I suppose it is allow-all by default. I don't have a spare card at the moment but would like to test if I can drop some packages destined to a server at a given port. Since I have only a production system at the moment I would like to ask you: - is it safe to add rules on the fly in BSDRP? - is it safe to implement drop only rules on a production server without breaking the other traffic (should I have an allow-all in the end)? I would like to test dropping all packets incoming on cxl0 from any host to host 192.168.1.122 with destination port 23. I suppose a rule like the following will do the job: #cxgbetool t5nex0 filter 10 iport 0 action drop dip 192.168.1.122 dport 23 If I want this persistent I should create a script probably and start it with the system boot? How many rules can I plug in? Regards, Lyubomir |
From: Сучасні С. Б. <gri...@gm...> - 2021-02-24 23:08:26
|
also - the latest bsdrp you gave a link is a freebsd 14.. https://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/nightly/2021-02-23/amd64/ So i can't find and install packages into the system... PS.the system resize-system-slice also is not working on bsdrp 1.97 and 1.98 i get "geom /dev/da0 operation is not permitted" On Wed, Feb 24, 2021 at 6:46 PM Сучасні Системи Безпеки <gri...@gm...> wrote: > i have tried to disable every possible ACPI subsystem using *debug.acpi.disabled > in /boot/loader.conf* > *but with no luck* > *i understand it's a bug neigher in bios or with cpu or with motherboard* > *so i tried to completely disable the ACPI system * > > but while boot a get a panic > > panic: running without device atpic requires a local APIC > > > please advise, how to disable ACPI in bsdrp and i can boot the system ? > > > On Wed, Feb 24, 2021 at 9:34 AM Сучасні Системи Безпеки < > gri...@gm...> wrote: > >> upgraded to latest bsdrp >> >> look at my sysctl dev.cpu output (it's on every cpu) >> >> dev.cpu.1.cx_usage: 100.00% 0.00% last 49455us >> dev.cpu.1.cx_lowest: C1 >> dev.cpu.1.cx_supported: C1/1/1 C2/2/41 >> >> i run pmcstat and see this >> >> 41.4 kernel acpi_cpu_idle_mwait acpi_cpu_idle >> >> ok. I have added in /etc/rc.conf -> performance_cx_lowest="LOW" >> rebooted the system >> >> now this is how the sysctl dev.cpu looks like >> >> dev.cpu.0.cx_usage: 2.25% 97.74% last 4711us >> dev.cpu.0.cx_lowest: C8 >> dev.cpu.0.cx_supported: C1/1/1 C2/2/41 >> >> in pmcstat there is a new string with high % >> >> %SAMP IMAGE FUNCTION CALLERS >> 54.4 kernel lock_delay _mtx_lock_spin_cookie >> 18.0 kernel acpi_cpu_idle_mwait acpi_cpu_idle >> >> by the way the acpi_cpu_idle is lower now but the mtx_lock_spin is high.. >> >> ok i run lockstat -x aggsize=4m sleep 10 > lock-type.txt >> >> Adaptive mutex spin: 2 events in 10.058 seconds (0 events/sec) >> >> Count indv cuml rcnt nsec Lock Caller >> >> ------------------------------------------------------------------------------- >> 2 100% 100% 0.00 2649 Giant >> uhub_explore_handle_re_enumerate+0x1d >> >> ------------------------------------------------------------------------------- >> >> Spin lock spin: 1912 events in 10.058 seconds (190 events/sec) >> >> Count indv cuml rcnt nsec Lock Caller >> >> ------------------------------------------------------------------------------- >> 1909 100% 100% 0.00 558053 ACPI lock (0xfffff80003954580) >> AcpiWriteBitRegister+0x2b >> 1 0% 100% 0.00 756 icu ithread_loop+0x33c >> 1 0% 100% 0.00 622 icu >> intr_event_handle+0x12f >> 1 0% 100% 0.00 769 callout >> _callout_stop_safe+0xda >> >> ------------------------------------------------------------------------------- >> >> Thread lock spin: 2 events in 10.058 seconds (0 events/sec) >> >> Count indv cuml rcnt nsec Lock Caller >> >> ------------------------------------------------------------------------------- >> 1 50% 50% 0.00 2449 sleepq chain >> softclock_call_cc+0x13d >> 1 50% 100% 0.00 885 sched lock 6 fork_exit+0x7e >> >> ------------------------------------------------------------------------------- >> >> >> now the lock-stack output >> >> Spin lock spin: 1917 events in 10.051 seconds (191 events/sec) >> >> >> ------------------------------------------------------------------------------- >> Count indv cuml rcnt nsec Lock Caller >> 1417 74% 74% 0.00 673953 ACPI lock (0xfffff80003954580) >> AcpiWriteBitRegister+0x2b >> >> nsec ------ Time Distribution ------ count Stack >> 512 | 1 >> AcpiWriteBitRegister+0x2b >> 1024 | 2 acpi_cpu_idle+0x2ab >> 2048 | 15 cpu_idle_acpi+0x3e >> 4096 | 15 cpu_idle+0x9f >> 8192 | 33 sched_idletd+0x2e4 >> 16384 |@ 84 fork_exit+0x7e >> 32768 |@ 84 0xffffffff80dd494e >> 65536 | 43 >> 131072 |@ 78 >> 262144 |@ 94 >> 524288 |@@@ 174 >> 1048576 |@@@@@@@@ 423 >> 2097152 |@@@@@@@ 333 >> 4194304 | 38 >> >> ------------------------------------------------------------------------------- >> Count indv cuml rcnt nsec Lock Caller >> 473 25% 99% 0.00 276647 ACPI lock (0xfffff80003954580) >> AcpiWriteBitRegister+0x2b >> >> nsec ------ Time Distribution ------ count Stack >> 512 | 1 >> AcpiWriteBitRegister+0x2b >> 1024 | 14 acpi_cpu_idle+0x1a7 >> 2048 | 5 cpu_idle_acpi+0x3e >> 4096 | 10 cpu_idle+0x9f >> 8192 |@@ 38 sched_idletd+0x2e4 >> 16384 |@@@@@@@ 112 fork_exit+0x7e >> 32768 |@ 31 0xffffffff80dd494e >> 65536 |@@@ 52 >> 131072 |@@@@ 78 >> 262144 |@ 22 >> 524288 |@ 20 >> 1048576 |@@ 45 >> 2097152 |@@ 39 >> 4194304 | 6 >> >> ------------------------------------------------------------------------------- >> Count indv cuml rcnt nsec Lock Caller >> 21 1% 100% 0.00 794069 ACPI lock (0xfffff80003954580) >> AcpiWriteBitRegister+0x2b >> >> nsec ------ Time Distribution ------ count Stack >> 8192 |@ 1 >> AcpiWriteBitRegister+0x2b >> 16384 |@ 1 acpi_cpu_idle+0x2b7 >> 32768 |@@@@ 3 cpu_idle_acpi+0x3e >> 65536 |@@ 2 cpu_idle+0x9f >> 131072 | 0 sched_idletd+0x2e4 >> 262144 |@ 1 fork_exit+0x7e >> 524288 |@@ 2 0xffffffff80dd494e >> 1048576 |@@@@@ 4 >> 2097152 |@@@@@@@@ 6 >> 4194304 |@ 1 >> >> ------------------------------------------------------------------------------- >> Count indv cuml rcnt nsec Lock Caller >> 5 0% 100% 0.00 79080 ACPI lock (0xfffff80003954580) >> AcpiWriteBitRegister+0x2b >> >> nsec ------ Time Distribution ------ count Stack >> 8192 |@@@@@@ 1 >> AcpiWriteBitRegister+0x2b >> 16384 |@@@@@@@@@@@@ 2 acpi_cpu_idle+0x1b6 >> 32768 |@@@@@@ 1 cpu_idle_acpi+0x3e >> 65536 | 0 cpu_idle+0x9f >> 131072 | 0 sched_idletd+0x2e4 >> 262144 | 0 fork_exit+0x7e >> 524288 |@@@@@@ 1 0xffffffff80dd494e >> >> ------------------------------------------------------------------------------- >> Count indv cuml rcnt nsec Lock Caller >> 1 0% 100% 0.00 465 icu >> intr_event_handle+0x12f >> >> nsec ------ Time Distribution ------ count Stack >> 512 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 >> intr_event_handle+0x12f >> >> intr_execute_handlers+0x4b >> 0xffffffff80dd4afc >> cpu_idle_acpi+0x3e >> cpu_idle+0x9f >> sched_idletd+0x2e4 >> fork_exit+0x7e >> 0xffffffff80dd494e >> >> ------------------------------------------------------------------------------- >> >> >> >> i have absolutely no idea what's this is about - so i request help from >> you %) >> >> >> On Wed, Feb 24, 2021 at 1:26 AM Olivier Cochard-Labbé <ol...@co...> >> wrote: >> >>> And I forgot about your lockstat bug, here the nightly release that >>> should fix it: >>> >>> >>> https://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/nightly/2021-02-23/amd64/ >>> >>> Regards, >>> Olivier >>> >>>> _______________________________________________ >>> Bsdrp-users mailing list >>> Bsd...@li... >>> https://lists.sourceforge.net/lists/listinfo/bsdrp-users >>> >> >> >> -- >> "Cучасні Системи Безпеки" >> http://www.itlogix.com.ua >> +38 (050) 523-66-17 >> +38 (067) 523-66-17 >> > > > -- > "Cучасні Системи Безпеки" > http://www.itlogix.com.ua > +38 (050) 523-66-17 > +38 (067) 523-66-17 > -- "Cучасні Системи Безпеки" http://www.itlogix.com.ua +38 (050) 523-66-17 +38 (067) 523-66-17 |
From: Сучасні С. Б. <gri...@gm...> - 2021-02-24 16:47:15
|
i have tried to disable every possible ACPI subsystem using *debug.acpi.disabled in /boot/loader.conf* *but with no luck* *i understand it's a bug neigher in bios or with cpu or with motherboard* *so i tried to completely disable the ACPI system * but while boot a get a panic panic: running without device atpic requires a local APIC please advise, how to disable ACPI in bsdrp and i can boot the system ? On Wed, Feb 24, 2021 at 9:34 AM Сучасні Системи Безпеки <gri...@gm...> wrote: > upgraded to latest bsdrp > > look at my sysctl dev.cpu output (it's on every cpu) > > dev.cpu.1.cx_usage: 100.00% 0.00% last 49455us > dev.cpu.1.cx_lowest: C1 > dev.cpu.1.cx_supported: C1/1/1 C2/2/41 > > i run pmcstat and see this > > 41.4 kernel acpi_cpu_idle_mwait acpi_cpu_idle > > ok. I have added in /etc/rc.conf -> performance_cx_lowest="LOW" > rebooted the system > > now this is how the sysctl dev.cpu looks like > > dev.cpu.0.cx_usage: 2.25% 97.74% last 4711us > dev.cpu.0.cx_lowest: C8 > dev.cpu.0.cx_supported: C1/1/1 C2/2/41 > > in pmcstat there is a new string with high % > > %SAMP IMAGE FUNCTION CALLERS > 54.4 kernel lock_delay _mtx_lock_spin_cookie > 18.0 kernel acpi_cpu_idle_mwait acpi_cpu_idle > > by the way the acpi_cpu_idle is lower now but the mtx_lock_spin is high.. > > ok i run lockstat -x aggsize=4m sleep 10 > lock-type.txt > > Adaptive mutex spin: 2 events in 10.058 seconds (0 events/sec) > > Count indv cuml rcnt nsec Lock Caller > > ------------------------------------------------------------------------------- > 2 100% 100% 0.00 2649 Giant > uhub_explore_handle_re_enumerate+0x1d > > ------------------------------------------------------------------------------- > > Spin lock spin: 1912 events in 10.058 seconds (190 events/sec) > > Count indv cuml rcnt nsec Lock Caller > > ------------------------------------------------------------------------------- > 1909 100% 100% 0.00 558053 ACPI lock (0xfffff80003954580) > AcpiWriteBitRegister+0x2b > 1 0% 100% 0.00 756 icu ithread_loop+0x33c > 1 0% 100% 0.00 622 icu > intr_event_handle+0x12f > 1 0% 100% 0.00 769 callout > _callout_stop_safe+0xda > > ------------------------------------------------------------------------------- > > Thread lock spin: 2 events in 10.058 seconds (0 events/sec) > > Count indv cuml rcnt nsec Lock Caller > > ------------------------------------------------------------------------------- > 1 50% 50% 0.00 2449 sleepq chain > softclock_call_cc+0x13d > 1 50% 100% 0.00 885 sched lock 6 fork_exit+0x7e > > ------------------------------------------------------------------------------- > > > now the lock-stack output > > Spin lock spin: 1917 events in 10.051 seconds (191 events/sec) > > > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock Caller > 1417 74% 74% 0.00 673953 ACPI lock (0xfffff80003954580) > AcpiWriteBitRegister+0x2b > > nsec ------ Time Distribution ------ count Stack > 512 | 1 > AcpiWriteBitRegister+0x2b > 1024 | 2 acpi_cpu_idle+0x2ab > 2048 | 15 cpu_idle_acpi+0x3e > 4096 | 15 cpu_idle+0x9f > 8192 | 33 sched_idletd+0x2e4 > 16384 |@ 84 fork_exit+0x7e > 32768 |@ 84 0xffffffff80dd494e > 65536 | 43 > 131072 |@ 78 > 262144 |@ 94 > 524288 |@@@ 174 > 1048576 |@@@@@@@@ 423 > 2097152 |@@@@@@@ 333 > 4194304 | 38 > > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock Caller > 473 25% 99% 0.00 276647 ACPI lock (0xfffff80003954580) > AcpiWriteBitRegister+0x2b > > nsec ------ Time Distribution ------ count Stack > 512 | 1 > AcpiWriteBitRegister+0x2b > 1024 | 14 acpi_cpu_idle+0x1a7 > 2048 | 5 cpu_idle_acpi+0x3e > 4096 | 10 cpu_idle+0x9f > 8192 |@@ 38 sched_idletd+0x2e4 > 16384 |@@@@@@@ 112 fork_exit+0x7e > 32768 |@ 31 0xffffffff80dd494e > 65536 |@@@ 52 > 131072 |@@@@ 78 > 262144 |@ 22 > 524288 |@ 20 > 1048576 |@@ 45 > 2097152 |@@ 39 > 4194304 | 6 > > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock Caller > 21 1% 100% 0.00 794069 ACPI lock (0xfffff80003954580) > AcpiWriteBitRegister+0x2b > > nsec ------ Time Distribution ------ count Stack > 8192 |@ 1 > AcpiWriteBitRegister+0x2b > 16384 |@ 1 acpi_cpu_idle+0x2b7 > 32768 |@@@@ 3 cpu_idle_acpi+0x3e > 65536 |@@ 2 cpu_idle+0x9f > 131072 | 0 sched_idletd+0x2e4 > 262144 |@ 1 fork_exit+0x7e > 524288 |@@ 2 0xffffffff80dd494e > 1048576 |@@@@@ 4 > 2097152 |@@@@@@@@ 6 > 4194304 |@ 1 > > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock Caller > 5 0% 100% 0.00 79080 ACPI lock (0xfffff80003954580) > AcpiWriteBitRegister+0x2b > > nsec ------ Time Distribution ------ count Stack > 8192 |@@@@@@ 1 > AcpiWriteBitRegister+0x2b > 16384 |@@@@@@@@@@@@ 2 acpi_cpu_idle+0x1b6 > 32768 |@@@@@@ 1 cpu_idle_acpi+0x3e > 65536 | 0 cpu_idle+0x9f > 131072 | 0 sched_idletd+0x2e4 > 262144 | 0 fork_exit+0x7e > 524288 |@@@@@@ 1 0xffffffff80dd494e > > ------------------------------------------------------------------------------- > Count indv cuml rcnt nsec Lock Caller > 1 0% 100% 0.00 465 icu > intr_event_handle+0x12f > > nsec ------ Time Distribution ------ count Stack > 512 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 > intr_event_handle+0x12f > > intr_execute_handlers+0x4b > 0xffffffff80dd4afc > cpu_idle_acpi+0x3e > cpu_idle+0x9f > sched_idletd+0x2e4 > fork_exit+0x7e > 0xffffffff80dd494e > > ------------------------------------------------------------------------------- > > > > i have absolutely no idea what's this is about - so i request help from > you %) > > > On Wed, Feb 24, 2021 at 1:26 AM Olivier Cochard-Labbé <ol...@co...> > wrote: > >> And I forgot about your lockstat bug, here the nightly release that >> should fix it: >> >> >> https://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/nightly/2021-02-23/amd64/ >> >> Regards, >> Olivier >> >>> _______________________________________________ >> Bsdrp-users mailing list >> Bsd...@li... >> https://lists.sourceforge.net/lists/listinfo/bsdrp-users >> > > > -- > "Cучасні Системи Безпеки" > http://www.itlogix.com.ua > +38 (050) 523-66-17 > +38 (067) 523-66-17 > -- "Cучасні Системи Безпеки" http://www.itlogix.com.ua +38 (050) 523-66-17 +38 (067) 523-66-17 |
From: Сучасні С. Б. <gri...@gm...> - 2021-02-24 07:35:14
|
upgraded to latest bsdrp look at my sysctl dev.cpu output (it's on every cpu) dev.cpu.1.cx_usage: 100.00% 0.00% last 49455us dev.cpu.1.cx_lowest: C1 dev.cpu.1.cx_supported: C1/1/1 C2/2/41 i run pmcstat and see this 41.4 kernel acpi_cpu_idle_mwait acpi_cpu_idle ok. I have added in /etc/rc.conf -> performance_cx_lowest="LOW" rebooted the system now this is how the sysctl dev.cpu looks like dev.cpu.0.cx_usage: 2.25% 97.74% last 4711us dev.cpu.0.cx_lowest: C8 dev.cpu.0.cx_supported: C1/1/1 C2/2/41 in pmcstat there is a new string with high % %SAMP IMAGE FUNCTION CALLERS 54.4 kernel lock_delay _mtx_lock_spin_cookie 18.0 kernel acpi_cpu_idle_mwait acpi_cpu_idle by the way the acpi_cpu_idle is lower now but the mtx_lock_spin is high.. ok i run lockstat -x aggsize=4m sleep 10 > lock-type.txt Adaptive mutex spin: 2 events in 10.058 seconds (0 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 2 100% 100% 0.00 2649 Giant uhub_explore_handle_re_enumerate+0x1d ------------------------------------------------------------------------------- Spin lock spin: 1912 events in 10.058 seconds (190 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 1909 100% 100% 0.00 558053 ACPI lock (0xfffff80003954580) AcpiWriteBitRegister+0x2b 1 0% 100% 0.00 756 icu ithread_loop+0x33c 1 0% 100% 0.00 622 icu intr_event_handle+0x12f 1 0% 100% 0.00 769 callout _callout_stop_safe+0xda ------------------------------------------------------------------------------- Thread lock spin: 2 events in 10.058 seconds (0 events/sec) Count indv cuml rcnt nsec Lock Caller ------------------------------------------------------------------------------- 1 50% 50% 0.00 2449 sleepq chain softclock_call_cc+0x13d 1 50% 100% 0.00 885 sched lock 6 fork_exit+0x7e ------------------------------------------------------------------------------- now the lock-stack output Spin lock spin: 1917 events in 10.051 seconds (191 events/sec) ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Caller 1417 74% 74% 0.00 673953 ACPI lock (0xfffff80003954580) AcpiWriteBitRegister+0x2b nsec ------ Time Distribution ------ count Stack 512 | 1 AcpiWriteBitRegister+0x2b 1024 | 2 acpi_cpu_idle+0x2ab 2048 | 15 cpu_idle_acpi+0x3e 4096 | 15 cpu_idle+0x9f 8192 | 33 sched_idletd+0x2e4 16384 |@ 84 fork_exit+0x7e 32768 |@ 84 0xffffffff80dd494e 65536 | 43 131072 |@ 78 262144 |@ 94 524288 |@@@ 174 1048576 |@@@@@@@@ 423 2097152 |@@@@@@@ 333 4194304 | 38 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Caller 473 25% 99% 0.00 276647 ACPI lock (0xfffff80003954580) AcpiWriteBitRegister+0x2b nsec ------ Time Distribution ------ count Stack 512 | 1 AcpiWriteBitRegister+0x2b 1024 | 14 acpi_cpu_idle+0x1a7 2048 | 5 cpu_idle_acpi+0x3e 4096 | 10 cpu_idle+0x9f 8192 |@@ 38 sched_idletd+0x2e4 16384 |@@@@@@@ 112 fork_exit+0x7e 32768 |@ 31 0xffffffff80dd494e 65536 |@@@ 52 131072 |@@@@ 78 262144 |@ 22 524288 |@ 20 1048576 |@@ 45 2097152 |@@ 39 4194304 | 6 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Caller 21 1% 100% 0.00 794069 ACPI lock (0xfffff80003954580) AcpiWriteBitRegister+0x2b nsec ------ Time Distribution ------ count Stack 8192 |@ 1 AcpiWriteBitRegister+0x2b 16384 |@ 1 acpi_cpu_idle+0x2b7 32768 |@@@@ 3 cpu_idle_acpi+0x3e 65536 |@@ 2 cpu_idle+0x9f 131072 | 0 sched_idletd+0x2e4 262144 |@ 1 fork_exit+0x7e 524288 |@@ 2 0xffffffff80dd494e 1048576 |@@@@@ 4 2097152 |@@@@@@@@ 6 4194304 |@ 1 ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Caller 5 0% 100% 0.00 79080 ACPI lock (0xfffff80003954580) AcpiWriteBitRegister+0x2b nsec ------ Time Distribution ------ count Stack 8192 |@@@@@@ 1 AcpiWriteBitRegister+0x2b 16384 |@@@@@@@@@@@@ 2 acpi_cpu_idle+0x1b6 32768 |@@@@@@ 1 cpu_idle_acpi+0x3e 65536 | 0 cpu_idle+0x9f 131072 | 0 sched_idletd+0x2e4 262144 | 0 fork_exit+0x7e 524288 |@@@@@@ 1 0xffffffff80dd494e ------------------------------------------------------------------------------- Count indv cuml rcnt nsec Lock Caller 1 0% 100% 0.00 465 icu intr_event_handle+0x12f nsec ------ Time Distribution ------ count Stack 512 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 1 intr_event_handle+0x12f intr_execute_handlers+0x4b 0xffffffff80dd4afc cpu_idle_acpi+0x3e cpu_idle+0x9f sched_idletd+0x2e4 fork_exit+0x7e 0xffffffff80dd494e ------------------------------------------------------------------------------- i have absolutely no idea what's this is about - so i request help from you %) On Wed, Feb 24, 2021 at 1:26 AM Olivier Cochard-Labbé <ol...@co...> wrote: > And I forgot about your lockstat bug, here the nightly release that should > fix it: > > > https://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/nightly/2021-02-23/amd64/ > > Regards, > Olivier > >> _______________________________________________ > Bsdrp-users mailing list > Bsd...@li... > https://lists.sourceforge.net/lists/listinfo/bsdrp-users > -- "Cучасні Системи Безпеки" http://www.itlogix.com.ua +38 (050) 523-66-17 +38 (067) 523-66-17 |
From: Olivier Cochard-L. <ol...@co...> - 2021-02-23 23:25:55
|
And I forgot about your lockstat bug, here the nightly release that should fix it: https://sourceforge.net/projects/bsdrp/files/BSD_Router_Project/nightly/2021-02-23/amd64/ Regards, Olivier > |