You can subscribe to this list here.
2004 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(5) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2005 |
Jan
(1) |
Feb
(31) |
Mar
(22) |
Apr
(31) |
May
(8) |
Jun
|
Jul
(2) |
Aug
|
Sep
(1) |
Oct
(3) |
Nov
(10) |
Dec
(7) |
2006 |
Jan
(9) |
Feb
(7) |
Mar
(8) |
Apr
|
May
(17) |
Jun
(11) |
Jul
(4) |
Aug
(2) |
Sep
(1) |
Oct
(7) |
Nov
(3) |
Dec
(14) |
2007 |
Jan
(12) |
Feb
(38) |
Mar
(100) |
Apr
(36) |
May
(41) |
Jun
(35) |
Jul
(37) |
Aug
(38) |
Sep
(5) |
Oct
(3) |
Nov
(29) |
Dec
(17) |
2008 |
Jan
(13) |
Feb
(5) |
Mar
(13) |
Apr
(11) |
May
|
Jun
|
Jul
(1) |
Aug
(19) |
Sep
(28) |
Oct
(1) |
Nov
|
Dec
|
2009 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
(3) |
Jul
|
Aug
|
Sep
|
Oct
(9) |
Nov
(8) |
Dec
|
2010 |
Jan
(1) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
(2) |
Sep
(2) |
Oct
|
Nov
|
Dec
|
2011 |
Jan
|
Feb
|
Mar
|
Apr
(23) |
May
|
Jun
(8) |
Jul
(4) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2012 |
Jan
|
Feb
|
Mar
|
Apr
(5) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Mihai C. <m.l...@uv...> - 2009-11-02 09:45:28
|
I'm trying to implement this feature: in an expression that ends with netfilter_toggle, "if there is one single filter that returns 0, then toggle should drop the packet." This is the current codeline of netfilter_toggle and is supposed to take the return value of the previous filter in the chain with netfilter_toggle: nf_override_return = iptr->class; How do I search for return value of all filters in the expression? Thanks, Mihai > -----Original Message----- > From: Willem de Bruijn [mailto:wil...@gm...] > Sent: 02 November 2009 01:25 > To: m.l...@uv... > Subject: Re: toggle question > > You could have it behave that way, sure. You have my blessing to > modify the filter > (not that you need that, of course :) > > On Sun, Nov 1, 2009 at 7:22 PM, Mihai Cristea <m.l...@uv...> > wrote: > > > >> all netfilter_toggle does is set a global variable. You can move it > >> before > >> tcpudp_csum > > > > Can we have this variable reset by any filter in the expression with > toggle? > > > > I mean - if there is one single filter that returns 0, then toggle > shouldn't > > pass it. > > > > Cheers, > > Mihai > > > > > >> -----Original Message----- > >> From: Mihai Cristea [mailto:m.l...@uv...] > >> Sent: 02 November 2009 01:07 > >> To: 'Willem de Bruijn' > >> Subject: toggle question > >> > >> Hi Willem, > >> > >> > netfilter_in --share=no eth3 | fpl > >> The parser fails to take this "--share=no" > >> > >> > (netfilter_in,share=no,expression=eth3) > (fpl) > >> This way works. > >> > >> I have a question to you on this expression example that includes > >> netfilter_toggle: > >> > >> (netfilter_fetch_in) > (fpl_tbs,expression="Token") > > >> (fpl_ipsource,expression=\"ipsrc2\") > (tcpudp_csum) -all-> > >> (netfilter_toggle) > >> > >> I understood that netfilter_toggle passes packets up to userspace > for > >> which the previous filter (e.g., tcpudp_csum) returned 1. > >> In this particular example, tcpudp_csum always returns 1! But, we > want > >> to have the fpl_tbs filter deciding whether drop the packet or pass > it > >> to user-space (netfilter_toggle). Would this work in this way? Or is > >> only going to work if we move the (fpl_tbs, "Token") before > >> (netfilter_toggle)? > >> If so, is going to be a performance drop due to lots of packets that > >> are not necessarily needed to compute checksum and pass to user- > space. > >> > >> I hope there's a good way to handle such a request. > >> > >> Cheers, > >> Mihai > > > > > > |
From: Mihai C. <m.l...@uv...> - 2009-11-02 00:23:14
|
> all netfilter_toggle does is set a global variable. You can move it > before > tcpudp_csum Can we have this variable reset by any filter in the expression with toggle? I mean - if there is one single filter that returns 0, then toggle shouldn't pass it. Cheers, Mihai > -----Original Message----- > From: Mihai Cristea [mailto:m.l...@uv...] > Sent: 02 November 2009 01:07 > To: 'Willem de Bruijn' > Subject: toggle question > > Hi Willem, > > > netfilter_in --share=no eth3 | fpl > The parser fails to take this "--share=no" > > > (netfilter_in,share=no,expression=eth3) > (fpl) > This way works. > > I have a question to you on this expression example that includes > netfilter_toggle: > > (netfilter_fetch_in) > (fpl_tbs,expression="Token") > > (fpl_ipsource,expression=\"ipsrc2\") > (tcpudp_csum) -all-> > (netfilter_toggle) > > I understood that netfilter_toggle passes packets up to userspace for > which the previous filter (e.g., tcpudp_csum) returned 1. > In this particular example, tcpudp_csum always returns 1! But, we want > to have the fpl_tbs filter deciding whether drop the packet or pass it > to user-space (netfilter_toggle). Would this work in this way? Or is > only going to work if we move the (fpl_tbs, "Token") before > (netfilter_toggle)? > If so, is going to be a performance drop due to lots of packets that > are not necessarily needed to compute checksum and pass to user-space. > > I hope there's a good way to handle such a request. > > Cheers, > Mihai |
From: Mihai C. <m.l...@uv...> - 2009-11-01 17:26:41
|
Could you be more specific, where shall I put the "--share=no flag" option? In Makefile? Talking about bugs: I think the slcli doesn't work properly: Do an insert of an expression ... whatever it is ... Remove it. Then scroll it back from history, and modify it (e.g., delete some parts) and try insert again. It fails to insert it! Quit Then insert the previously edited expression which failed before quit=> it works now! Cheers, Mihai > -----Original Message----- > From: Willem de Bruijn [mailto:wil...@gm...] > Sent: 01 November 2009 17:07 > To: m.l...@uv... > Cc: ffp...@li... > Subject: Re: [Ffpf-devel] toggle problem > > He Mihai, > > Streamline should be smart enough to create separate filters when the > arguments differ. > But I did simplify that code last summer and a bug may have crept in. > You can always > override the default prefix sharing algorithm by adding a --share=no > flag. I think I added > this flag for your SC demo last year, by chance! > > Let me know if that helps. I plan to have a look at this and another > bugreport later > today, but as there's so much on my plate it may actually take a few > days longer. > > willem > > On Sat, Oct 31, 2009 at 7:15 PM, Mihai Cristea <m.l...@uv...> > wrote: > > Hi Willem, > > > > Testing streamline with the new features we had, I've found the > following > > problem: > > If I have 2 expressions as follows, and at the 2nd insert => the host > > freezes completely. I need to reboot it. > > netfilter_fetch_in | fpl_tokenless |all netfilter_toggle > > netfilter_fetch_in eth3 | fpl_ipsrc "10.10.0.31" | tcpudp_csum |all > > netfilter_toggle > > > > Reggie identified the problem: is due to the newex call that is not > called > > 2nd time and hence, fpl_ipsrc doesn't get "eth3", but "all": this > would > > replace ipsrc to all packets, including our ssh! > > > > The proof comes from dmesg: > > [Start] netfilter in dev "all" > > [Start] netfilter in f91cc554 > > [WARN ] Failed to set callback. Already in use > > [WARN ] Failed to set callback. Already in use > > Where the first 2 lines seems to belong to 1st netfilter while the > warnings > > are for the second netfilter_in > > > > How can we get every netfilter_xx instance with its own parameter? > > > > BTW. Update streamline from svn to include Reggie's fix on the > hook_in > > problem. > > > > Cheers, > > Mihai > > > > > > --------------------------------------------------------------------- > --------- > > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > > is the only developer event you need to attend this year. Jumpstart > your > > developing skills, take BlackBerry mobile applications to market and > stay > > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > > http://p.sf.net/sfu/devconference > > _______________________________________________ > > Ffpf-devel mailing list > > Ffp...@li... > > https://lists.sourceforge.net/lists/listinfo/ffpf-devel > > |
From: Mihai C. <m.l...@uv...> - 2009-11-01 00:56:48
|
Hi Willem, Testing streamline with the new features we had, I've found the following problem: If I have 2 expressions as follows, and at the 2nd insert => the host freezes completely. I need to reboot it. netfilter_fetch_in | fpl_tokenless |all netfilter_toggle netfilter_fetch_in eth3 | fpl_ipsrc "10.10.0.31" | tcpudp_csum |all netfilter_toggle Reggie identified the problem: is due to the newex call that is not called 2nd time and hence, fpl_ipsrc doesn't get "eth3", but "all": this would replace ipsrc to all packets, including our ssh! The proof comes from dmesg: [Start] netfilter in dev "all" [Start] netfilter in f91cc554 [WARN ] Failed to set callback. Already in use [WARN ] Failed to set callback. Already in use Where the first 2 lines seems to belong to 1st netfilter while the warnings are for the second netfilter_in How can we get every netfilter_xx instance with its own parameter? BTW. Update streamline from svn to include Reggie's fix on the hook_in problem. Cheers, Mihai |
From: Mihai C. <m.l...@uv...> - 2009-10-28 12:56:18
|
Well, I started a fresh test (rebooted machines) and I don't have such error messages. See below. Unfortunately, the client still sends RST pkts. These pkts are "intercepted in transmit.c", but there some still go out of the box and reaches server! Mihai [Start] messaging [WARN ] created class streamline [Start] route local.1 [Start] channel instance local from 1 to 1 (#1) DEBUG buf 1 added impl fixed sized slot buffer, now has stacklen 1 BeltwayFS is not mounted. Cannot create node DEBUG buf 1 added impl variable sized slot buffer, now has stacklen 1 BeltwayFS is not mounted. Cannot create node DEBUG buf 1 added impl variable sized slot buffer, now has stacklen 1 BeltwayFS is not mounted. Cannot create node DEBUG buf 1 added impl fixed sized slot buffer, now has stacklen 1 BeltwayFS is not mounted. Cannot create node [Info ] Default DBuf size is 1048576B, default IBuf size is 8192B [Start] transport [Start] porttable [Start] datapath [Start] Splicing v2 enabled tb: no version for "boot_run" found: kernel tainted. Inserted FPL component Inserted FPL component Inserted FPL component Inserted FPL component Inserted FPL component Inserted FPL component e1000: eth2: e1000_watchdog_task: NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None device eth3 entered promiscuous mode [Start] route sysfs.2 [Start] channel instance sysfs from 1 to 2 (#1) [Start] route sysfs_up_message.2 [Start] route sysfs_up_signal.2 [Start] route sysfs area.2 [Info ] route update: 2 became 3 [Info ] update ID: 2 becomes 3 [Start] channel instance sysfs_up_signal from 1 to 3 (#2) [Stop ] channel instance sysfs_up_signal from 1 to 3 (#2) [Stop ] channel instance sysfs from 1 to 3 (#1) [Stop ] route sysfs.3 [Stop ] route sysfs_up_message.3 [Stop ] route sysfs_up_signal.3 [Stop ] route sysfs area.3 [Start] route sysfs.2 [Start] channel instance sysfs from 1 to 2 (#3) [Start] route sysfs_up_message.2 [Start] route sysfs_up_signal.2 [Start] route sysfs area.2 [Info ] route update: 2 became 4 [Info ] update ID: 2 becomes 4 [Start] channel instance sysfs_up_signal from 1 to 4 (#4) [Start] netfilter in dev eth3 [Start] netfilter in f912c574 [Start] netfilter out dev eth3 [Start] netfilter out f912c558 RESET BIT RESET BIT RESET BIT RESET BIT RESET BIT RESET BIT RESET BIT RESET BIT RESET BIT |
From: Mihai C. <m.l...@uv...> - 2009-10-28 07:37:47
|
Good morning, Willem! I tried what you suggested me: commented out the slblock_writable(). Unfortunately, the behaviour didn't change, but I have other error message in dmesg now: "iptransmit out-of-bounds: 2974"? Here is dmesg including stop/start of streamline + FPL modules: Removed FPL component Removed FPL component Removed FPL component Removed FPL component Removed FPL component Removed FPL component [Stop ] porttable [Start] porttable [Stop ] porttable [WARN ] del buffer dev_rx:165 [WARN ] del buffer large:11 [WARN ] del buffer compressed:186 [WARN ] del buffer main:361 [Stop ] channel instance local from 1 to 1 (#1) [Stop ] route local.1 [Stop ] channel sysfs area [Stop ] channel sysfs [WARN ] destroyed class streamline [Stop ] messaging [Stop ] transport [Stop ] datapath [Start] messaging [WARN ] created class streamline [Start] route local.1 [Start] channel instance local from 1 to 1 (#1) DEBUG buf 1 added impl fixed sized slot buffer, now has stacklen 1 BeltwayFS is not mounted. Cannot create node DEBUG buf 1 added impl variable sized slot buffer, now has stacklen 1 BeltwayFS is not mounted. Cannot create node DEBUG buf 1 added impl variable sized slot buffer, now has stacklen 1 BeltwayFS is not mounted. Cannot create node DEBUG buf 1 added impl fixed sized slot buffer, now has stacklen 1 BeltwayFS is not mounted. Cannot create node [Info ] Default DBuf size is 1048576B, default IBuf size is 8192B [Start] transport [Start] porttable [Start] datapath [Start] Splicing v2 enabled Inserted FPL component Inserted FPL component Inserted FPL component Inserted FPL component Inserted FPL component Inserted FPL component [Start] route sysfs.2 [Start] channel instance sysfs from 1 to 2 (#1) [Start] route sysfs_up_message.2 [Start] route sysfs_up_signal.2 [Start] route sysfs area.2 [Info ] route update: 2 became 3 [Info ] update ID: 2 becomes 3 [Start] channel instance sysfs_up_signal from 1 to 3 (#2) [Start] netfilter out dev eth3 [Start] netfilter out f966a558 [Start] netfilter in dev eth3 [Start] netfilter in f966a574 iptransmit out-of-bounds: 2974 iptransmit out-of-bounds: 2974 iptransmit out-of-bounds: 2974 [Stop ] netfilter in dev eth3 [Stop ] netfilter in [Stop ] netfilter out dev eth3 [Stop ] netfilter out [Stop ] channel instance sysfs_up_signal from 1 to 3 (#2) [Stop ] channel instance sysfs from 1 to 3 (#1) [Stop ] route sysfs.3 [Stop ] route sysfs_up_message.3 [Stop ] route sysfs_up_signal.3 [Stop ] route sysfs area.3 [root@DAS2-100-33 coop]# Cheers, Mihai > -----Original Message----- > From: Willem de Bruijn [mailto:wil...@gm...] > Sent: 28 October 2009 01:09 > To: m.l...@uv... > Cc: ffp...@li... > Subject: Re: ip mangling with ipdest/ipsource > > I'm beginning to wonder what timezone you are on, Mihai.. > > > I come again to you with this dmesg that happens while I insert and > > transferred tcp in the manner I told you before (client/server). > > > BeltwayFS is not mounted. Cannot create node > > This is odd. > > > Block alloc failed > > [WARN ] overwrite write acquired failed > > this will cause slblock_writable to return with an error. If your > filter respects that, > it will not modify the packet. If it doesn't, it will modify the > packet in place, which > happens to be exactly what you want. > > In your application, where there are no parallel paths to tbs that may > want to access > the same block and could cause race conditions, it's safe to disable > the check and just > write immediately. Try that and see if your problems go away. |
From: Mihai C. <m.l...@uv...> - 2009-10-27 22:49:01
|
> ok. I don't think the change in syntax will have introduced new bugs > (although we can never be sure). Could you tell me how this old syntax should look like in new syntax? () -all-> (netfilter_toggle) () ? | ? netfilter_toggle |
From: Mihai C. <m.l...@uv...> - 2009-10-27 22:21:58
|
Hi Willem, tcp_csum seems to be ok! If we dont' use it, then the connection is not even established! ip_csum seems to be useless in the way we used it (perhaps there is no check of ip-csum after we modified the ipsource). So I cannot say it works or not. I have another question, about netfilter_toggle: I remember you told me to use in this way last year: (netfilter_fetch_in, expression="eth3") > (tbs,expression="0xAA") -all-> (netfilter_toggle) Where tbs is a classifier: if pkts contains a token = AA, then it passes into user-space; if not -> drop. We now want to use it to replace ipsource: netfilter_fetch_in eth3 | ipsource "10.10.0.1" | netfilter_toggle However, I didn't use the "-all->" stuff. Do we still need it in the new language format? BTW. I also tried in the old language format, but it doesn't change the wrong behaviour with tcp manipulation. (netfilter_fetch_in, expression="eth3") > (ipsource,expression="10.10.0.1") -all-> (netfilter_toggle) Cheers, Mihai |
From: Mihai C. <m.l...@uv...> - 2009-10-21 15:59:35
|
> Just to be certain: you say that SSH traffic continues to work even > while X11 does > not, right? Because an alternative cause could be that X11 forwarding > adds so > much traffic that some resource limits are reached. You did show me > some > out of (ring) memory errors in your original report. I don't think the system should be overloaded by just xeyes (very little graphical application which is almost static on the screen). But, we never know. Here's how it behaves: I do ssh from my laptop into the streamline host. I open xeyes: it works fine. Close it In another terminal, I insert the filter expression (as root). When I try to open xeyes in the first terminal: the prompt is waiting, but I see no graphical application on the screen! If I do ctrl+C, I get back to prompter. If I remove the expression in the root terminal - then try again xeyes shows! Cheers, Mihai |
From: Mihai C. <m.l...@uv...> - 2009-10-21 12:00:36
|
Hi Willem, I checked my filter and I don't see a problem. However, I might do something wrong. Here is the core part of this tokenless filter. Checks IP, IPhdr_len>=7 (for IPoptions), then a specific value 0xCD in IPoptions field. #define eth_contains_ip(data) ( data[12] == 0x8 && !data[13]) #define eth_contains_TokenR(data) (eth_contains_ip(data) && ((data[14]&0xF) >= 0x7) && ((data[34]&0xFF) == 0xCD)) if (!eth_contains_TokenR(rawdata)) return 1; else return 0; Do you see any problem? How can I debug it? I don't know how can I use basic functions to spot what packets pass or not pass the filter, etc? [root@DAS2-100-37 streamline]# slcli -o Streamline CLI Written by Willem de Bruijn Copyright (C) 2007 Vrije Universiteit Amsterdam Version SVN-696 sl > insert (netfilter_fetch_in) > (fpl_tokenless) -all-> (netfilter_toggle) Ok. request=1 sl > remove 1 sl > BTW. Once I remove it, xeyes works again! Moreover, this bug is strange because the filter works on (netfilter_fetch_in), while I don't see X11 on my remote screen (they're not _in packets, but some acks might be!?) Cheers, Mihai > -----Original Message----- > From: Willem de Bruijn [mailto:wil...@gm...] > Sent: 20 October 2009 21:19 > To: m.l...@uv... > Cc: ffp...@li... > Subject: Re: [Ffpf-devel] netfilter_toggle problems > > > I'm not sure I understand it. Why ssh connection works fine, but X11 > over > > ssh doesn't!? > > I don't know either, Mihai. Can it be related to how X11 tunnels > through SSH? > I have no idea how this works. > > > I need an expression that let's all packets which "fpl_tokenless" > returns 1 > > to do their default job (including X11 :). > > The filtering is dead simple. If it works for ssh, I have no idea why > it would > fail for X11-over-ssh. Can the problem be somewhere else? |
From: Mihai C. <m.l...@uv...> - 2009-10-20 19:03:29
|
Willem, I'm not sure I understand it. Why ssh connection works fine, but X11 over ssh doesn't!? I need an expression that let's all packets which "fpl_tokenless" returns 1 to do their default job (including X11 :). We're using other expressions next to this one that would manipulate the traffic according to the built-in token (which is classified by the fpl_tokenless and returns 0). >> insert (netfilter_fetch_in) > (fpl_tokenless) -all-> (netfilter_toggle) > If I'm right, making this expression work by overriding the NF_STOLEN > flag with NF_ACCEPT (i.e., the other way around) is straightforward. > > > Unfortunately, we have the following problem: we can login via ssh, > but we > > never get X11 forwarded! I mention that we properly setup everything > and if > > we don't insert this expression, ssh works fine with X11. Once we > insert > > this expression (the dmesg output is below), we can't see X11 xeyes > anymore, > > while the system doesn't complain at all. > > If all packets get stolen, that seems correct. Mihai |
From: Mihai C. <m.l...@uv...> - 2009-10-20 16:08:42
|
Hi Willem, You did implement a new feature for us last year. We recently found some problems, as follows. The feature we asked was related to have a way to put a filter which decides on which packets to let go up to userspace. More specific, we needed a way to say "packets that do not contain tokens in IPoptions field" should go directly to userspace and hence, bypass any specific processing. This is needed to be able to let remote ssh sessions' traffic. For this purpose, we always insert this expression before any other: insert (netfilter_fetch_in) > (fpl_tokenless) -all-> (netfilter_toggle) Unfortunately, we have the following problem: we can login via ssh, but we never get X11 forwarded! I mention that we properly setup everything and if we don't insert this expression, ssh works fine with X11. Once we insert this expression (the dmesg output is below), we can't see X11 xeyes anymore, while the system doesn't complain at all. Even if we remove the expression and try again, xeyes never shows up. Only after a clean reboot - we can have it back. I see some error/warning messages below, and you might guess what the problem is? [Start] route sysfs.2 [Start] channel instance sysfs from 1 to 2 (#3) [Start] route sysfs_up_message.2 [Start] route sysfs_up_signal.2 [Start] route sysfs area.2 [Info ] route update: 2 became 4 [Info ] update ID: 2 becomes 4 [Start] channel instance sysfs_up_signal from 1 to 4 (#4) [Start] netfilter in dev "all" [Start] netfilter in f9151f74 Block alloc failed [WARN ] overwrite write acquired failed Block alloc failed [WARN ] overwrite write acquired failed Block alloc failed [WARN ] overwrite write acquired failed Block alloc failed [WARN ] overwrite write acquired failed [demo@DAS2-100-34 ~]$ Cheers, Mihai |
From: Mihai C. <m.l...@uv...> - 2009-10-14 20:14:43
|
Willem, >Btw: did the > module restart hacks fix your problems? I solved the module restart problem by commenting out the try_module_get/module_put calls in FPL filters. I'm not sure that your hack helped this. You may put it back. I understood that this module_put() of a filter is not called when we close or kill slcli. Cheers, Mihai |
From: Willem de B. <w.d...@fe...> - 2009-06-23 14:22:19
|
Hi Jan, Sorry for the delay. I would like to create the demos you requested, but really don't have the time write now. Do you need to evaluate this on short notice? Next month I'll have more time to work on Streamline. > > > On the demo page at netstreamline.org, you mention > > > that you have plans to create a demo with filter programming: > > > please consider this a request for this demo. Ok. For now, there's some information on filter programming in the development manual http://netstreamline.org/doc/devman.php#func some of the interfaces will have changed a bit since writing the manual, but the main points are still valid. I'll see if I can cook up a video of a quick filter programming exercise. > The third example is excellent, but it would be helpful to see > demonstrations of some of the other functions built into the CLI, > since I haven't found much documentation for them . (e.g., > I first tried the "info buffer" command to try to read a buffer's > contents). Good idea. > I'd like to see a live filter demo (not one using a pcap-dump file) -- > where packets are captured and processed. For example, changing the > destination port of the packet, or changing the L2 address so a different > interface is used. You can try a live version immediately: pcap input is replaced by live data by issuing rx | [rest of query] instead of pcapin tracefile | [rest of query] while having the kernel module loaded (in a kernel with netfilter support). willem |
From: Mihai C. <m.l...@uv...> - 2009-06-07 19:54:55
|
Hi Jan, Here you have my answers: > 1. On a multiple-CPU system NOT using an Intel IXP NP, is it > possible to use the SPLIT construct? (Specifically, to > load-balance CPU usage.) Could you give an example? Unfortunately, not. SPLIT assumes you have different hosts that run streamline. > 3. Do you have any documentation (informal or otherwise) that > gives a list of EXTERN functions already available in > streamline (and some information about their interfaces). > For example, SavePktData[TU][CD]P seems to save data > using information in M[0], M[1], M[2], and perhaps R[0]. I had a look for the specific example of EXTERN you mentioned, but I couldn't find any. I've thought they were included in streamline fpl. So, there is no EXTERN functions in streamline. I made several functions in the past, but they're custom code which need to be inserted in the kernel before loading or using the FPL function that calls the EXTERN. For API, you may look at: src/functions/fpl/fpl_compiler/fpl_gcc_template.h for definitions of EXTFUNC, and any filter you compiled (without EXTERN used) - has EXTFUNC g_structExternFunctions = {0, { {"",0,0,NULL} }}; where you can find out the syntax needed by EXTERN as they'll be declared global once used in your code. Regards, Mihai |
From: Willem de B. <w.d...@fe...> - 2009-06-07 09:12:07
|
Hi Jan, > I sent this letter to the FFPF-devel mailing list > at Source Forge, but haven't received a copy yet, > nor have I seen this message show up at netstreamline.org. Sorry about that. Due to a barrage of spam, I decided to automatically drop posts by non members. The announcement of that policy was buried deep in the news thread. I've now put a clear notice at the top of the page. > I have several questions after trying some of the > built-in functions and some of the sample filters. > > 1. On a multiple-CPU system NOT using an Intel IXP NP, is it > possible to use the SPLIT construct? (Specifically, to > load-balance CPU usage.) Could you give an example? Do you mean the language feature of the FPL packet language? I should explain that I am the maintainer of the streamline package, but that another former PhD student is the principal author of the FPL language and its compiler. Mihai Cristea is better capable of answering this question (I've cc:ed him) > 2. How does one examine the contents of the buffers? (e.g., > for debugging and verifying a filter.) Buffering is by default opaque to the user, so that the system is free to implement it as efficient as possible. If you want to access a stream, you first have to explicitly say so. For instance, if you write an application in C, insert a request as follows: sd = slrequest_add("(netfilter_in) > (fpl3, export=yes, name=mybuf) > (accept)") then access the buffer through Streamline's equivalent of the common Posix open/read/write/close functions: fd = sl_open("mybuf", O_RDONLY); if (fd >= 0) sl_read(fd, ....etcetera... You can find this simple example in src/apps/demo/programming.c > 3. Do you have any documentation (informal or otherwise) that > gives a list of EXTERN functions already available in > streamline (and some information about their interfaces). > For example, SavePktData[TU][CD]P seems to save data > using information in M[0], M[1], M[2], and perhaps R[0]. Mihai? > On the demo page at netstreamline.org, you mention > that you have plans to create a demo with filter programming: > please consider this a request for this demo. Which page is this? I think that's probably outdated or FPL related. To get a feel for streamline, you can start by trying the examples from http://netstreamline.org/doc/examples.php If you are interested in a specific kind of demo, let me know and I'll see if it's easy to cook up. > Thanks in advance, > > Jan Ariyasu Thanks for trying out the software, Jan. cheers, Willem |
From: Willem de B. <w.d...@fe...> - 2009-02-02 11:20:13
|
Hi all, Saturday I released a new 1.8 version and today created the first update. From the release notes: "This version updates the stream language to be a superset of Unix shell pipelines and improves support for hybrid pipelines consisting of streamline filters (e.g., in the network stack) and full Unix processes. Because of the extensive interface changes, this version is less mature than 1.7.4.5 and has little documentation (other than the mailing list)." Willem ps: note that only the CLI uses the new grammar. The old langauge is still available by running slcli -o and is still the default for applications that execute slrequest_add directly (because all unittests use this). |
From: Willem de B. <w.d...@fe...> - 2008-10-16 07:07:37
|
Hi all, I just released a new version of Streamline. Most changes date back to August, I forgot to roll a release before I went traveling. From the announcement: "This is mainly a bugfix release for PipesFS, which proved to have many (even shameful) bugs in its first release. All known issues but one have been resolved. Unrelated new features include the mpipe() call for multi-consumer pipes, an interface to directly manipulate active filters, and an expanded automated test set to minimize future regressions." Changelog at https://gforge.cs.vu.nl/frs/shownotes.php?release_id=111 cheers, Willem |
From: Mihai C. <M.L...@uv...> - 2008-09-10 07:34:24
|
Willem, I checked the memory alloc: the _new function allocates a 10Bytes slot (see attached sources), and the entry I write with "slnode_setdata" has 8 bytes. It's a bit close, but it shouldn't be a problem. Anything else I should investigate? Mihai > -----Original Message----- > From: Willem de Bruijn [mailto:w.d...@fe...] > Sent: 10 September 2008 03:54 > To: m.l...@uv... > Cc: ffp...@li... > Subject: Re: [Ffpf-devel] Kernel crash > > Mihai, > > > > BUG: unable to handle kernel NULL pointer dereference at 000002d0 > > [f9206863]: streamline:slrun+0x12b/0x162 > > > The crash probably doesn't occur directly in slrun(), but in your > function, right? > Is it possible that you have not allocated a memory region in fpl_new? > Because > you do reference the pointer to the function's memory region in > fpl_process, exactly > in the code that you see is causing the problem. > > > int fpl_process2(struct iptr *iptr, struct fptr *fptr, const struct > > functiondata *fdata) > > { > > > [...] > > if (fdata != NULL) > > { > > g_MemPublic.lBuffer = (long*)fdata->mem.data; > > pEntry = (TBS_ENTRY*)fdata->mem.data; > > } > > > [...] > > if (GetToken() == pEntry[0].Token) // 0x626C7565 "blue" > > { > > printk("blue: "); > > iph->daddr = htonl(pEntry[0].IPdest); > > |
From: Willem de B. <w.d...@fe...> - 2008-09-10 01:57:57
|
Mihai, > BUG: unable to handle kernel NULL pointer dereference at 000002d0 > [f9206863]: streamline:slrun+0x12b/0x162 > The crash probably doesn't occur directly in slrun(), but in your function, right? Is it possible that you have not allocated a memory region in fpl_new? Because you do reference the pointer to the function's memory region in fpl_process, exactly in the code that you see is causing the problem. > int fpl_process2(struct iptr *iptr, struct fptr *fptr, const struct > functiondata *fdata) > { > [...] > if (fdata != NULL) > { > g_MemPublic.lBuffer = (long*)fdata->mem.data; > pEntry = (TBS_ENTRY*)fdata->mem.data; > } > [...] > if (GetToken() == pEntry[0].Token) // 0x626C7565 "blue" > { > printk("blue: "); > iph->daddr = htonl(pEntry[0].IPdest); > |
From: Mihai C. <M.L...@uv...> - 2008-09-09 14:31:21
|
Hi Willem, It seems that the crash I met occasionally it becomes important: It happens every few times of running tbs.c user-tool (attached). The result of crash I hand-copied from the terminal screen of the DAS2 node to here: BUG: unable to handle kernel NULL pointer dereference at 000002d0 [f9206863]: streamline:slrun+0x12b/0x162 Oops: 0000 [#1SMP] Modules linked in: tbs(F) streamline nfs .... [last unloaded:tbs] Pid:0, comm.: swapper Tainted: GF (2.6.25 #2) [f9206863] EFLAGS: 00010296 CPU:1 Unfortunately, I don't know a way to do ksymoops because it crashes completely. However, we know that it comes from tbs. Here is the kernel module: (The complete sources are in das1:/opt/streamline/src/functions/fpl, as you know) int fpl_process2(struct iptr *iptr, struct fptr *fptr, const struct functiondata *fdata) { //DECLARE_REG //DECLARE_SHAREDMEM int dRes = 0; TBS_ENTRY* pEntry=NULL; struct ethhdr *eth; struct iphdr *iph; unsigned int* pIPdest = NULL; if (fptr->len < 34) { printk("Pkt too small for fpl\n"); return 0; } if (fdata != NULL) { g_MemPublic.lBuffer = (long*)fdata->mem.data; pEntry = (TBS_ENTRY*)fdata->mem.data; } eth = (struct ethhdr *) fptr->data; iph = (struct iphdr *) (unsigned long)&fptr->data[ETH_HLEN]; pIPdest = (unsigned int*)&fptr->data[30]; g_pPkt = (char*) &fptr->data[ETH_HLEN]; if (eth_contains_TokenR(fptr->data)) { printk("Token_pkt: 0x%X -> Token_new 0x%X\t\t", GetToken(), pEntry[0].Token); printk("IPdest_pkt: 0x%X -> IPdest_new: 0x%X\n", *pIPdest, pEntry[0].IPdest); // return 0; // for crash-debug: if I return here -> no crash!!! if (GetToken() == pEntry[0].Token) // 0x626C7565 "blue" { printk("blue: "); iph->daddr = htonl(pEntry[0].IPdest); iph->check = 0; return 0; // for crash-debug: avoids sending pkts! 3; } } return 0; ================= An interesting observation: if I return 0 before the last "if", then I haven't seen any crash. Once we do the if -> after 2 or 3 runs => it crashes. The "if" checks whether the built-in token (from IPoptions) matches the first entry in the shared memory (that was setup by the tbs.c user-tool) Any idea what might be wrong with the "if"? Cheers, Mihai |
From: Willem de B. <w.d...@fe...> - 2008-09-09 03:52:21
|
> Using netfilter_fetch instead of netfilter_in seems to help such that the > host doesn't hang anymore. The difference between the two is that netfilter_in reads all incoming packets and then lets Linux handle them, similar to how libpcap reads packets, while netfilter_fetch grabs control of the packet and never gives it back to Linux. You should use netfilter_fetch. Because skbtransmit alters packet contents, if you use netfilter_in, Linux will likely handle a packet on its reception path that Streamline has modified. This can cause all sorts of weirdness, I guess. I can't guess why your machine hung specifically; if you're interested, run in a console and look at the kernel panic message. > However, tcpdump still reports those packets with checksum=0. I think the > problem is due to what tcpdump "sees" on the monitored interface. I cannot > explain why tcpdump would see the packets between my fpl_tbs function and > the skb_transmit function (that is the only moment when the packets have > their checksum=0). > You probably have tcpdump monitoring both incoming and outgoing traffic, i.e., running unfiltered? Perhaps pcap/bpf is called before Streamline's netfilter_fetch, it schedules the packet for userspace handling, hands it off to Streamline, which mangles it, etc..? Try to run only rx and/or only tx checking and see if this makes a difference. I expect you will see the garbled packet on rx. More important is to verify that the machine does not actually transmit the garbled packet. If it doesn't (as I expect and the tcpdump on das2 indicates), I don't think we should spend time on this. willem |
From: Mihai C. <M.L...@uv...> - 2008-09-08 18:47:31
|
Willem, Using netfilter_fetch instead of netfilter_in seems to help such that the host doesn't hang anymore. I think you could explain us what happened in this particular case of getting packets, replacing their ipdest, and send the packets out over the same NIC. The difference in usage of netfilter_in and the netfilter_fetch seems to avoid the blocking of Linux kernel. However, tcpdump still reports those packets with checksum=0. I think the problem is due to what tcpdump "sees" on the monitored interface. I cannot explain why tcpdump would see the packets between my fpl_tbs function and the skb_transmit function (that is the only moment when the packets have their checksum=0). Cheers, Mihai > -----Original Message----- > From: ffp...@li... [mailto:ffpf-devel- > bo...@li...] On Behalf Of Mihai Cristea > Sent: 08 September 2008 14:22 > To: ffp...@li... > Cc: 'Willem de Bruijn' > Subject: Re: [Ffpf-devel] Testing process in streamline > > > Some more details on what happens now: > > The test scenario: generates vlc packets from das4 (192.168.1.30) towards > das1 (192.168.1.27). das1 has streamline to replace the packets with > IPoptions (IPdest to das2 = 192.168.1.28). > > I monitored the link of the streamline host till it died, and also the > end-host (the one IPdest replaced to be forwarded to). These are the last > messages I captured with tcpdump. > > ================================================== > We can observe that the packets with good ip-checksum are the forwarded > ones > (.27 -> .28), followed by a packet with bad ip-checksum that is the copy > of > the original received packet (.30-> .27)!? > ================================= > Das1 (192.168.1.27): > > 14:05:48.336281 00:02:55:ad:19:c3 > 00:02:55:ad:19:85, ethertype IPv4 > (0x0800), length 1366: (tos 0x0, ttl 64, id 52009, offset 0, flags [DF], > proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 > ), > bad cksum 0 (->4018)!) 192.168.1.30.33520 > 192.168.1.28.search-agent: > UDP, > length 1316 > 0x0000: 4700 0548 cb29 4000 4011 0000 c0a8 011e G..H.)@.@....... > 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... > 0x0020: 052c 3e1a 4700 4514 fecb 50bc bafe 5e65 .,>.G.E...P...^e > 0x0030: d503 d699 2159 be04 bfc3 c06c f884 96ab ....!Y.....l.... > 0x0040: 2ebf 6601 84da cb3c 060e 52b7 74b2 12f1 ..f....<..R.t... > 0x0050: 618b a. > 14:05:48.338593 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 > (0x0800), length 1366: (tos 0x0, ttl 64, id 52009, offset 0, flags [DF], > proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 > )) > 192.168.1.27.33520 > 192.168.1.28.search-agent: UDP, length 1316 > 0x0000: 4700 0548 cb29 4000 4011 401b c0a8 011b G..H.)@.@.@..... > 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... > 0x0020: 052c 0002 4700 4514 fecb 50bc bafe 5e65 .,..G.E...P...^e > 0x0030: d503 d699 2159 be04 bfc3 c06c f884 96ab ....!Y.....l.... > 0x0040: 2ebf 6601 84da cb3c 060e 52b7 74b2 12f1 ..f....<..R.t... > 0x0050: 618b a. > 14:05:48.338601 00:02:55:ad:19:c3 > 00:02:55:ad:19:85, ethertype IPv4 > (0x0800), length 1366: (tos 0x0, ttl 64, id 52010, offset 0, flags [DF], > proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 > ), > bad cksum 0 (->4017)!) 192.168.1.30.33520 > 192.168.1.28.search-agent: > UDP, > length 1316 > 0x0000: 4700 0548 cb2a 4000 4011 0000 c0a8 011e G..H.*@.@....... > 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... > 0x0020: 052c cc80 4700 451b c85f c142 8dad d027 .,..G.E.._.B...' > 0x0030: 97a5 0341 a2b6 d90b fe5c 063f 9d99 a45b ...A.....\.?...[ > 0x0040: f9ee da7b 6108 29ba 196d 6789 1f1f eb5d ...{a.)..mg....] > 0x0050: ac25 .% > 14:05:48.340907 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 > (0x0800), length 1366: (tos 0x0, ttl 64, id 52010, offset 0, flags [DF], > proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 > )) > 192.168.1.27.33520 > 192.168.1.28.search-agent: UDP, length 1316 > 0x0000: 4700 0548 cb2a 4000 4011 401a c0a8 011b G..H.*@.@.@..... > 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... > 0x0020: 052c 0002 4700 451b c85f c142 8dad d027 .,..G.E.._.B...' > 0x0030: 97a5 0341 a2b6 d90b fe5c 063f 9d99 a45b ...A.....\.?...[ > 0x0040: f9ee da7b 6108 29ba 196d 6789 1f1f eb5d ...{a.)..mg....] > 0x0050: ac25 .% > 14:05:48.340915 00:02:55:ad:19:c3 > 00:02:55:ad:19:85, ethertype IPv4 > (0x0800), length 1366: (tos 0x0, ttl 64, id 52011, offset 0, flags [DF], > proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 > ), > bad cksum 0 (->4016)!) 192.168.1.30.33520 > 192.168.1.28.search-agent: > UDP, > length 1316 > 0x0000: 4700 0548 cb2b 4000 4011 0000 c0a8 011e G..H.+@.@....... > 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... > Read from remote host das1: Connection reset by peer > Connection to das1 closed. > > > ================================= > > Here are the packets arrived at the end-host Das2 (192.168.1.28). They're > ok, but with a big delay in sequence number!? (compared to the packets > from > the last seconds of the das1' life). > > 13:58:49.384183 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 > (0x0800), length 1366: (tos 0x0, ttl 64, id 52695, offset 0, flags [DF], > proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 > )) > 192.168.1.27.33520 > 192.168.1.28.1234: UDP, length 1316 > 0x0000: 4700 0548 cdd7 4000 4011 3d6d c0a8 011b G..H..@.@.=m.... > 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... > 0x0020: 052c 0002 4700 451f 5464 338f 4b7c 24b2 .,..G.E.Td3.K|$. > 0x0030: 374b 3c48 a989 80c2 ffeb 60e0 e9cb 8302 7K<H......`..... > 0x0040: ad20 3089 9f6e 82a1 ada2 f3d4 0ab1 2fe0 ..0..n......../. > 0x0050: 5879 Xy > 13:58:49.387474 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 > (0x0800), length 1366: (tos 0x0, ttl 64, id 52696, offset 0, flags [DF], > proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 > )) > 192.168.1.27.33520 > 192.168.1.28.1234: UDP, length 1316 > 0x0000: 4700 0548 cdd8 4000 4011 3d6c c0a8 011b G..H..@.@.=l.... > 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... > 0x0020: 052c 0002 4700 4516 c480 5e14 3caa 403a .,..G.E...^.<.@: > 0x0030: 995e 8e3f d4c9 e235 4d95 6812 41b1 0cb3 .^.?...5M.h.A... > 0x0040: a711 1292 ffe3 9549 f81f fb8b c45f c417 .......I....._.. > 0x0050: 22e8 ". > 13:58:49.388734 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 > (0x0800), length 1366: (tos 0x0, ttl 64, id 52697, offset 0, flags [DF], > proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 > )) > 192.168.1.27.33520 > 192.168.1.28.1234: UDP, length 1316 > 0x0000: 4700 0548 cdd9 4000 4011 3d6b c0a8 011b G..H..@.@.=k.... > 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... > 0x0020: 052c 0002 4700 451d b17f fe92 4261 b041 .,..G.E.....Ba.A > 0x0030: 00f8 92c4 89b0 3f1e b61d 318b a80d ff22 ......?...1...." > 0x0040: f20a 5712 0205 cda3 f1ee 87e2 5fd1 b746 ..W........._..F > 0x0050: cc06 .. > 13:58:49.392136 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 > (0x0800), length 1366: (tos 0x0, ttl 64, id 52698, offset 0, flags [DF], > proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 > )) > 192.168.1.27.33520 > 192.168.1.28.1234: UDP, length 1316 > 0x0000: 4700 0548 cdda 4000 4011 3d6a c0a8 011b G..H..@.@.=j.... > 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... > 0x0020: 052c 0002 4700 4514 e61f 271a 11f8 55e2 .,..G.E...'...U. > 0x0030: e68a 25a1 978c 17fa 610f 518f f301 8a1a ..%.....a.Q..... > 0x0040: 5013 aac4 e39a 1fe8 3a40 c42d 0361 807a P.......:@.-.a.z > 0x0050: c086 .. > 13:58:55.663863 00:02:55:ad:19:c3 > ff:ff:ff:ff:ff:ff, ethertype ARP > (0x0806), length 60: arp who-has 192.168.1.27 tell 192.168.1.30 > 0x0000: 0001 0800 0604 0001 0002 55ad 19c3 c0a8 ..........U..... > 0x0010: 011e 0000 0000 0000 c0a8 011b 0000 0000 ................ > 0x0020: 0000 0000 0000 0000 0000 0000 0000 .............. > 13:58:56.664622 00:02:55:ad:19:c3 > ff:ff:ff:ff:ff:ff, ethertype ARP > (0x0806), length 60: arp who-has 192.168.1.27 tell 192.168.1.30 > 0x0000: 0001 0800 0604 0001 0002 55ad 19c3 c0a8 ..........U..... > 0x0010: 011e 0000 0000 0000 c0a8 011b 0000 0000 ................ > 0x0020: 0000 0000 0000 0000 0000 0000 0000 .............. > ^C > 970 packets captured > 1940 packets received by filter > 0 packets dropped by kernel > > > > > > -----Original Message----- > > From: ffp...@li... [mailto:ffpf-devel- > > bo...@li...] On Behalf Of Mihai Cristea > > Sent: 08 September 2008 14:03 > > To: ffp...@li... > > Cc: 'Willem de Bruijn' > > Subject: Re: [Ffpf-devel] Testing process in streamline > > > > Thanks for this fix. It works fine now with the IPoptions packets. > > > > Unfortunately, I run in a different problem: > > My das host hangs when the user-tool ends. I use this user-tool (tbs.c > as > > attached) to run the streamline fpl_tbs function for IPdest replace. It > > works fine (I see the forwarded packets with the replaced IPdest, but > when > > it should close -> it hangs the entire host that hard that I must do a > > cold-restart. > > > > Any idea why this happens? > > > > Mihai > > > > > > This is part of the fpl_tbs function (that works fine when the filter > > expression was inserted by the tbs user-tool): > > > > ... > > if (fptr->len >= 34 && eth_contains_TokenR(fptr->data)) > > { > > eth = (struct ethhdr *) fptr->data; > > iph = (struct iphdr *) (unsigned long)&fptr->data[ETH_HLEN]; > > > > pToken = (unsigned int*)&fptr->data[36]; > > pIPdest = (unsigned int*)&fptr->data[30]; > > printk("Token_pkt: 0x%X -> Token_new 0x%X\t\t", ntohl(*pToken), > > pEntry[0].Token); > > printk("IPdest_pkt: 0x%X -> IPdest_new: 0x%X\n", *pIPdest, > > pEntry[0].IPdest); > > if (ntohl(*pToken) == pEntry[0].Token) // 0x626C7565 "blue" > > { > > printk("blue: "); > > printk("Mih: IPhdr_len=%d ", iph->ihl); > > iph->daddr = htonl(pEntry[0].IPdest); > > iph->check = htons(0); > > // iph->check = ip_fast_csum((void*) iph, iph->ihl); > > return 3; > > } > > > > } > > return 0; > > > > > > > I expect that some code in Streamline overwrites IP fields after the > ip > > > checksum is generated. The only code > > > that I found so far is in src/functions/transmit, and it is only a few > > > lines. Indeed, here I did spot one occurrence where > > > the offset of the start of the UDP header from the start of the IP > > > header was hardcoded to be 20B and something was > > > written in the UDP header afterwards.... which in the case of a packet > > > with IP options would end up in the wrong place. > > > > > > Revision 548 fixes this issue, but I didn't have time to test it yet. > > > Could you see if it solves your issue? If not, I'll run > > > wireshark and debug the issue myself later on. > > > > > > cheers, > > > > > > Willem > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the > world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Ffpf-devel mailing list > Ffp...@li... > https://lists.sourceforge.net/lists/listinfo/ffpf-devel |
From: Mihai C. <M.L...@uv...> - 2008-09-08 12:22:15
|
Some more details on what happens now: The test scenario: generates vlc packets from das4 (192.168.1.30) towards das1 (192.168.1.27). das1 has streamline to replace the packets with IPoptions (IPdest to das2 = 192.168.1.28). I monitored the link of the streamline host till it died, and also the end-host (the one IPdest replaced to be forwarded to). These are the last messages I captured with tcpdump. ================================================== We can observe that the packets with good ip-checksum are the forwarded ones (.27 -> .28), followed by a packet with bad ip-checksum that is the copy of the original received packet (.30-> .27)!? ================================= Das1 (192.168.1.27): 14:05:48.336281 00:02:55:ad:19:c3 > 00:02:55:ad:19:85, ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 64, id 52009, offset 0, flags [DF], proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 ), bad cksum 0 (->4018)!) 192.168.1.30.33520 > 192.168.1.28.search-agent: UDP, length 1316 0x0000: 4700 0548 cb29 4000 4011 0000 c0a8 011e G..H.)@.@....... 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... 0x0020: 052c 3e1a 4700 4514 fecb 50bc bafe 5e65 .,>.G.E...P...^e 0x0030: d503 d699 2159 be04 bfc3 c06c f884 96ab ....!Y.....l.... 0x0040: 2ebf 6601 84da cb3c 060e 52b7 74b2 12f1 ..f....<..R.t... 0x0050: 618b a. 14:05:48.338593 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 64, id 52009, offset 0, flags [DF], proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 )) 192.168.1.27.33520 > 192.168.1.28.search-agent: UDP, length 1316 0x0000: 4700 0548 cb29 4000 4011 401b c0a8 011b G..H.)@.@.@..... 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... 0x0020: 052c 0002 4700 4514 fecb 50bc bafe 5e65 .,..G.E...P...^e 0x0030: d503 d699 2159 be04 bfc3 c06c f884 96ab ....!Y.....l.... 0x0040: 2ebf 6601 84da cb3c 060e 52b7 74b2 12f1 ..f....<..R.t... 0x0050: 618b a. 14:05:48.338601 00:02:55:ad:19:c3 > 00:02:55:ad:19:85, ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 64, id 52010, offset 0, flags [DF], proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 ), bad cksum 0 (->4017)!) 192.168.1.30.33520 > 192.168.1.28.search-agent: UDP, length 1316 0x0000: 4700 0548 cb2a 4000 4011 0000 c0a8 011e G..H.*@.@....... 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... 0x0020: 052c cc80 4700 451b c85f c142 8dad d027 .,..G.E.._.B...' 0x0030: 97a5 0341 a2b6 d90b fe5c 063f 9d99 a45b ...A.....\.?...[ 0x0040: f9ee da7b 6108 29ba 196d 6789 1f1f eb5d ...{a.)..mg....] 0x0050: ac25 .% 14:05:48.340907 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 64, id 52010, offset 0, flags [DF], proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 )) 192.168.1.27.33520 > 192.168.1.28.search-agent: UDP, length 1316 0x0000: 4700 0548 cb2a 4000 4011 401a c0a8 011b G..H.*@.@.@..... 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... 0x0020: 052c 0002 4700 451b c85f c142 8dad d027 .,..G.E.._.B...' 0x0030: 97a5 0341 a2b6 d90b fe5c 063f 9d99 a45b ...A.....\.?...[ 0x0040: f9ee da7b 6108 29ba 196d 6789 1f1f eb5d ...{a.)..mg....] 0x0050: ac25 .% 14:05:48.340915 00:02:55:ad:19:c3 > 00:02:55:ad:19:85, ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 64, id 52011, offset 0, flags [DF], proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 ), bad cksum 0 (->4016)!) 192.168.1.30.33520 > 192.168.1.28.search-agent: UDP, length 1316 0x0000: 4700 0548 cb2b 4000 4011 0000 c0a8 011e G..H.+@.@....... 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... Read from remote host das1: Connection reset by peer Connection to das1 closed. ================================= Here are the packets arrived at the end-host Das2 (192.168.1.28). They're ok, but with a big delay in sequence number!? (compared to the packets from the last seconds of the das1' life). 13:58:49.384183 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 64, id 52695, offset 0, flags [DF], proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 )) 192.168.1.27.33520 > 192.168.1.28.1234: UDP, length 1316 0x0000: 4700 0548 cdd7 4000 4011 3d6d c0a8 011b G..H..@.@.=m.... 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... 0x0020: 052c 0002 4700 451f 5464 338f 4b7c 24b2 .,..G.E.Td3.K|$. 0x0030: 374b 3c48 a989 80c2 ffeb 60e0 e9cb 8302 7K<H......`..... 0x0040: ad20 3089 9f6e 82a1 ada2 f3d4 0ab1 2fe0 ..0..n......../. 0x0050: 5879 Xy 13:58:49.387474 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 64, id 52696, offset 0, flags [DF], proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 )) 192.168.1.27.33520 > 192.168.1.28.1234: UDP, length 1316 0x0000: 4700 0548 cdd8 4000 4011 3d6c c0a8 011b G..H..@.@.=l.... 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... 0x0020: 052c 0002 4700 4516 c480 5e14 3caa 403a .,..G.E...^.<.@: 0x0030: 995e 8e3f d4c9 e235 4d95 6812 41b1 0cb3 .^.?...5M.h.A... 0x0040: a711 1292 ffe3 9549 f81f fb8b c45f c417 .......I....._.. 0x0050: 22e8 ". 13:58:49.388734 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 64, id 52697, offset 0, flags [DF], proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 )) 192.168.1.27.33520 > 192.168.1.28.1234: UDP, length 1316 0x0000: 4700 0548 cdd9 4000 4011 3d6b c0a8 011b G..H..@.@.=k.... 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... 0x0020: 052c 0002 4700 451d b17f fe92 4261 b041 .,..G.E.....Ba.A 0x0030: 00f8 92c4 89b0 3f1e b61d 318b a80d ff22 ......?...1...." 0x0040: f20a 5712 0205 cda3 f1ee 87e2 5fd1 b746 ..W........._..F 0x0050: cc06 .. 13:58:49.392136 00:02:55:ad:19:85 > 00:02:55:ad:18:97, ethertype IPv4 (0x0800), length 1366: (tos 0x0, ttl 64, id 52698, offset 0, flags [DF], proto: UDP (17), length: 1352, options ( unknown (205) len 6EOL (0) len 1 )) 192.168.1.27.33520 > 192.168.1.28.1234: UDP, length 1316 0x0000: 4700 0548 cdda 4000 4011 3d6a c0a8 011b G..H..@.@.=j.... 0x0010: c0a8 011c cd06 626c 7565 0000 82f0 04d2 ......blue...... 0x0020: 052c 0002 4700 4514 e61f 271a 11f8 55e2 .,..G.E...'...U. 0x0030: e68a 25a1 978c 17fa 610f 518f f301 8a1a ..%.....a.Q..... 0x0040: 5013 aac4 e39a 1fe8 3a40 c42d 0361 807a P.......:@.-.a.z 0x0050: c086 .. 13:58:55.663863 00:02:55:ad:19:c3 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 192.168.1.27 tell 192.168.1.30 0x0000: 0001 0800 0604 0001 0002 55ad 19c3 c0a8 ..........U..... 0x0010: 011e 0000 0000 0000 c0a8 011b 0000 0000 ................ 0x0020: 0000 0000 0000 0000 0000 0000 0000 .............. 13:58:56.664622 00:02:55:ad:19:c3 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: arp who-has 192.168.1.27 tell 192.168.1.30 0x0000: 0001 0800 0604 0001 0002 55ad 19c3 c0a8 ..........U..... 0x0010: 011e 0000 0000 0000 c0a8 011b 0000 0000 ................ 0x0020: 0000 0000 0000 0000 0000 0000 0000 .............. ^C 970 packets captured 1940 packets received by filter 0 packets dropped by kernel > -----Original Message----- > From: ffp...@li... [mailto:ffpf-devel- > bo...@li...] On Behalf Of Mihai Cristea > Sent: 08 September 2008 14:03 > To: ffp...@li... > Cc: 'Willem de Bruijn' > Subject: Re: [Ffpf-devel] Testing process in streamline > > Thanks for this fix. It works fine now with the IPoptions packets. > > Unfortunately, I run in a different problem: > My das host hangs when the user-tool ends. I use this user-tool (tbs.c as > attached) to run the streamline fpl_tbs function for IPdest replace. It > works fine (I see the forwarded packets with the replaced IPdest, but when > it should close -> it hangs the entire host that hard that I must do a > cold-restart. > > Any idea why this happens? > > Mihai > > > This is part of the fpl_tbs function (that works fine when the filter > expression was inserted by the tbs user-tool): > > ... > if (fptr->len >= 34 && eth_contains_TokenR(fptr->data)) > { > eth = (struct ethhdr *) fptr->data; > iph = (struct iphdr *) (unsigned long)&fptr->data[ETH_HLEN]; > > pToken = (unsigned int*)&fptr->data[36]; > pIPdest = (unsigned int*)&fptr->data[30]; > printk("Token_pkt: 0x%X -> Token_new 0x%X\t\t", ntohl(*pToken), > pEntry[0].Token); > printk("IPdest_pkt: 0x%X -> IPdest_new: 0x%X\n", *pIPdest, > pEntry[0].IPdest); > if (ntohl(*pToken) == pEntry[0].Token) // 0x626C7565 "blue" > { > printk("blue: "); > printk("Mih: IPhdr_len=%d ", iph->ihl); > iph->daddr = htonl(pEntry[0].IPdest); > iph->check = htons(0); > // iph->check = ip_fast_csum((void*) iph, iph->ihl); > return 3; > } > > } > return 0; > > > > I expect that some code in Streamline overwrites IP fields after the ip > > checksum is generated. The only code > > that I found so far is in src/functions/transmit, and it is only a few > > lines. Indeed, here I did spot one occurrence where > > the offset of the start of the UDP header from the start of the IP > > header was hardcoded to be 20B and something was > > written in the UDP header afterwards.... which in the case of a packet > > with IP options would end up in the wrong place. > > > > Revision 548 fixes this issue, but I didn't have time to test it yet. > > Could you see if it solves your issue? If not, I'll run > > wireshark and debug the issue myself later on. > > > > cheers, > > > > Willem |
From: Mihai C. <M.L...@uv...> - 2008-09-08 12:03:04
|
Thanks for this fix. It works fine now with the IPoptions packets. Unfortunately, I run in a different problem: My das host hangs when the user-tool ends. I use this user-tool (tbs.c as attached) to run the streamline fpl_tbs function for IPdest replace. It works fine (I see the forwarded packets with the replaced IPdest, but when it should close -> it hangs the entire host that hard that I must do a cold-restart. Any idea why this happens? Mihai This is part of the fpl_tbs function (that works fine when the filter expression was inserted by the tbs user-tool): ... if (fptr->len >= 34 && eth_contains_TokenR(fptr->data)) { eth = (struct ethhdr *) fptr->data; iph = (struct iphdr *) (unsigned long)&fptr->data[ETH_HLEN]; pToken = (unsigned int*)&fptr->data[36]; pIPdest = (unsigned int*)&fptr->data[30]; printk("Token_pkt: 0x%X -> Token_new 0x%X\t\t", ntohl(*pToken), pEntry[0].Token); printk("IPdest_pkt: 0x%X -> IPdest_new: 0x%X\n", *pIPdest, pEntry[0].IPdest); if (ntohl(*pToken) == pEntry[0].Token) // 0x626C7565 "blue" { printk("blue: "); printk("Mih: IPhdr_len=%d ", iph->ihl); iph->daddr = htonl(pEntry[0].IPdest); iph->check = htons(0); // iph->check = ip_fast_csum((void*) iph, iph->ihl); return 3; } } return 0; > I expect that some code in Streamline overwrites IP fields after the ip > checksum is generated. The only code > that I found so far is in src/functions/transmit, and it is only a few > lines. Indeed, here I did spot one occurrence where > the offset of the start of the UDP header from the start of the IP > header was hardcoded to be 20B and something was > written in the UDP header afterwards.... which in the case of a packet > with IP options would end up in the wrong place. > > Revision 548 fixes this issue, but I didn't have time to test it yet. > Could you see if it solves your issue? If not, I'll run > wireshark and debug the issue myself later on. > > cheers, > > Willem |