Menu
▾
▴
como-devel — Development questions, bug reports, patch requests, etc.
You can subscribe to this list here.
| 2005 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(25) |
Jul
(26) |
Aug
(33) |
Sep
(30) |
Oct
(28) |
Nov
(96) |
Dec
(64) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2006 |
Jan
(125) |
Feb
(66) |
Mar
(23) |
Apr
(2) |
May
|
Jun
|
Jul
(1) |
Aug
(2) |
Sep
|
Oct
(1) |
Nov
(1) |
Dec
|
| 2007 |
Jan
|
Feb
(2) |
Mar
(1) |
Apr
(2) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2008 |
Jan
(1) |
Feb
(29) |
Mar
(15) |
Apr
(6) |
May
(6) |
Jun
|
Jul
(8) |
Aug
|
Sep
|
Oct
(3) |
Nov
(3) |
Dec
|
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
(2) |
May
(3) |
Jun
|
Jul
(3) |
Aug
(6) |
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Iannaccone, G. <gia...@in...> - 2009-08-31 23:28:00
|
Yes. but cannot tell how long it will take to get there. -gianluca > -----Original Message----- > From: Oana Goga [mailto:oan...@en...] > Sent: Monday, August 31, 2009 3:18 PM > To: Iannaccone, Gianluca > Cc: com...@li... > Subject: RE: [como-devel] Como at 10Gbit/s > > Hello, > > Thank you for your answer. > > Do you plan to adapt some changes in the new version of Como in order > to support traffic captures at 10Gbit/s? > > Thanks, > Oana > > Quoting "Iannaccone, Gianluca" <gia...@in...>: > > > Hi Oana, > > > > the trace module makes a copy of each packet (with the snaplen > > parameters you can set how many bytes per packet to keep) and then > > writes them to disk (with a second copy). This is a restriction of > > our design that forces one more memory copy. Your tweaks are good > > ones but I am not sure how to improve from there without major > > changes in the core (e.g., the code right makes very poor use of > > multicore) > > > > cheers, > > gianluca > > > > > > > > From: Oana Goga [mailto:oan...@en...] > > Sent: Monday, August 31, 2009 1:21 PM > > To: com...@li... > > Subject: [como-devel] Como at 10Gbit/s > > > > Hello, > > > > Lately I have been working on a driver for GtrcNET-10 box. Device > > that is able to capture traffic from 10Gbit/s links by capturing all > > the packets headers and sending them in datagrams that contain > > multiple headers to a collector machine. > > For testing it I used como v1.5 and it is able to run for 10Gbit/s > > traffic on the link without loss (or very little ones) for the > > majority of the modules. > > But not for the trace module, if I am sending traffic with a speed > > of more than 1Gbit/s I am getting this warnings and errors: > > > > [66833.915407 ca] sorry out of memory for 400048 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:168)! > > [66833.915409 ca] out of memory for trace, skipping pkt > > [66833.915412 ca] sorry out of memory for 400048 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:168)! > > [66833.915415 ca] out of memory for trace, skipping pkt > > [66834.040731 su] WARNING!! CAPTURE (pid 18653) terminated (signal: > > 6) mdl 4/4 > > [66834.027094 ex] error on IPC handle from 8 (-1) > > - up 0d00h03m21s; mem 750/289/1023/2048MB (579); pkts 2641262 drops > > 0; mdl 4/4 > > or > > [65763.171187 ca] sorry out of memory for 4000048 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:168)! > > [65763.171190 ca] out of memory for tuple, skipping pkt > > 4 > > [65763.171193 ca] sorry out of memory for 104 bytes > > (/home/lyon/ogoga/como-1.5/base/memory.c:459)! > > [65763.171196 ca] sorry out of memory for 4000048 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:168)! > > [65763.171200 ca] out of memory for tuple, skipping pkt > > 4 > > [65763.171204 ca] sorry out of memory for 20 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:242)! > > [65763.264745 ex] error on IPC handle from 8 (-1) > > 4 > > - up 0d01h59m41s; mem 868/334/982/2048MB (698); pkts 534161124 drops > > 0; mdl 4/4 > > > > after I tried to do some tweaking: I put the flush interval from > > capture.c at 10 microseconds, the POLL_WAIT at 10 and the maximum > > number of packets in the sniffer at 6200, the error was this: > > > > [72579.970915 ca] flexible flush for trace occurred > > dl 1/1 > > [72579.970915 ca] flexible flush for trace occurred > > l 1/11 > > -- Last message repeated 340 times. > > [72583.388661 ca] sorry out of memory for 2068 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > > [72583.388661 ca] sorry out of memory for 2068 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > > -- Last message repeated 65106 times. > > [72583.525725 ca] flexible flush for trace occurred > > dl 1/1 > > - up 0d00h01m51s; mem 1919/942/2047/2048MB (450119); pkts 16329064 > > drops 0; mdl 1/11 > > while sending traffic at 2Gbit/s. > > > > and link this with traffic at around 10Gbit/s: > > > > [72814.439422 ca] flexible flush for trace occurred > > 15; mdl 1/1 > > [72814.441319 ca] sorry out of memory for 2068 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > > [72814.441319 ca] sorry out of memory for 2068 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > > -- Last message repeated 6199 times. > > [72814.482128 ca] flexible flush for trace occurred > > 15; mdl 1/1 > > [72814.483788 ca] sorry out of memory for 2068 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > > [72814.483788 ca] sorry out of memory for 2068 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > > -- Last message repeated 6199 times. > > [72814.522719 ca] flexible flush for trace occurred > > 15; mdl 1/1 > > [72814.524799 ca] sorry out of memory for 2068 bytes > > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > > - up 0d00h05m59s; mem 0/0/2047/2048MB (0); pkts 54098100 drops > > 440615; mdl 1/1 > > in this case the system is not crushing, but it's not storing many > > packets either. > > > > In the como.conf the memory field is 2048 (I modified in memory.c so > > I can address it). > > > > Do you know what other tweaking could I make in order to be able to > > capture more? > > > > Do you plan to make como compilable on 64bits or make some > > structures that are able to allocate more than 2GB of memory? > > > > The system on which I am working has 8GB RAM and was previously > > tested with MAPI to capture traffic from 10Gbit/s links and it was > > working without loss. > > > > Thanks, > > Oana > > > > > > > > > > > > > > |
|
From: Oana G. <oan...@en...> - 2009-08-31 22:18:06
|
Hello, Thank you for your answer. Do you plan to adapt some changes in the new version of Como in order to support traffic captures at 10Gbit/s? Thanks, Oana Quoting "Iannaccone, Gianluca" <gia...@in...>: > Hi Oana, > > the trace module makes a copy of each packet (with the snaplen > parameters you can set how many bytes per packet to keep) and then > writes them to disk (with a second copy). This is a restriction of > our design that forces one more memory copy. Your tweaks are good > ones but I am not sure how to improve from there without major > changes in the core (e.g., the code right makes very poor use of > multicore) > > cheers, > gianluca > > > > From: Oana Goga [mailto:oan...@en...] > Sent: Monday, August 31, 2009 1:21 PM > To: com...@li... > Subject: [como-devel] Como at 10Gbit/s > > Hello, > > Lately I have been working on a driver for GtrcNET-10 box. Device > that is able to capture traffic from 10Gbit/s links by capturing all > the packets headers and sending them in datagrams that contain > multiple headers to a collector machine. > For testing it I used como v1.5 and it is able to run for 10Gbit/s > traffic on the link without loss (or very little ones) for the > majority of the modules. > But not for the trace module, if I am sending traffic with a speed > of more than 1Gbit/s I am getting this warnings and errors: > > [66833.915407 ca] sorry out of memory for 400048 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:168)! > [66833.915409 ca] out of memory for trace, skipping pkt > [66833.915412 ca] sorry out of memory for 400048 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:168)! > [66833.915415 ca] out of memory for trace, skipping pkt > [66834.040731 su] WARNING!! CAPTURE (pid 18653) terminated (signal: > 6) mdl 4/4 > [66834.027094 ex] error on IPC handle from 8 (-1) > - up 0d00h03m21s; mem 750/289/1023/2048MB (579); pkts 2641262 drops > 0; mdl 4/4 > or > [65763.171187 ca] sorry out of memory for 4000048 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:168)! > [65763.171190 ca] out of memory for tuple, skipping pkt > 4 > [65763.171193 ca] sorry out of memory for 104 bytes > (/home/lyon/ogoga/como-1.5/base/memory.c:459)! > [65763.171196 ca] sorry out of memory for 4000048 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:168)! > [65763.171200 ca] out of memory for tuple, skipping pkt > 4 > [65763.171204 ca] sorry out of memory for 20 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:242)! > [65763.264745 ex] error on IPC handle from 8 (-1) > 4 > - up 0d01h59m41s; mem 868/334/982/2048MB (698); pkts 534161124 drops > 0; mdl 4/4 > > after I tried to do some tweaking: I put the flush interval from > capture.c at 10 microseconds, the POLL_WAIT at 10 and the maximum > number of packets in the sniffer at 6200, the error was this: > > [72579.970915 ca] flexible flush for trace occurred > dl 1/1 > [72579.970915 ca] flexible flush for trace occurred > l 1/11 > -- Last message repeated 340 times. > [72583.388661 ca] sorry out of memory for 2068 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > [72583.388661 ca] sorry out of memory for 2068 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > -- Last message repeated 65106 times. > [72583.525725 ca] flexible flush for trace occurred > dl 1/1 > - up 0d00h01m51s; mem 1919/942/2047/2048MB (450119); pkts 16329064 > drops 0; mdl 1/11 > while sending traffic at 2Gbit/s. > > and link this with traffic at around 10Gbit/s: > > [72814.439422 ca] flexible flush for trace occurred > 15; mdl 1/1 > [72814.441319 ca] sorry out of memory for 2068 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > [72814.441319 ca] sorry out of memory for 2068 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > -- Last message repeated 6199 times. > [72814.482128 ca] flexible flush for trace occurred > 15; mdl 1/1 > [72814.483788 ca] sorry out of memory for 2068 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > [72814.483788 ca] sorry out of memory for 2068 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > -- Last message repeated 6199 times. > [72814.522719 ca] flexible flush for trace occurred > 15; mdl 1/1 > [72814.524799 ca] sorry out of memory for 2068 bytes > (/home/lyon/ogoga/como-1.5/base/capture.c:437)! > - up 0d00h05m59s; mem 0/0/2047/2048MB (0); pkts 54098100 drops > 440615; mdl 1/1 > in this case the system is not crushing, but it's not storing many > packets either. > > In the como.conf the memory field is 2048 (I modified in memory.c so > I can address it). > > Do you know what other tweaking could I make in order to be able to > capture more? > > Do you plan to make como compilable on 64bits or make some > structures that are able to allocate more than 2GB of memory? > > The system on which I am working has 8GB RAM and was previously > tested with MAPI to capture traffic from 10Gbit/s links and it was > working without loss. > > Thanks, > Oana > > > > > > |
|
From: Iannaccone, G. <gia...@in...> - 2009-08-31 21:57:36
|
Hi Oana,
the trace module makes a copy of each packet (with the snaplen parameters you can set how many bytes per packet to keep) and then writes them to disk (with a second copy). This is a restriction of our design that forces one more memory copy. Your tweaks are good ones but I am not sure how to improve from there without major changes in the core (e.g., the code right makes very poor use of multicore)
cheers,
gianluca
From: Oana Goga [mailto:oan...@en...]
Sent: Monday, August 31, 2009 1:21 PM
To: com...@li...
Subject: [como-devel] Como at 10Gbit/s
Hello,
Lately I have been working on a driver for GtrcNET-10 box. Device that is able to capture traffic from 10Gbit/s links by capturing all the packets headers and sending them in datagrams that contain multiple headers to a collector machine.
For testing it I used como v1.5 and it is able to run for 10Gbit/s traffic on the link without loss (or very little ones) for the majority of the modules.
But not for the trace module, if I am sending traffic with a speed of more than 1Gbit/s I am getting this warnings and errors:
[66833.915407 ca] sorry out of memory for 400048 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:168)!
[66833.915409 ca] out of memory for trace, skipping pkt
[66833.915412 ca] sorry out of memory for 400048 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:168)!
[66833.915415 ca] out of memory for trace, skipping pkt
[66834.040731 su] WARNING!! CAPTURE (pid 18653) terminated (signal: 6) mdl 4/4
[66834.027094 ex] error on IPC handle from 8 (-1)
- up 0d00h03m21s; mem 750/289/1023/2048MB (579); pkts 2641262 drops 0; mdl 4/4
or
[65763.171187 ca] sorry out of memory for 4000048 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:168)!
[65763.171190 ca] out of memory for tuple, skipping pkt 4
[65763.171193 ca] sorry out of memory for 104 bytes (/home/lyon/ogoga/como-1.5/base/memory.c:459)!
[65763.171196 ca] sorry out of memory for 4000048 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:168)!
[65763.171200 ca] out of memory for tuple, skipping pkt 4
[65763.171204 ca] sorry out of memory for 20 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:242)!
[65763.264745 ex] error on IPC handle from 8 (-1) 4
- up 0d01h59m41s; mem 868/334/982/2048MB (698); pkts 534161124 drops 0; mdl 4/4
after I tried to do some tweaking: I put the flush interval from capture.c at 10 microseconds, the POLL_WAIT at 10 and the maximum number of packets in the sniffer at 6200, the error was this:
[72579.970915 ca] flexible flush for trace occurred dl 1/1
[72579.970915 ca] flexible flush for trace occurred l 1/11
-- Last message repeated 340 times.
[72583.388661 ca] sorry out of memory for 2068 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:437)!
[72583.388661 ca] sorry out of memory for 2068 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:437)!
-- Last message repeated 65106 times.
[72583.525725 ca] flexible flush for trace occurred dl 1/1
- up 0d00h01m51s; mem 1919/942/2047/2048MB (450119); pkts 16329064 drops 0; mdl 1/11
while sending traffic at 2Gbit/s.
and link this with traffic at around 10Gbit/s:
[72814.439422 ca] flexible flush for trace occurred 15; mdl 1/1
[72814.441319 ca] sorry out of memory for 2068 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:437)!
[72814.441319 ca] sorry out of memory for 2068 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:437)!
-- Last message repeated 6199 times.
[72814.482128 ca] flexible flush for trace occurred 15; mdl 1/1
[72814.483788 ca] sorry out of memory for 2068 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:437)!
[72814.483788 ca] sorry out of memory for 2068 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:437)!
-- Last message repeated 6199 times.
[72814.522719 ca] flexible flush for trace occurred 15; mdl 1/1
[72814.524799 ca] sorry out of memory for 2068 bytes (/home/lyon/ogoga/como-1.5/base/capture.c:437)!
- up 0d00h05m59s; mem 0/0/2047/2048MB (0); pkts 54098100 drops 440615; mdl 1/1
in this case the system is not crushing, but it's not storing many packets either.
In the como.conf the memory field is 2048 (I modified in memory.c so I can address it).
Do you know what other tweaking could I make in order to be able to capture more?
Do you plan to make como compilable on 64bits or make some structures that are able to allocate more than 2GB of memory?
The system on which I am working has 8GB RAM and was previously tested with MAPI to capture traffic from 10Gbit/s links and it was working without loss.
Thanks,
Oana
|
|
From: Oana G. <oan...@en...> - 2009-08-31 20:21:24
|
Hello,
Lately I have been working on a driver for GtrcNET-10 box. Device that
is able to capture traffic from 10Gbit/s links by capturing all the
packets headers and sending them in datagrams that contain multiple
headers to a collector machine.
For testing it I used como v1.5 and it is able to run for 10Gbit/s
traffic on the link without loss (or very little ones) for the majority
of the modules.
But not for the trace module, if I am sending traffic with a speed of
more than 1Gbit/s I am getting this warnings and errors:
[66833.915407 ca] sorry out of memory for 400048 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:168)!
[66833.915409 ca] out of memory for trace, skipping pkt
[66833.915412 ca] sorry out of memory for 400048 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:168)!
[66833.915415 ca] out of memory for trace, skipping pkt
[66834.040731 su] WARNING!! CAPTURE (pid 18653) terminated (signal: 6)
mdl 4/4
[66834.027094 ex] error on IPC handle from 8 (-1)
- up 0d00h03m21s; mem 750/289/1023/2048MB (579); pkts 2641262 drops 0;
mdl 4/4
or
[65763.171187 ca] sorry out of memory for 4000048 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:168)!
[65763.171190 ca] out of memory for tuple, skipping
pkt 4
[65763.171193 ca] sorry out of memory for 104 bytes
(/home/lyon/ogoga/como-1.5/base/memory.c:459)!
[65763.171196 ca] sorry out of memory for 4000048 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:168)!
[65763.171200 ca] out of memory for tuple, skipping
pkt 4
[65763.171204 ca] sorry out of memory for 20 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:242)!
[65763.264745 ex] error on IPC handle from 8 (-1) *
* 4
- up 0d01h59m41s; mem 868/334/982/2048MB (698); pkts 534161124 drops 0;
mdl 4/4
after I tried to do some tweaking: I put the flush interval from
capture.c at 10 microseconds, the POLL_WAIT at 10 and the maximum
number of packets in the sniffer at 6200, the error was this:
[72579.970915 ca] flexible flush for trace
occurred dl 1/1
[72579.970915 ca] flexible flush for trace
occurred l 1/11
-- Last message repeated 340 times.
[72583.388661 ca] sorry out of memory for 2068 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:437)!
[72583.388661 ca] sorry out of memory for 2068 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:437)!
-- Last message repeated 65106 times.
[72583.525725 ca] flexible flush for trace
occurred dl 1/1
- up 0d00h01m51s; mem 1919/942/2047/2048MB (450119); pkts 16329064 drops
0; mdl 1/11
while sending traffic at 2Gbit/s.
and link this with traffic at around 10Gbit/s:
[72814.439422 ca] flexible flush for trace
occurred 15; mdl 1/1
[72814.441319 ca] sorry out of memory for 2068 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:437)!
[72814.441319 ca] sorry out of memory for 2068 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:437)!
-- Last message repeated 6199 times.
[72814.482128 ca] flexible flush for trace
occurred 15; mdl 1/1
[72814.483788 ca] sorry out of memory for 2068 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:437)!
[72814.483788 ca] sorry out of memory for 2068 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:437)!
-- Last message repeated 6199 times.
[72814.522719 ca] flexible flush for trace
occurred 15; mdl 1/1
[72814.524799 ca] sorry out of memory for 2068 bytes
(/home/lyon/ogoga/como-1.5/base/capture.c:437)!
- up 0d00h05m59s; mem 0/0/2047/2048MB (0); pkts 54098100 drops 440615;
mdl 1/1
in this case the system is not crushing, but it's not storing many
packets either.
In the como.conf the memory field is 2048 (I modified in memory.c so I
can address it).
Do you know what other tweaking could I make in order to be able to
capture more?
Do you plan to make como compilable on 64bits or make some structures
that are able to allocate more than 2GB of memory?
The system on which I am working has 8GB RAM and was previously tested
with MAPI to capture traffic from 10Gbit/s links and it was working
without loss.
Thanks,
Oana
|
|
From: Iannaccone, G. <gia...@in...> - 2009-08-10 20:08:23
|
I don't think Como on 64bit OS has ever been tested. I'm not surprised of seeing weird behaviors... -gianluca From: Oana Goga [mailto:oan...@en...] Sent: Monday, August 10, 2009 12:07 PM To: com...@li... Subject: [como-devel] Como on x86_64 Hello, I am trying to run CoMo v1.5 on a system with a: - 64bits processor : Dual-Core AMD Opteron 2222 and - kernel: Linux 2.6.26-2-amd64 #1 SMP Thu May 28 21:28:49 UTC 2009 x86_64 GNU/Linux. The installation works without problems, but, when I am trying to analyze a pcap trace file (example-trace.pcap) it doesn't read any packet, and I can't see any error in the log: $ /usr/local/bin/como -c como.conf ---------------------------------------------------- CoMo v1.0 (built Aug 10 2009 20:34:04) Copyright (c) 2004-2007, Intel Corporation All rights reserved. ---------------------------------------------------- ... log level: UI WARN STORAGE CAPTURE EXPORT QUERY SNIFFER TIMER ... sniffer [pcap] /usr/local/etc/como/example-trace.pcap [67813.091157 su] allocated 2048 MB of mapped memory [67813.091311 su] new_mem need 15184 have 4096 [67813.091384 su] create_socket SERVER [/tmp/comot5xAHw/SUPERVISOR.sock] 3 [67813.091405 su] create_socket SERVER [/tmp/comot5xAHw/STORAGE.sock] 4 [67813.091427 su] create_socket SERVER [/tmp/comot5xAHw/CAPTURE.sock] 5 [67813.108745 su] create_socket SERVER [http://localhost:44444/] 6 [67813.108880 su] new_mem need 208 have 512 [67813.108903 su] new_mem need 263304 have 16384 [67813.109823 su] new_mem need 8 have 8 [67813.109832 su] new_mem need 12 have 2 [67813.109843 su] new_mem need 4 have 4 [67813.109852 su] new_mem need 56 have 8 [67813.109861 su] new_mem need 80 have 4 [67813.109867 su] new_mem need 20 have 2 [67813.109873 su] new_mem need 100 have 16 [67813.109878 su] new_mem need 20 have 8 [67813.109885 su] new_mem need 88 have 4 [67813.109889 su] new_mem need 20 have 2 [67813.109894 su] new_mem need 56 have 16 [67813.109899 su] new_mem need 100 have 8 [67813.109905 su] new_mem need 19 have 4 [67813.109956 su] new_mem need 208 have 32 [67813.109966 su] new_mem need 11552 have 512 [67813.109992 su] new_mem need 56 have 8 [67813.109998 su] new_mem need 100 have 16 [67813.110003 su] new_mem need 19 have 2 [67813.110054 su] new_mem need 208 have 8 [67813.110063 su] new_mem need 20 have 4 [67813.110072 su] new_mem need 56 have 32 [67813.110079 su] new_mem need 100 have 8 [67813.110084 su] new_mem need 19 have 2 [67813.110132 su] new_mem need 208 have 16 [67813.110142 su] new_mem need 1048592 have 65536 [67813.113404 su] new_mem need 56 have 4 [67813.113414 su] new_mem need 100 have 8 [67813.113421 su] new_mem need 20 have 64 [67813.113428 su] new_mem need 88 have 4 [67813.113433 su] new_mem need 20 have 2 [67813.113477 su] new_mem need 208 have 8 [67813.113487 su] new_mem need 8 have 16 [67813.113494 su] new_mem need 56 have 4 [67813.113501 su] new_mem need 80 have 8 [67813.113506 su] new_mem need 20 have 2 [67813.113555 su] new_mem need 208 have 32 [67813.113565 su] new_mem need 16 have 4 [67813.113573 su] new_mem need 56 have 8 [67813.113579 su] new_mem need 100 have 16 [67813.113584 su] new_mem need 19 have 2 [67813.113590 su] new_mem need 120 have 8 [67813.113595 su] new_mem need 19 have 4 [67813.113600 su] new_mem need 108 have 64 [67813.113606 su] new_mem need 19 have 2 [67813.113614 su] new_mem need 56 have 8 [67813.113619 su] new_mem need 132 have 16 [67813.113626 su] new_mem need 18 have 4 [67813.113631 su] new_mem need 152 have 8 [67813.113637 su] new_mem need 18 have 2 [67813.113643 su] new_mem need 140 have 32 [67813.113649 su] new_mem need 18 have 8 [67813.113709 su] new connection from peer STORAGE on fd 7 [67813.091846 st] new connection from peer EXPORT on fd 7 [67813.113768 su] new connection from peer EXPORT on fd 8 [67813.091816 ex] create_socket CLIENT [/tmp/comot5xAHw/STORAGE.sock] 7 [67813.113808 su] new connection from peer CAPTURE on fd 9 [67813.091826 ex] create_socket CLIENT [/tmp/comot5xAHw/CAPTURE.sock] 8 [67813.092114 ca] sniffer-pcap: opening file /usr/local/etc/como/example-trace.pcap [67813.092152 ca] datalink Ethernet (1) [67813.092198 ca] new connection from peer EXPORT on fd 8 [67813.114020 ca] module apps activated with filter ip [67813.114112 ca] module protocol activated with filter ( ip ) and (ip) [67813.114181 ca] module topaddr activated with filter ( ip ) and (ip) [67813.114239 ca] module topports activated with filter ( udp or tcp ) and (tcp or udp) [67813.114301 ca] module traffic activated with filter all [67813.114358 ca] module tuple activated with filter ( ip ) and (ip) [67813.139904 st] new client for [/capture/analyse_oana/data/apps], id 0 [67813.140460 ca] sniffers enabled [67813.140040 st] new client for [/capture/analyse_oana/data/protocol], id 1 [67813.140467 ca] new_mem need 65536 have 32768 [67813.140146 st] new client for [/capture/analyse_oana/data/topaddr], id 2 [67813.140244 st] new client for [/capture/analyse_oana/data/topports], id 3 [67813.140332 st] new client for [/capture/analyse_oana/data/traffic], id 4 [67813.140438 st] new client for [/capture/analyse_oana/data/tuple], id 5 - up 0d00h04m15s; mem 2/1/2/2048MB (0); pkts 0 drops 0; mdl 6/6 I observed that soon after the launching como, one processor arrives at 100% and stays blocked there. Anyway, when I am trying to capture packets from eth0 with the libpcap sniffer it works perfectly fine. Do you have any idea why I am getting this behavior? Thanks, Oana |
|
From: Oana G. <oan...@en...> - 2009-08-10 19:07:49
|
Hello, I am trying to run CoMo v1.5 on a system with a: - 64bits processor : Dual-Core AMD Opteron 2222 and - kernel: Linux 2.6.26-2-amd64 #1 SMP Thu May 28 21:28:49 UTC 2009 x86_64 GNU/Linux. The installation works without problems, but, when I am trying to analyze a pcap trace file (example-trace.pcap) it doesn't read any packet, and I can't see any error in the log: $ /usr/local/bin/como -c como.conf ---------------------------------------------------- CoMo v1.0 (built Aug 10 2009 20:34:04) Copyright (c) 2004-2007, Intel Corporation All rights reserved. ---------------------------------------------------- ... log level: UI WARN STORAGE CAPTURE EXPORT QUERY SNIFFER TIMER *... sniffer [pcap] /usr/local/etc/como/example-trace.pcap* [67813.091157 su] allocated 2048 MB of mapped memory [67813.091311 su] new_mem need 15184 have 4096 [67813.091384 su] create_socket SERVER [/tmp/comot5xAHw/SUPERVISOR.sock] 3 [67813.091405 su] create_socket SERVER [/tmp/comot5xAHw/STORAGE.sock] 4 [67813.091427 su] create_socket SERVER [/tmp/comot5xAHw/CAPTURE.sock] 5 [67813.108745 su] create_socket SERVER [http://localhost:44444/] 6 [67813.108880 su] new_mem need 208 have 512 [67813.108903 su] new_mem need 263304 have 16384 [67813.109823 su] new_mem need 8 have 8 [67813.109832 su] new_mem need 12 have 2 [67813.109843 su] new_mem need 4 have 4 [67813.109852 su] new_mem need 56 have 8 [67813.109861 su] new_mem need 80 have 4 [67813.109867 su] new_mem need 20 have 2 [67813.109873 su] new_mem need 100 have 16 [67813.109878 su] new_mem need 20 have 8 [67813.109885 su] new_mem need 88 have 4 [67813.109889 su] new_mem need 20 have 2 [67813.109894 su] new_mem need 56 have 16 [67813.109899 su] new_mem need 100 have 8 [67813.109905 su] new_mem need 19 have 4 [67813.109956 su] new_mem need 208 have 32 [67813.109966 su] new_mem need 11552 have 512 [67813.109992 su] new_mem need 56 have 8 [67813.109998 su] new_mem need 100 have 16 [67813.110003 su] new_mem need 19 have 2 [67813.110054 su] new_mem need 208 have 8 [67813.110063 su] new_mem need 20 have 4 [67813.110072 su] new_mem need 56 have 32 [67813.110079 su] new_mem need 100 have 8 [67813.110084 su] new_mem need 19 have 2 [67813.110132 su] new_mem need 208 have 16 [67813.110142 su] new_mem need 1048592 have 65536 [67813.113404 su] new_mem need 56 have 4 [67813.113414 su] new_mem need 100 have 8 [67813.113421 su] new_mem need 20 have 64 [67813.113428 su] new_mem need 88 have 4 [67813.113433 su] new_mem need 20 have 2 [67813.113477 su] new_mem need 208 have 8 [67813.113487 su] new_mem need 8 have 16 [67813.113494 su] new_mem need 56 have 4 [67813.113501 su] new_mem need 80 have 8 [67813.113506 su] new_mem need 20 have 2 [67813.113555 su] new_mem need 208 have 32 [67813.113565 su] new_mem need 16 have 4 [67813.113573 su] new_mem need 56 have 8 [67813.113579 su] new_mem need 100 have 16 [67813.113584 su] new_mem need 19 have 2 [67813.113590 su] new_mem need 120 have 8 [67813.113595 su] new_mem need 19 have 4 [67813.113600 su] new_mem need 108 have 64 [67813.113606 su] new_mem need 19 have 2 [67813.113614 su] new_mem need 56 have 8 [67813.113619 su] new_mem need 132 have 16 [67813.113626 su] new_mem need 18 have 4 [67813.113631 su] new_mem need 152 have 8 [67813.113637 su] new_mem need 18 have 2 [67813.113643 su] new_mem need 140 have 32 [67813.113649 su] new_mem need 18 have 8 [67813.113709 su] new connection from peer STORAGE on fd 7 [67813.091846 st] new connection from peer EXPORT on fd 7 [67813.113768 su] new connection from peer EXPORT on fd 8 [67813.091816 ex] create_socket CLIENT [/tmp/comot5xAHw/STORAGE.sock] 7 [67813.113808 su] new connection from peer CAPTURE on fd 9 [67813.091826 ex] create_socket CLIENT [/tmp/comot5xAHw/CAPTURE.sock] 8 [67813.092114 ca] sniffer-pcap: opening file /usr/local/etc/como/example-trace.pcap [67813.092152 ca] datalink Ethernet (1) [67813.092198 ca] new connection from peer EXPORT on fd 8 [67813.114020 ca] module apps activated with filter ip [67813.114112 ca] module protocol activated with filter ( ip ) and (ip) [67813.114181 ca] module topaddr activated with filter ( ip ) and (ip) [67813.114239 ca] module topports activated with filter ( udp or tcp ) and (tcp or udp) [67813.114301 ca] module traffic activated with filter all [67813.114358 ca] module tuple activated with filter ( ip ) and (ip) [67813.139904 st] new client for [/capture/analyse_oana/data/apps], id 0 [67813.140460 ca] sniffers enabled [67813.140040 st] new client for [/capture/analyse_oana/data/protocol], id 1 [67813.140467 ca] new_mem need 65536 have 32768 [67813.140146 st] new client for [/capture/analyse_oana/data/topaddr], id 2 [67813.140244 st] new client for [/capture/analyse_oana/data/topports], id 3 [67813.140332 st] new client for [/capture/analyse_oana/data/traffic], id 4 [67813.140438 st] new client for [/capture/analyse_oana/data/tuple], id 5 *- up 0d00h04m15s; mem 2/1/2/2048MB (0); pkts 0 drops 0; mdl 6/6* I observed that soon after the launching como, one processor arrives at 100% and stays blocked there. Anyway, when I am trying to capture packets from eth0 with the libpcap sniffer it works perfectly fine. Do you have any idea why I am getting this behavior? Thanks, Oana |
|
From: Oana G. <oan...@en...> - 2009-07-15 09:07:56
|
Hello, I tested the patch and is working correctly, no more memory leak. Thanks, Oana On 15/07/09 0:49, Iannaccone, Gianluca wrote: > Hi Oana, > > thanks for the detailed report. It appears that we are not handling correctly the calls to glob() leading to a large memory leak for directories with many files. > > I've attached a patch that should fix this problem. Unfortunately I cannot test whether it works right now (don't have the right setup here with me). > > cheers, > gianluca > > > >> -----Original Message----- >> From: Oana Goga [mailto:oan...@en...] >> Sent: Tuesday, July 14, 2009 10:46 AM >> To: com...@li... >> Subject: [como-devel] Memory leak problem >> >> Hello, >> >> Lately I have been testing CoMo and I observed a memory leak problem >> when this is running with flowtools sniffer. >> >> I am using CoMo v1.5 because the v2 is not working correctly with >> flowtools sniffer (is only reading files till the current moment, and >> after, when flowtools is generating new files they are not read by >> CoMo). >> >> I observed that the problem is starting after CoMo has finished to read >> all the flowtools files and is just waiting for new ones, and >> eventually reads them when they appear. >> I tested CoMo with valgrind, and this, indeed, showed that there are >> memory leaks: >> >> LEAK SUMMARY: >> ==25325== definitely lost: 8,391,809 bytes in 2,325 blocks. >> ==25325== indirectly lost: 67,726,144 bytes in 1,026,184 blocks. >> ==25325== possibly lost: 13,728 bytes in 14 blocks. >> ==25325== still reachable: 202,062 bytes in 1,721 blocks. >> ==25325== suppressed: 0 bytes in 0 blocks. >> >> >> I attached to the email a png image that is showing the evolution of >> the memory and also the log of valgrind. >> >> My concern is that after a few hours of running CoMo the system crashes >> because of insufficient memory. >> >> Thank you, >> Oana >> >> >> >> >> >> > > |
|
From: Iannaccone, G. <gia...@in...> - 2009-07-14 22:50:02
|
Hi Oana, thanks for the detailed report. It appears that we are not handling correctly the calls to glob() leading to a large memory leak for directories with many files. I've attached a patch that should fix this problem. Unfortunately I cannot test whether it works right now (don't have the right setup here with me). cheers, gianluca > -----Original Message----- > From: Oana Goga [mailto:oan...@en...] > Sent: Tuesday, July 14, 2009 10:46 AM > To: com...@li... > Subject: [como-devel] Memory leak problem > > Hello, > > Lately I have been testing CoMo and I observed a memory leak problem > when this is running with flowtools sniffer. > > I am using CoMo v1.5 because the v2 is not working correctly with > flowtools sniffer (is only reading files till the current moment, and > after, when flowtools is generating new files they are not read by > CoMo). > > I observed that the problem is starting after CoMo has finished to read > all the flowtools files and is just waiting for new ones, and > eventually reads them when they appear. > I tested CoMo with valgrind, and this, indeed, showed that there are > memory leaks: > > LEAK SUMMARY: > ==25325== definitely lost: 8,391,809 bytes in 2,325 blocks. > ==25325== indirectly lost: 67,726,144 bytes in 1,026,184 blocks. > ==25325== possibly lost: 13,728 bytes in 14 blocks. > ==25325== still reachable: 202,062 bytes in 1,721 blocks. > ==25325== suppressed: 0 bytes in 0 blocks. > > > I attached to the email a png image that is showing the evolution of > the memory and also the log of valgrind. > > My concern is that after a few hours of running CoMo the system crashes > because of insufficient memory. > > Thank you, > Oana > > > > > |
|
From: Oana G. <oan...@en...> - 2009-07-14 18:04:18
|
Hello, Lately I have been testing CoMo and I observed a memory leak problem when this is running with flowtools sniffer. I am using CoMo v1.5 because the v2 is not working correctly with flowtools sniffer (is only reading files till the current moment, and after, when flowtools is generating new files they are not read by CoMo). I observed that the problem is starting after CoMo has finished to read all the flowtools files and is just waiting for new ones, and eventually reads them when they appear. I tested CoMo with valgrind, and this, indeed, showed that there are memory leaks: LEAK SUMMARY: ==25325== definitely lost: 8,391,809 bytes in 2,325 blocks. ==25325== indirectly lost: 67,726,144 bytes in 1,026,184 blocks. ==25325== possibly lost: 13,728 bytes in 14 blocks. ==25325== still reachable: 202,062 bytes in 1,721 blocks. ==25325== suppressed: 0 bytes in 0 blocks. I attached to the email a png image that is showing the evolution of the memory and also the log of valgrind. My concern is that after a few hours of running CoMo the system crashes because of insufficient memory. Thank you, Oana |
|
From: Iannaccone, G. <gia...@in...> - 2009-05-20 17:50:52
|
Hi Oana, v2.0 is really unstable and we have not tested sflow ever on that version. I would go back to v1.5 for flowtools and sflow support. That aspect of processing new files was tested and working in v1.5. cheers, gianliuca From: Oana Goga [mailto:oan...@en...] Sent: Wednesday, May 20, 2009 9:52 AM To: com...@li... Subject: [como-devel] Como and sFlow Hello again, I am trying to run como with sflow and I encountered some problems. I enabeled the sflow sniffer in the configuration file: sniffer "sflow" "localhost" "port=3001" but como does not capture any traffic: como -c mycomo.conf [58876.539046 SU ] ---------------------------------------------------- [58876.539087 SU ] CoMo v2.0 #sh: ./comover-helper.sh: Permission denied (built Apr 30 2009 17:57:09) [58876.539111 SU ] Copyright (c) 2004-2006, Intel Corporation [58876.539124 SU ] All rights reserved. [58876.539133 SU ] ---------------------------------------------------- [58876.539142 SU ] ... workdir /tmp/comoOu6eYN [58876.539159 SU ] log level: ERROR WARNING NOTICE MESSAGE DEBUG [58876.539199 SU ] allocated 1073741824 of mapped memory [58876.539874 SU ] Initialized sniffer `sflow` on device `localhost`. [58876.542610 SU ] starting process STORAGE pid 88210/8 [58876.543081 SU ] starting process CAPTURE pid 8822 [58876.544029 CA ] sniffer sflow (localhost) started [58876.547939 CA ] adding module traffic [58876.548068 CA ] adding module flowcount [58876.548179 CA ] adding module protocol [58876.548292 CA ] adding module topaddr [58876.560474 CA ] adding module topports [58876.561211 CA ] adding module trace [58876.561384 CA ] adding module tuple [58876.564474 CA ] adding module apps [58876.568699 SU ] starting process EXPORT pid 8823 8/8 [58876.593033 CA ] starting to capture packets; mdl 8/8 - up 0d00h01m50s; mem 0/0/1024MB (0); pkts 0 drops 0; mdl 8/8 Do you have any idea why is not working? (I am sure that the sflow datagrams are send to this host) I also tried to use flowtools. I am redirecting the sflow datagrams to flowtools: sflowtool -p 3001 -c flows.lyon.grid5000.fr -d 3000 flow-capture -b little -w /home/ogoga/flows/ -S 1 -n 550 0/0/3000 I enabled the flowtools sniffer: sniffer "flowtools" "/home/ogoga/flows/2009/2009-05/*/ft-*" "sampling=128 stream" but the problem here is that como is only reading files till the current moment, if afterwards flowtools is generating other files, they won't be read by como: como -c mycomo1.conf opening file /home/ogoga/flows/2009/2009-05/2009-05-20/ft-v05.2009-05-20.180436+0200 [58096.562366 CA ] sniffing from /home/ogoga/flows/2009/2009-05/*/ft-* no more files to read, but want more going to sleep for 10minutes [58096.582715 CA ] no sniffers left. waiting for queries Is there any way to force como to read the next files? Thank you, Oana |
|
From: Oana G. <oan...@en...> - 2009-05-20 16:52:07
|
Hello again, I am trying to run como with sflow and I encountered some problems. I enabeled the sflow sniffer in the configuration file: sniffer "sflow" "localhost" "port=3001" but como does not capture any traffic: como -c mycomo.conf [58876.539046 SU ] ---------------------------------------------------- [58876.539087 SU ] CoMo v2.0 #sh: ./comover-helper.sh: Permission denied (built Apr 30 2009 17:57:09) [58876.539111 SU ] Copyright (c) 2004-2006, Intel Corporation [58876.539124 SU ] All rights reserved. [58876.539133 SU ] ---------------------------------------------------- [58876.539142 SU ] ... workdir /tmp/comoOu6eYN [58876.539159 SU ] log level: ERROR WARNING NOTICE MESSAGE DEBUG [58876.539199 SU ] allocated 1073741824 of mapped memory [58876.539874 SU ] Initialized sniffer `sflow` on device `localhost`. [58876.542610 SU ] starting process STORAGE pid 88210/8 [58876.543081 SU ] starting process CAPTURE pid 8822 [58876.544029 CA ] sniffer sflow (localhost) started [58876.547939 CA ] adding module traffic [58876.548068 CA ] adding module flowcount [58876.548179 CA ] adding module protocol [58876.548292 CA ] adding module topaddr [58876.560474 CA ] adding module topports [58876.561211 CA ] adding module trace [58876.561384 CA ] adding module tuple [58876.564474 CA ] adding module apps [58876.568699 SU ] starting process EXPORT pid 8823 8/8 [58876.593033 CA ] starting to capture packets; mdl 8/8 *- up 0d00h01m50s; mem 0/0/1024MB (0); pkts 0 drops 0; mdl 8/8 * Do you have any idea why is not working? (I am sure that the sflow datagrams are send to this host) I also tried to use flowtools. I am redirecting the sflow datagrams to flowtools: sflowtool -p 3001 -c flows.lyon.grid5000.fr -d 3000 flow-capture -b little -w /home/ogoga/flows/ -S 1 -n 550 0/0/3000 I enabled the flowtools sniffer: sniffer "flowtools" "/home/ogoga/flows/2009/2009-05/*/ft-*" "sampling=128 stream" but the problem here is that como is only reading files till the current moment, if afterwards flowtools is generating other files, they won't be read by como: como -c mycomo1.conf opening file /home/ogoga/flows/2009/2009-05/2009-05-20/ft-v05.2009-05-20.180436+0200 [58096.562366 CA ] sniffing from /home/ogoga/flows/2009/2009-05/*/ft-* *no more files to read, but want more going to sleep for 10minutes* [58096.582715 CA ] no sniffers left. waiting for queries Is there any way to force como to read the next files? Thank you, Oana |
|
From: Gass, R. <ric...@in...> - 2009-05-01 00:03:35
|
>> Does comolive work with the last version from svn of como? I >am getting >> the following error: "Fatal error: Call to undefined >function: file_put_contents() >> in /var/www/comolive/config/index.php on line 228" > >working on making comolive work... This error looks like you are using an older version of PHP that doesn't include the function "file_put_contents()" Are you using a version >= 5? I thought there was a check in the code for that but it has been a long time since I looked at it. Richard > >thanks, >gianluca > >--------------------------------------------------------------- >--------------- >Register Now & Save for Velocity, the Web Performance & Operations >Conference from O'Reilly Media. Velocity features a full day of >expert-led, hands-on workshops and two days of sessions from industry >leaders in dedicated Performance & Operations tracks. Use code >vel09scf >and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf >_______________________________________________ >como-devel mailing list >com...@li... >https://lists.sourceforge.net/lists/listinfo/como-devel > |
|
From: Iannaccone, G. <gia...@in...> - 2009-04-30 23:13:00
|
Hi Oana, > Documentation > I already read it on the mailing list, but I coming back with the > question, is there any documentation on how to write a new module? > At the lab we are using a specific hardware device for packet capturing > called GNET (http://www.gtrc.aist.go.jp/gnet ), in order to integrate > it with CoMo I need to write a driver for it, is there any documentation > on how to do this? Unfortunately there is very little documentation on how to build modules and sniffers (the code that is needed to work with other hardware devices for capturing packets). However, you can use the counter.c module and sniffer-libpcap.c as examples on how to write them. We are also currently moving to v2.0 that has made major changes both in the modules and the sniffers and (hopefully) we will have more documentation about that. We should be able to release v2.0 around June 2009. > Storage > Also, is it possible to filter the packets from a capture and store > the filtered capture in another file? yes, all you have to do is to add in the config file a module that defines the filter and stores the results in a directory of your choice. You can use the "trace" module for that. > If I enable CoMo to permanently capture packets from sFlow and NetFlow > on 10Gb links, for how long the packets are stored? and the analysis? You can use the "streamsize" parameter in each module for deciding how much disk space to dedicate to storing the packets and the analysis. At the moment you cannot define an amount of time but just disk space. Once the disk space is exhausted, the modules will start overwriting the files (in a circular manner). > Queries > Is it possible to split a capture file (pcap file) in flows and then > make other analysis on this flows? yes, via the replay() callback in the modules that support it. for example a query of the type http://<node>:<port>/topdest?source=tuple will do exactly that (i.e. use the output of the tuple module, that is flows, to compute the top-k destinations). > Is it possible to make modules that computes the distribution of > some metrics like flow size distribution, packet size distribution? yes. > Is it possible to integrate the output of the queries with RRD? there is no support for that right now but it is definitely doable. > I am for sure missing something, but how can I save the output of a > query into a file? you need to issue a query to the node via http and then save to file: e.g. "curl http://<node>:<port>/tuple > output_file" > If I am having a pcap trace file, on which I enabled the traffic module, > how can i get the evolution of the traffic in time, not just the > final results? you can do it two ways. run the query: http://<node>:<port>/traffic?start=0 or run directly como without configuration file like: "como -s pcap:<tracefile> traffic" > Performance > Is it able to capture and analyze packet from 10Gb links without loss? Unlikely. But it depends on the HW you have, the modules you turn on, etc. > Misc > Are there any other types of documentation that will help me in understanding > how the system is build beside the articles on the CoMo page? nope. :( > Is it possible to define a query priority that stops all the other queries? no. > Does comolive work with the last version from svn of como? I am getting > the following error: "Fatal error: Call to undefined function: file_put_contents() > in /var/www/comolive/config/index.php on line 228" working on making comolive work... thanks, gianluca |
|
From: Oana G. <oan...@en...> - 2009-04-22 17:21:48
|
Hello, I am a R&D engineer in Grid'5000 and I am currently working on developing a network monitoring platform for it. I am interested in integrating CoMo in the current platform. For doing this I need your help in the following matters: Documentation I already read it on the mailing list, but I coming back with the question, is there any documentation on how to write a new module? At the lab we are using a specific hardware device for packet capturing called GNET (_http://www.gtrc.aist.go.jp/gnet_ ), in order to integrate it with CoMo I need to write a driver for it, is there any documentation on how to do this? Storage Also, is it possible to filter the packets from a capture and store the filtered capture in another file? If I enable CoMo to permanently capture packets from sFlow and NetFlow on 10Gb links, for how long the packets are stored? and the analysis? Queries Is it possible to split a capture file (pcap file) in flows and then make other analysis on this flows? Is it possible to make modules that computes the distribution of some metrics like flow size distribution, packet size distribution? Is it possible to integrate the output of the queries with RRD? I am for sure missing something, but how can I save the output of a query into a file? If I am having a pcap trace file, on which I enabled the traffic module, how can i get the evolution of the traffic in time, not just the final results? Performance Is it able to capture and analyze packet from 10Gb links without loss? Misc Are there any other types of documentation that will help me in understanding how the system is build beside the articles on the CoMo page? Is it possible to define a query priority that stops all the other queries? Does comolive work with the last version from svn of como? I am getting the following error: "Fatal error: Call to undefined function: file_put_contents() in /var/www/comolive/config/index.php on line 228" Thank you very much, Oana Goga |
|
From: Iannaccone, G. <gia...@in...> - 2008-11-21 20:57:36
|
Hi Luca, thanks for the patches. It looks very reasonable to me. I will commit them in the v2.0 branch soon. Thanks! -gianluca >-----Original Message----- >From: Luca Niccolini [mailto:luc...@gm...] >Sent: Friday, November 21, 2008 1:30 AM >To: com...@li... >Subject: [como-devel] CoMo install package > >Hello, >I am working on CoMo v2.0-onelab and I need to create >both RPM and DEB install packages starting from CoMo sources. >The process to create that kind of packages requires >the program to be installed in a directory like ><src_dir>/debian/<package_name> >before it can be packaged. >Unfortunately the installation process hardcode the value of variable >CMAKE_INSTALL_PREFIX >in como.conf.example and in como-build.h. > >I have introduced a new variable called CMAKE_DEST_PREFIX in which the >final destination >(after the installation via packet manager is done) >of the binaries can be specified. >It defaults to CMAKE_INSTALL_PREFIX if it is not defined. > >In the attachments you could find this little patch. >I think it can be useful for install packages creation. > >Luca |
|
From: Luca N. <luc...@gm...> - 2008-11-21 09:30:32
|
Hello, I am working on CoMo v2.0-onelab and I need to create both RPM and DEB install packages starting from CoMo sources. The process to create that kind of packages requires the program to be installed in a directory like <src_dir>/debian/<package_name> before it can be packaged. Unfortunately the installation process hardcode the value of variable CMAKE_INSTALL_PREFIX in como.conf.example and in como-build.h. I have introduced a new variable called CMAKE_DEST_PREFIX in which the final destination (after the installation via packet manager is done) of the binaries can be specified. It defaults to CMAKE_INSTALL_PREFIX if it is not defined. In the attachments you could find this little patch. I think it can be useful for install packages creation. Luca |
|
From: Luca N. <luc...@gm...> - 2008-11-20 13:31:15
|
Hello, I am working on CoMo v2.0-onelab and I need to create both RPM and DEB install packages starting from CoMo sources. The process to create that kind of packages requires the program to be installed in a directory like <src_dir>/debian/<package_name> before it can be packaged. Unfortunately the installation process hardcode the value of variable CMAKE_INSTALL_PREFIX in como.conf.example and in como-build.h. I have introduced a new variable called CMAKE_DEST_PREFIX in which the final destination (after the installation via packet manager is done) of the binaries can be specified. It defaults to CMAKE_INSTALL_PREFIX if it is not defined. In the attachments you could find this little patch. I think it can be useful for install packages creation. Luca |
|
From: Iannaccone, G. <gia...@in...> - 2008-10-15 00:49:55
|
Hi Luca, unfortunately we don't have any specific documentation for building the sniffers. I'd suggest to look into the sniffer-libpcap.c as a reference. Regarding packet losses, the sniffer cannot guarantee that (it depends on the load of the machine) but all sniffers try to keep track of loss events. Also, there is some basic support for sampling if needed to reduce the data in a controlled fashion. cheers, gianluca >-----Original Message----- >From: Luca Niccolini [mailto:luc...@gm...] >Sent: Tuesday, October 14, 2008 8:51 AM >To: com...@li... >Subject: [como-devel] CoMo sniffer > >Hello, >my name is Luca and I am working on CoMo v2.0 OneLab edition. >I would like to use CoMo to capture packets from a serial device, >thus I need to implement my own sniffer. > >Where can I find some documentation about sniffer implementation? > >Also, the received data represent samples from an ECG signal. >Can I be sure that no packets are lost? > >Thanks in advance. >Luca > >------------------------------------------------------------------------ >- >This SF.Net email is sponsored by the Moblin Your Move Developer's >challenge >Build the coolest Linux based applications with Moblin SDK & win great >prizes >Grand prize is a trip for two to an Open Source event anywhere in the >world >http://moblin-contest.org/redirect.php?banner_id=100&url=/ >_______________________________________________ >como-devel mailing list >com...@li... >https://lists.sourceforge.net/lists/listinfo/como-devel |
|
From: Luca N. <luc...@gm...> - 2008-10-14 15:53:07
|
Hello, my name is Luca and I am working on CoMo v2.0 OneLab edition. I would like to use CoMo to capture packets from a serial device, thus I need to implement my own sniffer. Where can I find some documentation about sniffer implementation? Also, the received data represent samples from an ECG signal. Can I be sure that no packets are lost? Thanks in advance. Luca |
|
From: Luca N. <luc...@gm...> - 2008-10-14 15:51:52
|
Hello, my name is Luca and I am working on CoMo v2.0 OneLab edition. I would like to use CoMo to capture packets from a serial device, thus I need to implement my own sniffer. Where can I find some documentation about sniffer implementation? Also, the received data represent samples from an ECG signal. Can I be sure that no packets are lost? Thanks in advance. Luca |
|
From: Josep S. <jsa...@ac...> - 2008-07-21 19:58:49
|
Hi Angelica, You are you using 2.0, right? I cannot reproduce this in my machine. Make sure that you are using the latest revision from svn, just in case. Have you tried first unloading a module, and then editing the cfg to load the other, just to see if it makes any difference? From the output that you pasted it seems that you are unloading a module and loading an other one at once. Also, does it always happen when you unload a module that you was loaded at launch time? If the bug is still there, it would help if you commented the line that reads "#define LOG_DEBUG_DISABLE" in supervisor.c, then recompile and rerun. Supervisor will now be more verbose about the changes it sees in the config file, hopefully this helps narrow down the problem. Send us the full output and I'll try to further help you. Regards, Josep > >Hi to all! >I am trying to dynamically add/remove a module on CoMo. I have noted a >strange behavior of CoMo. Infact if I remove a module uploaded >statically (that is when CoMo is launched the first time), deletion is >successful, but if I try to remove a dynamically uploaded module the >following error occurs: > >[27394.590335 SU ] new module: `tuple' >[27394.590561 CA ] adding module tuple >[27441.855983 SU ] removing module: `�� > �� > J��MP� > ,� > ' >[27441.856040 CA ] WARNING: deletion of unknown module `�� > �� > J��MP� > ,� > ' > requested >[27441.856076 SU ] FATAL ERROR: capture failed to remove a >module! [27441.856371 ST IPC ] ipc_peer_destroy -- closing fd 4 >[27441.856403 EX IPC ] ipc_peer_destroy -- closing fd 4 >Aborted > >Could someone help me? >My purpose is to delete dynamically uploaded modules. >In particular to upload/remove modules I send to CoMo the following >signal: pkill -HUP como > >Thanks in advance, >Angelica > >_________________________________________________________________ >Invite your mail contacts to join your friends list with Windows Live >Spaces. It's easy! >http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us |
|
From: Angelica Lo D. <ang...@ho...> - 2008-07-21 07:41:13
|
Hi to all!
I am trying to dynamically add/remove a module on CoMo. I have noted a strange behavior of CoMo. Infact if I remove a module uploaded statically (that is when CoMo is launched the first time), deletion is successful, but if I try to remove a dynamically uploaded module the following error occurs:
[27394.590335 SU ] new module: `tuple'
[27394.590561 CA ] adding module tuple
[27441.855983 SU ] removing module: `��
��
J��MP�
,�
'
[27441.856040 CA ] WARNING: deletion of unknown module `��
��
J��MP�
,�
' requested
[27441.856076 SU ] FATAL ERROR: capture failed to remove a module!
[27441.856371 ST IPC ] ipc_peer_destroy -- closing fd 4
[27441.856403 EX IPC ] ipc_peer_destroy -- closing fd 4
Aborted
Could someone help me?
My purpose is to delete dynamically uploaded modules.
In particular to upload/remove modules I send to CoMo the following signal:
pkill -HUP como
Thanks in advance,
Angelica
_________________________________________________________________
Invite your mail contacts to join your friends list with Windows Live Spaces. It's easy!
http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us |
|
From: Angelica Lo D. <ang...@ho...> - 2008-07-11 10:57:12
|
Hi to all!
I'm trying to update my como svn version to the actual (that seems to be the 1261), but when I try to compile it some errors occur:
when I run make and make install all is ok, but when I run make modules the following error occurs:
Building CoMo module autofocus (/home/angelica/CoMo/modules/autofocus)
-- Inferring module dependencies
datadef: file /home/angelica/CoMo/modules/autofocus/data.h
capture: file /home/angelica/CoMo/modules/autofocus/capture.c
init: file /home/angelica/CoMo/modules/autofocus/init.c
export: implemented in mono
query: implemented in mono
CMake Error: module autofocus requires mono
-- Configuring done
Building module
make[1]: Entering directory `/home/angelica/CoMo/debug/modules/autofocus'
make[1]: *** No targets specified and no makefile found. Stop.
make[1]: Leaving directory `/home/angelica/CoMo/debug/modules/autofocus'
Error building autofocus
exit: 36: Illegal number: -1
make: *** [modules] Error 2
In particular another error occurred before:
Building module
[: 82: ==: unexpected operator
make[1]: Entering directory `/home/angelica/CoMo/debug/modules/apps'
make[2]: Entering directory `/home/angelica/CoMo/debug/modules/apps'
make[3]: Entering directory `/home/angelica/CoMo/debug/modules/apps'
but I have filled it replacing in the file template-como_module.sh the old line 82 with the following one:
if [ "$ACTION" = "build" ];
Could someone help me?
Thanks
Angelica
_________________________________________________________________
Discover the new Windows Vista
http://search.msn.com/results.aspx?q=windows+vista&mkt=en-US&form=QBRE |
|
From: Josep S. <jsa...@ac...> - 2008-07-10 16:49:50
|
Hi Angelica, When you press ctrl+c, since you kill the http client, you also break the connection, and como should notice and close its end of the connection. In what way do you see that the connection is not being closed, how do you notice? BTW are you using 2.0 or 1.5? Regards, Josep > >Hi to all! >I would like to know whether there is a method to stop a query, >without press Ctrl+C. In other words, when I launch a query I write: > >curl http://localhost:44444/tuple > >if I want to stop that query I press ctrl+C, but in this way the tcp >connection remains pending (in particular the server waits for a CLOSE >packet). Is there a method to "softly" stop a query? > >Thank you, >Angelica > >_________________________________________________________________ >News, entertainment and everything you care about at Live.com. Get it >now! http://www.live.com/getstarted.aspx |
|
From: Angelica Lo D. <ang...@ho...> - 2008-07-10 14:17:31
|
Hi to all! I would like to know whether there is a method to stop a query, without press Ctrl+C. In other words, when I launch a query I write: curl http://localhost:44444/tuple if I want to stop that query I press ctrl+C, but in this way the tcp connection remains pending (in particular the server waits for a CLOSE packet). Is there a method to "softly" stop a query? Thank you, Angelica _________________________________________________________________ News, entertainment and everything you care about at Live.com. Get it now! http://www.live.com/getstarted.aspx |
