nfdump-discuss Mailing List for NFDUMP - Netflow processing tools (Page 46)
netflow collecting and processing tools
Brought to you by:
phaag
You can subscribe to this list here.
2005 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(3) |
Aug
(4) |
Sep
(1) |
Oct
(1) |
Nov
(4) |
Dec
(4) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2006 |
Jan
(20) |
Feb
(14) |
Mar
(12) |
Apr
(4) |
May
(9) |
Jun
(15) |
Jul
(23) |
Aug
(12) |
Sep
(5) |
Oct
(5) |
Nov
(1) |
Dec
(5) |
2007 |
Jan
(7) |
Feb
(9) |
Mar
(6) |
Apr
(9) |
May
(11) |
Jun
(6) |
Jul
(25) |
Aug
(35) |
Sep
(10) |
Oct
(21) |
Nov
(13) |
Dec
(10) |
2008 |
Jan
(3) |
Feb
(5) |
Mar
(9) |
Apr
(5) |
May
(1) |
Jun
(2) |
Jul
(13) |
Aug
(10) |
Sep
(1) |
Oct
(5) |
Nov
(1) |
Dec
|
2009 |
Jan
|
Feb
(4) |
Mar
(1) |
Apr
(4) |
May
(5) |
Jun
(17) |
Jul
(17) |
Aug
(18) |
Sep
(4) |
Oct
(11) |
Nov
(22) |
Dec
(24) |
2010 |
Jan
(13) |
Feb
(6) |
Mar
(5) |
Apr
(9) |
May
(4) |
Jun
(43) |
Jul
(4) |
Aug
(11) |
Sep
(7) |
Oct
(6) |
Nov
(4) |
Dec
(7) |
2011 |
Jan
(14) |
Feb
(20) |
Mar
(19) |
Apr
(2) |
May
(6) |
Jun
(15) |
Jul
(17) |
Aug
(10) |
Sep
(14) |
Oct
(15) |
Nov
(7) |
Dec
(1) |
2012 |
Jan
(16) |
Feb
(7) |
Mar
(6) |
Apr
(6) |
May
(5) |
Jun
(14) |
Jul
(15) |
Aug
(27) |
Sep
(9) |
Oct
(11) |
Nov
(10) |
Dec
(8) |
2013 |
Jan
(25) |
Feb
(11) |
Mar
(11) |
Apr
(15) |
May
(22) |
Jun
(17) |
Jul
(27) |
Aug
(32) |
Sep
(18) |
Oct
(3) |
Nov
(37) |
Dec
(12) |
2014 |
Jan
(11) |
Feb
(10) |
Mar
(2) |
Apr
(15) |
May
(10) |
Jun
(5) |
Jul
(12) |
Aug
(4) |
Sep
(10) |
Oct
(6) |
Nov
(11) |
Dec
(3) |
2015 |
Jan
(7) |
Feb
(6) |
Mar
(8) |
Apr
(9) |
May
(12) |
Jun
(1) |
Jul
(16) |
Aug
(18) |
Sep
(11) |
Oct
(12) |
Nov
(15) |
Dec
(3) |
2016 |
Jan
(2) |
Feb
(12) |
Mar
(3) |
Apr
(14) |
May
(14) |
Jun
(18) |
Jul
(5) |
Aug
|
Sep
|
Oct
(27) |
Nov
(15) |
Dec
(5) |
2017 |
Jan
(2) |
Feb
|
Mar
(6) |
Apr
|
May
(1) |
Jun
(1) |
Jul
|
Aug
(1) |
Sep
(3) |
Oct
(4) |
Nov
(1) |
Dec
(8) |
2018 |
Jan
(2) |
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
(1) |
Aug
|
Sep
|
Oct
|
Nov
(3) |
Dec
(2) |
2019 |
Jan
|
Feb
|
Mar
(2) |
Apr
|
May
(1) |
Jun
(1) |
Jul
|
Aug
(3) |
Sep
(1) |
Oct
(2) |
Nov
|
Dec
|
2020 |
Jan
|
Feb
|
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(11) |
Nov
(2) |
Dec
(1) |
2021 |
Jan
(2) |
Feb
(1) |
Mar
|
Apr
|
May
(3) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
2022 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(2) |
From: Peter H. <pet...@sw...> - 2009-02-19 09:54:58
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Tony, The -E format still works - also with snapshot 20081221: ./nfcapd -E Add extension: 2 byte input/output interface index Add extension: 4 byte input/output interface index Add extension: 2 byte src/dst AS number Add extension: 4 byte src/dst AS number File Block Header: NumBlocks = 0 Size = 0 id = 2 Flow Record: Flags = 0x00 size = 44 first = 1235036708 [2009-02-19 10:45:08] last = 1235036727 [2009-02-19 10:45:27] msec_first = 160 msec_last = 253 src addr = x.x.x.x dst addr = z.z.z.z src port = 55115 dst port = 443 fwd status = 0 tcp flags = 0x1b .AP.SF proto = 6 (src)tos = 0 (in)packets = 26 (in)bytes = 12784 There is still some Debug output at the beginning, but all th records are printed. - Peter Tony Gray wrote: > Hi, > Has the output format for nfcapd -E changed between nfdump-1.5.7 and the > current snapshot 1.5.7-20081221? > > The output i am getting from the snapshot version looks like: > > Add extension: 2 byte input/output interface index > Add extension: 4 byte input/output interface index > Add extension: 2 byte src/dst AS number > Add extension: 4 byte src/dst AS number > File Block Header: > NumBlocks = 0 > Size = 0 > id = 2 > File Block Header: > NumBlocks = 1 > Size = 14 > id = 2 > > Where as with the stable version i was getting: > Flow Record: > Flags = 0x00000000 > size = 52 > mark = 0 > srcaddr = X.X.X.X > dstaddr = X.X.X.X > first = 1234522029 [2009-02-13 10:47:09] > last = 1234522029 [2009-02-13 10:47:09] > msec_first = 246 > msec_last = 943 > dir = 1 > tcp_flags = 0x10 .A.... > prot = 6 > tos = 0 > input = 26 > output = 42 > srcas = 0 > dstas = 0 > srcport = 34984 > dstport = 80 > dPkts = 2 > dOctets = 80 > > Thanks, > Tony - -- _______ SWITCH - The Swiss Education and Research Network ______ Peter Haag, Security Engineer, Member of SWITCH CERT PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7 SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland E-mail: pet...@sw... Web: http://www.switch.ch/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBSZ0saf5AbZRALNr/AQLKagQAgwU0Z9voix/W7UrwGPpJ7SI9sT07e6Kn kj8FkOyBwX8jjh3I/QXVX9N8duA98TyIO3JUvS18TncfYvMtofuCSbMjIT3PyY6I WtWH5CACpBqMHnFTDv6moz9bcFO7r+3+A4KFyl0LWU46nYgIoEM3n4vWgKCCQx1I 9ihKN+DEzIo= =mxxE -----END PGP SIGNATURE----- |
From: Tony G. <ton...@he...> - 2009-02-17 13:49:19
|
Hi, Has the output format for nfcapd -E changed between nfdump-1.5.7 and the current snapshot 1.5.7-20081221? The output i am getting from the snapshot version looks like: Add extension: 2 byte input/output interface index Add extension: 4 byte input/output interface index Add extension: 2 byte src/dst AS number Add extension: 4 byte src/dst AS number File Block Header: NumBlocks = 0 Size = 0 id = 2 File Block Header: NumBlocks = 1 Size = 14 id = 2 Where as with the stable version i was getting: Flow Record: Flags = 0x00000000 size = 52 mark = 0 srcaddr = X.X.X.X dstaddr = X.X.X.X first = 1234522029 [2009-02-13 10:47:09] last = 1234522029 [2009-02-13 10:47:09] msec_first = 246 msec_last = 943 dir = 1 tcp_flags = 0x10 .A.... prot = 6 tos = 0 input = 26 output = 42 srcas = 0 dstas = 0 srcport = 34984 dstport = 80 dPkts = 2 dOctets = 80 Thanks, Tony -- Tony Gray HEAnet Limited Network Operations Registered in Ireland, no. 275301 Telephone: +353-1-6609040 1st floor, 5 George's Dock, Fax: +353-1-6603666 I.F.S.C., Dublin 1, Ireland |
From: Dafydd, S. <sid...@uw...> - 2008-11-07 14:23:06
|
Hello, We have just started using Netflow to help us in detecting anomalous traffic. For testing we've configured our core router to export netflow data for one VLAN, which I've been collecting using nfcapd. Along with the IN, OUT and layer 2 traffic for this VLAN, for some reason I am getting netflow data for traffic moving between other VLANs, and am a bit confused as to why as they haven't been configured to export netflow data. I've attached the configuration which we used on the router and was wondering if anybody could shed a light as to why this was happening? Regards, Sion ====================================================================== Sion Dafydd Technical Security Analyst ====================================================================== Tel: 029 2041 6222 Systems and Communications Email: sid...@uw... Information Services Division Web: www.uwic.ac.uk University of Wales Institute, Cardiff ====================================================================== |
From: Drew W. <dre...@th...> - 2008-10-22 17:04:10
|
Hi there. I've been using nfcap to capture data for quite some time, something has been bothering me though for a little while. Mainly whenever I need certain data, I don't seem to be able to quite 'ask the right question' using nfdump. Does anyone have any example commands for common things such as: -Find the top 10 tcp/udp (bandwidth) talkers on the network (meaning local) for the last 5 minutes. -Find the top 10 tcp 25 talkers on the network for the last 5 minutes. Also has anyone been able to use nfdump to locate malicious activity such as botnets or DDoS attacks originating from your own network? I.e. looking for IRC connections or UDP port 80, or high levels or ICMP... Sorry if these are wacky questions, I am trying to avoid getting one of those really pricy 'netflow analyzers' like Orion because I have a feeling that nfdump can do the same thing without the pretty graphs but I am just a little weak on how to mine the data at this time. Thanks, -Drew |
From: Jose M. A. C. <ja...@us...> - 2008-10-22 07:27:37
|
Strange, I have six 6509 reporting netflow with the same configuration. Perhaps you can try to add: mls netflow interface mls flow ip interface-full mls nde sender version 5 The other error I suppose that is derived for the lack of data in this timeslot (needed for detail stats). If you obtain data now must be fixed automatically. El Tuesday 21 October 2008 19:01:54 Jake Zack escribió: > Cisco 6509. > > ip flow-export source <nuked> > ip flow-export version 5 > ip flow-export destination <nuked> 9995 > ip flow-aggregation cache as > ip flow-aggregation cache protocol-port > ip flow-aggregation cache source-prefix > ip flow-aggregation cache destination-prefix > ip flow-aggregation cache prefix > ip flow-aggregation cache prefix-port > > This same configuration was working with 'pfcapd', a nfdump-like > clone used by "Psyche", another NetFlow analyzer. > > So...despite getting that error over and over again consistently, I > do seem to be importing some netflow data right now. I am seeing > flows/packets/bits graphed now. > > I don't see any method for breaking it down into traffic types like > ICMP/TCP/UDP, though. > > Also, when I try to 'process' details, I get: > > ** nfdump -M /usr/local/nfsen/profiles-data/live/rt01-ott -T -r > 2008/10/21/nfcapd.200810211201 -n 10 -s ip/flows > nfdump filter: > any > stat() error '/usr/local/nfsen/profiles-data/live/rt01-ott/2008/10/21/ > nfcapd.200810211201': File not found! > > monitor# ls -al /usr/local/nfsen/profiles-data/live/rt01-ott/ > 2008/10/21/nfcapd.* > -rw-r--r-- 1 www www 1575120 Oct 21 12:35 /usr/local/nfsen/ > profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211230 > -rw-r--r-- 1 www www 9462912 Oct 21 12:40 /usr/local/nfsen/ > profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211235 > -rw-r--r-- 1 www www 13820884 Oct 21 12:45 /usr/local/nfsen/ > profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211240 > -rw-r--r-- 1 www www 14017652 Oct 21 12:50 /usr/local/nfsen/ > profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211245 > -rw-r--r-- 1 www www 14103088 Oct 21 12:55 /usr/local/nfsen/ > profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211250 > -rw-r--r-- 1 www www 14030704 Oct 21 13:00 /usr/local/nfsen/ > profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211255 > > ...so my question on this one is...why is it looking for 200810211201? > > On 21-Oct-08, at 12:52 PM, Jose Manuel Agudo Cuesta wrote: > > Seems that router sends malformed netflow packets. > > > > If you post the brand/model and configuration, I'll try to help. > > > > Best Regards, > > > > Jose Manuel > > > > El Tuesday 21 October 2008 18:15:51 Jake Zack escribió: > >> Installed today: > >> > >> nfdump-1.5.7 > >> -rwxr-xr-x 1 root bin 235308 Oct 21 11:34 /usr/local/bin/nfdump > >> > >> Oct 21 12:10:15 monitor /usr/local/bin/nfcapd[51182]: Error reading > >> netflow header: Unexpected netflow version 2048 > >> Oct 21 12:10:46 monitor last message repeated 879 times > >> Oct 21 12:12:48 monitor last message repeated 3850 times > >> > >> Appears to generate that message for every single flow packet > >> received. > >> > >> On the router I'm specifying netflow version 5. nfdump documentation > >> says it supports versions 5,7,9 transparently, and there's nowhere in > >> the config file I can specify this anyways. > >> > >> What am I missing? > >> > >> Thanks all, > >> > >> > >> --------------------------------------------------------------------- > >> ---- > >> This SF.Net email is sponsored by the Moblin Your Move Developer's > >> challenge Build the coolest Linux based applications with Moblin > >> SDK & win > >> great prizes Grand prize is a trip for two to an Open Source event > >> anywhere > >> in the world http://moblin-contest.org/redirect.php? > >> banner_id=100&url=/ > >> _______________________________________________ > >> Nfdump-discuss mailing list > >> Nfd...@li... > >> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss > > > > <signature.asc><ATT00001.txt><ATT00002.txt> |
From: Jake Z. <jak...@ci...> - 2008-10-21 17:02:12
|
Cisco 6509. ip flow-export source <nuked> ip flow-export version 5 ip flow-export destination <nuked> 9995 ip flow-aggregation cache as ip flow-aggregation cache protocol-port ip flow-aggregation cache source-prefix ip flow-aggregation cache destination-prefix ip flow-aggregation cache prefix ip flow-aggregation cache prefix-port This same configuration was working with 'pfcapd', a nfdump-like clone used by "Psyche", another NetFlow analyzer. So...despite getting that error over and over again consistently, I do seem to be importing some netflow data right now. I am seeing flows/packets/bits graphed now. I don't see any method for breaking it down into traffic types like ICMP/TCP/UDP, though. Also, when I try to 'process' details, I get: ** nfdump -M /usr/local/nfsen/profiles-data/live/rt01-ott -T -r 2008/10/21/nfcapd.200810211201 -n 10 -s ip/flows nfdump filter: any stat() error '/usr/local/nfsen/profiles-data/live/rt01-ott/2008/10/21/ nfcapd.200810211201': File not found! monitor# ls -al /usr/local/nfsen/profiles-data/live/rt01-ott/ 2008/10/21/nfcapd.* -rw-r--r-- 1 www www 1575120 Oct 21 12:35 /usr/local/nfsen/ profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211230 -rw-r--r-- 1 www www 9462912 Oct 21 12:40 /usr/local/nfsen/ profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211235 -rw-r--r-- 1 www www 13820884 Oct 21 12:45 /usr/local/nfsen/ profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211240 -rw-r--r-- 1 www www 14017652 Oct 21 12:50 /usr/local/nfsen/ profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211245 -rw-r--r-- 1 www www 14103088 Oct 21 12:55 /usr/local/nfsen/ profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211250 -rw-r--r-- 1 www www 14030704 Oct 21 13:00 /usr/local/nfsen/ profiles-data/live/rt01-ott/2008/10/21/nfcapd.200810211255 ...so my question on this one is...why is it looking for 200810211201? On 21-Oct-08, at 12:52 PM, Jose Manuel Agudo Cuesta wrote: > Seems that router sends malformed netflow packets. > > If you post the brand/model and configuration, I'll try to help. > > Best Regards, > > Jose Manuel > > El Tuesday 21 October 2008 18:15:51 Jake Zack escribió: >> Installed today: >> >> nfdump-1.5.7 >> -rwxr-xr-x 1 root bin 235308 Oct 21 11:34 /usr/local/bin/nfdump >> >> Oct 21 12:10:15 monitor /usr/local/bin/nfcapd[51182]: Error reading >> netflow header: Unexpected netflow version 2048 >> Oct 21 12:10:46 monitor last message repeated 879 times >> Oct 21 12:12:48 monitor last message repeated 3850 times >> >> Appears to generate that message for every single flow packet >> received. >> >> On the router I'm specifying netflow version 5. nfdump documentation >> says it supports versions 5,7,9 transparently, and there's nowhere in >> the config file I can specify this anyways. >> >> What am I missing? >> >> Thanks all, >> >> >> --------------------------------------------------------------------- >> ---- >> This SF.Net email is sponsored by the Moblin Your Move Developer's >> challenge Build the coolest Linux based applications with Moblin >> SDK & win >> great prizes Grand prize is a trip for two to an Open Source event >> anywhere >> in the world http://moblin-contest.org/redirect.php? >> banner_id=100&url=/ >> _______________________________________________ >> Nfdump-discuss mailing list >> Nfd...@li... >> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss > > > <signature.asc><ATT00001.txt><ATT00002.txt> |
From: Jose M. A. C. <ja...@us...> - 2008-10-21 16:52:57
|
Seems that router sends malformed netflow packets. If you post the brand/model and configuration, I'll try to help. Best Regards, Jose Manuel El Tuesday 21 October 2008 18:15:51 Jake Zack escribió: > Installed today: > > nfdump-1.5.7 > -rwxr-xr-x 1 root bin 235308 Oct 21 11:34 /usr/local/bin/nfdump > > Oct 21 12:10:15 monitor /usr/local/bin/nfcapd[51182]: Error reading > netflow header: Unexpected netflow version 2048 > Oct 21 12:10:46 monitor last message repeated 879 times > Oct 21 12:12:48 monitor last message repeated 3850 times > > Appears to generate that message for every single flow packet received. > > On the router I'm specifying netflow version 5. nfdump documentation > says it supports versions 5,7,9 transparently, and there's nowhere in > the config file I can specify this anyways. > > What am I missing? > > Thanks all, > > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge Build the coolest Linux based applications with Moblin SDK & win > great prizes Grand prize is a trip for two to an Open Source event anywhere > in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Nfdump-discuss mailing list > Nfd...@li... > https://lists.sourceforge.net/lists/listinfo/nfdump-discuss |
From: Jake Z. <jak...@ci...> - 2008-10-21 16:44:33
|
Installed today: nfdump-1.5.7 -rwxr-xr-x 1 root bin 235308 Oct 21 11:34 /usr/local/bin/nfdump Oct 21 12:10:15 monitor /usr/local/bin/nfcapd[51182]: Error reading netflow header: Unexpected netflow version 2048 Oct 21 12:10:46 monitor last message repeated 879 times Oct 21 12:12:48 monitor last message repeated 3850 times Appears to generate that message for every single flow packet received. On the router I'm specifying netflow version 5. nfdump documentation says it supports versions 5,7,9 transparently, and there's nowhere in the config file I can specify this anyways. What am I missing? Thanks all, |
From: Giorgos D. P. <gp...@cc...> - 2008-09-24 13:24:09
|
Hello to all! I am experiencing a strange behavior: I suspected that something weird was happening in the way that SYN flags are recorded into the exports. So I did the following experiment: I telneted to a non-existing IP: 100.100.100.101 from a host A. I captured the connection initial SYN packet with tcpdump. I replayed it over and over, to see what would appear in the nfdump export. We are talking about a unidirected flow of SYN packets towards 100.100.100.101. So, I searched the raw nfdump output for this flow record, and I found it, but no flags were set! Has anybody else seen this behavior? Is it something expected that eludes me? Thanks! Giorgos Pallas |
From: Nino C. <nin...@ga...> - 2008-08-20 14:20:17
|
Hi all, Have anyone used NetflowV9 in JunOS? I'm working on Juniper routers with NetflowV5 now, but I would work on NetflowV9. Seems that JunOS support three static templates only. In this templates same new mpls fields are added but same old NetflowV5 fields (like SRC and DST AS) are missing... Anyone know more details? Tank you Nino Ciurleo |
From: Peter H. <pet...@sw...> - 2008-08-15 12:53:33
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, I just uploaded another branch of nfdump at Sourceforge: nfdump-1.5.7-nsel This nfdump release contains a 3rd party extension to process flows Cisco's ASA (adaptive security appliance) box flow data. The code was kindly provided by CISCO. The contributed code will be integrated into the standard code tree of nfdump. nfdump-1.5.7-nsel can be used together with NfSen 1.3. A dedicated NfSen plugin NSELTracker is provided with the NfSen project at Sourceforge shortly. Many thanks to CISCO for the code contribution. - Peter - -- _______ SWITCH - The Swiss Education and Research Network ______ Peter Haag, Security Engineer, Member of SWITCH CERT PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7 SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland E-mail: pet...@sw... Web: http://www.switch.ch/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBSKV8T/5AbZRALNr/AQIuuwP+Jvb9KZ+4C3pHGTC3DwPq73672U6FK8NN kONj+FY7oNIEmWa3RklXetaqT82zlaKKBc5OvO9Rk8xVaEjfGyb45i9V28HdoFZx aHDm5edQ1XWaOYh4XUrERg/vAEhugNzv37CeGISwIclBLZ4g54LoTMxUpP84zw+x diGaVcYUaTg= =zfN6 -----END PGP SIGNATURE----- |
From: Peter H. <pet...@sw...> - 2008-08-15 12:37:55
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, I just uploaded a branch of nfdump: nfdump-1.5.7-packeteer This stable release contains a 3rd party extension to process flows for Packeteer PacketShaper flow data. The code was kindly provided by Steven Gianvecchio inspired from an idea by Clarke Morledge from the College of William & Mary Williamsburg. The contributed code will be integrated into the standard code tree of nfdump. nfdump-packeteer is a drop-in replacement and can be used together with NfSen. Many thanks to the contributers. - Peter - -- _______ SWITCH - The Swiss Education and Research Network ______ Peter Haag, Security Engineer, Member of SWITCH CERT PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7 SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland E-mail: pet...@sw... Web: http://www.switch.ch/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBSKV4pP5AbZRALNr/AQIOzgP9Hf8MDU23Zd0TB8k6/rV/PrKK+OFWdXYG xnauPkDo2DivQeME11OlYxXkpjsVMC6tYSqmvH829zpE8xoNnAAFVN5pZTVVaR9S F/z1kJ7LZmq7V+q6bcfBNMIZidcXdniRjGqhQqN3fufd8Lv6erRI1laAN1u+cunD gnjv8N6E/JU= =raqD -----END PGP SIGNATURE----- |
From: Peter H. <pet...@sw...> - 2008-08-15 12:09:17
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi all, I just uploaded a next nfdump snapshot. Feedback is appreciated. Even if the changes list does not look so impressive, is was a major change for adding more v9 tags and other feature. ( see below ) The next snapshot will include sampling. So stay tuned. And - no - there is not yet a final date for stable nfdump-1.6 :) - Peter Note: ** THIS IS A DEVELOPER SNAPSHOT ** ** NOT TO BE USED IN PODUCTION ** Date: Aug, 15th 2008 nfdump-snapshot-1.5.7-20080815 is a development snapshot. It's not recommended to be used for production environments. NEW in snapshot: - ---------------- - - Added option for multiple netflow stream to same port. ~ -n <Ident,IP,base_directory> ~ Example: -n router1,192.168.100.1,/var/nfdump/router1 ~ So multiple -n options may be given at the command line ~ Old style syntax still works for compatibility, ( -I .. -l ... ) ~ but then only one source is supported. - - Added more v9 tags for netflow v9. ~ The detailed tags are listed in nfcapd(1) ~ Adding new tags also extended the binary file format with ~ data block format 2, which is extension based. File format ~ for version <= 1.5.* ( Data block format 1 ) is read ~ transparently. Data block 2 are skipped by nfdump 1.5.7. - - All new tags can be selected in -o fmt:... see nfdump(1) - - topN stat for all new tags is implemented - - Add flexible storage option for nfcapd. To save disk space, the ~ data extensions to be stored in the data file are user selectable. - - Switch scaling factor ( k, M, G ) from 1024 to 1000. - - Make nfdump fully 64bit compliant. ( 8bit data alignments and access ) - - Fix a few bugs Notes: - - The tools ft2nfdump and all sflow tools are not yet updated for the new tags. - - The flow record statistics ( -s record ) does not yet show the additional tags. ** THIS RELEASE IS NOT INTENDED FOR PRODUCTION ** This release works with NfSen 1.3, however, the interface is not yet able to profit from the new options. - Peter - -- _______ SWITCH - The Swiss Education and Research Network ______ Peter Haag, Security Engineer, Member of SWITCH CERT PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7 SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland E-mail: pet...@sw... Web: http://www.switch.ch/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBSKVx7/5AbZRALNr/AQLqHwQAkZRf7op2GqBVwm640p9hh4/J25wbJdi6 rmN+nN/TsUeQa4Bd7hKwCBzY/MTyirvturjoUbP+i/MgXLk2NAM0KIkfNTIL+DDB qUfJf2qhvEGnhPta3JywSoOH5RI9LZJeIwKHdzb+aPFPPplbEviFUKZ+B4GykhLe GFo4vlqP45U= =6EIU -----END PGP SIGNATURE----- |
From: Peter H. <pet...@sw...> - 2008-08-05 09:17:05
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Pavel Celeda wrote: | Hi Peter, | | thank you for the explanation. The current situation is fully acceptable. | If you need some beta testers for pre-release testing of new snapshot let | me known :) Yes - indeed I will need lab rats, as the next snapshot has lot's of more v9 extensions - so stay tuned! It will be ready soon. - Peter | | best regards | Pavel | | * Peter Haag (pet...@sw...) wrote: |> Hi Pavel, |> |> Yes - this is a known issue. NfSen switched already to the 1000 scaling |> factors - nfdump up to and including 1.5.7 did |> not yet. There will be no changes to 1.5.7 any more. The man pages also |> mention the 1024 scaling, therefore the users |> should know, how to deal with. |> |> NfSen reports correctly all numbers in scalings of 1000. You will have a |> difference in nfdump outputs, which I guess is |> acceptable so far. |> |> The new developer snapshot of nfdump has corrected this already and also |> uses scaling factors of 1000. |> This snapshot is being tested at the moment before it gets released as a |> next next mile stone towards nfdump-1.6 |> |> - Peter | | ------------------------------------------------------------------------- | This SF.Net email is sponsored by the Moblin Your Move Developer's challenge | Build the coolest Linux based applications with Moblin SDK & win great prizes | Grand prize is a trip for two to an Open Source event anywhere in the world | http://moblin-contest.org/redirect.php?banner_id=100&url=/ | _______________________________________________ | Nfdump-discuss mailing list | Nfd...@li... | https://lists.sourceforge.net/lists/listinfo/nfdump-discuss - -- _______ SWITCH - The Swiss Education and Research Network ______ Peter Haag, Security Engineer, Member of SWITCH CERT PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7 SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland E-mail: pet...@sw... Web: http://www.switch.ch/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBSJgalf5AbZRALNr/AQKKiQP/Xctkumas9hUfkEfLFa3bhQjWxsEBv+TX lmn5qcPcfVwknGKN9J+JaDF6FQogWuz13Sx9GMMBU+NeYsu4NddcpJwssOKBJyJH 4FfizEuJ5PacSb0Y4OEFRKbIoNSdsBkSMnJ7ePDxYviCJbP+RiJP2DYQf0Ah4+Am ZhIYnXI/SoQ= =KTdb -----END PGP SIGNATURE----- |
From: Pavel C. <ce...@li...> - 2008-08-05 08:11:04
|
Hi Peter, thank you for the explanation. The current situation is fully acceptable. If you need some beta testers for pre-release testing of new snapshot let me known :) best regards Pavel * Peter Haag (pet...@sw...) wrote: > Hi Pavel, > > Yes - this is a known issue. NfSen switched already to the 1000 scaling > factors - nfdump up to and including 1.5.7 did > not yet. There will be no changes to 1.5.7 any more. The man pages also > mention the 1024 scaling, therefore the users > should know, how to deal with. > > NfSen reports correctly all numbers in scalings of 1000. You will have a > difference in nfdump outputs, which I guess is > acceptable so far. > > The new developer snapshot of nfdump has corrected this already and also > uses scaling factors of 1000. > This snapshot is being tested at the moment before it gets released as a > next next mile stone towards nfdump-1.6 > > - Peter |
From: Peter H. <pet...@sw...> - 2008-08-05 07:55:20
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Pavel, Yes - this is a known issue. NfSen switched already to the 1000 scaling factors - nfdump up to and including 1.5.7 did not yet. There will be no changes to 1.5.7 any more. The man pages also mention the 1024 scaling, therefore the users should know, how to deal with. NfSen reports correctly all numbers in scalings of 1000. You will have a difference in nfdump outputs, which I guess is acceptable so far. The new developer snapshot of nfdump has corrected this already and also uses scaling factors of 1000. This snapshot is being tested at the moment before it gets released as a next next mile stone towards nfdump-1.6 - Peter Pavel Celeda wrote: | Hi Peter, | | we have discovered issue with MiB, MB, ... prefixes in NFDUMP similar to | NfSen one a I have reported last year. NFDUMP reports the total bytes and | probably other values in MiB e.g. | | 90949535 ... nfcapd database entry | 90949535 / 1024 / 1024 = 86,7 MiPackets (reported by NFDUMP) | 90949535 / 1000 / 1000 = 90,9 MPackets (reported by NfSen) | | Detailed bug description is included in attached file. | | best regards | Pavel | | | ------------------------------------------------------------------------ | | ------------------------------------------------------------------------- | This SF.Net email is sponsored by the Moblin Your Move Developer's challenge | Build the coolest Linux based applications with Moblin SDK & win great prizes | Grand prize is a trip for two to an Open Source event anywhere in the world | http://moblin-contest.org/redirect.php?banner_id=100&url=/ | | | ------------------------------------------------------------------------ | | _______________________________________________ | Nfdump-discuss mailing list | Nfd...@li... | https://lists.sourceforge.net/lists/listinfo/nfdump-discuss - -- _______ SWITCH - The Swiss Education and Research Network ______ Peter Haag, Security Engineer, Member of SWITCH CERT PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7 SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland E-mail: pet...@sw... Web: http://www.switch.ch/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBSJgHZ/5AbZRALNr/AQLPDQP/TaKTsrkeLjH0f0R30quu/ZYpcWYa+YxP WFDUBuWk4btwY/NdoipLnk93Er+MwW2Bt/YaR/0N+3+OrguXQtjo/bPYQektWZJw VEHdWx6uxrHktYkCYYiW1toNCnmOIhOL4gUII3BTN7Wdd7EtIvgzYL0PXW2ECeKS yUh1Ukwxkl0= =vg8y -----END PGP SIGNATURE----- |
From: Pavel C. <ce...@li...> - 2008-08-05 07:14:14
|
Hi Peter, we have discovered issue with MiB, MB, ... prefixes in NFDUMP similar to NfSen one a I have reported last year. NFDUMP reports the total bytes and probably other values in MiB e.g. 90949535 ... nfcapd database entry 90949535 / 1024 / 1024 = 86,7 MiPackets (reported by NFDUMP) 90949535 / 1000 / 1000 = 90,9 MPackets (reported by NfSen) Detailed bug description is included in attached file. best regards Pavel |
From: Mike M. <one...@gm...> - 2008-08-02 01:47:16
|
Can anyone help me with what I am doing wrong, I have netflow working for inbound traffic on a cisco 2811, using nfdump and nfsen. I have enabled egress flow on the cisco router, but I am still only seeing inbound. Assuming I need to export something else, or tell nfdump / nfsen that I have 2 flows? Help Please? Thanks Mike Montgomery Network Administrator City of Scottsburg Citizens Communications |
From: Mike M. <mmo...@c3...> - 2008-08-02 01:46:31
|
Can anyone help me with what I am doing wrong, I have netflow working for inbound traffic on a cisco 2811, using nfdump and nfsen. I have enabled egress flow on the cisco router, but I am still only seeing inbound. Assuming I need to export something else, or tell nfdump / nfsen that I have 2 flows? Help Please? Thanks Mike Montgomery Network Administrator City of Scottsburg Citizens Communications |
From: Peter H. <pet...@sw...> - 2008-07-18 09:15:56
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 You may change that to any number of your coice in nfdump.c ~ if ((aggregate || flow_stat) && ( topN > 1000 || topN == 0) ) { ~ printf("TopN for record statistic: 0 < topN < 1000 only allowed for IP statistics\n"); ~ exit(255); ~ } Please note: the limit is there, as this stat becomes very slow for large n, and is therefor not recommended. - Peter Nino Ciurleo wrote: | Can anyone help me to change 1000 limit in aggregate query to an upper | number? | Tanks | Nino Ciurleo | | ------------------------------------------------------------------------- | This SF.Net email is sponsored by the Moblin Your Move Developer's challenge | Build the coolest Linux based applications with Moblin SDK & win great prizes | Grand prize is a trip for two to an Open Source event anywhere in the world | http://moblin-contest.org/redirect.php?banner_id=100&url=/ | _______________________________________________ | Nfdump-discuss mailing list | Nfd...@li... | https://lists.sourceforge.net/lists/listinfo/nfdump-discuss - -- _______ SWITCH - The Swiss Education and Research Network ______ Peter Haag, Security Engineer, Member of SWITCH CERT PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7 SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland E-mail: pet...@sw... Web: http://www.switch.ch/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBSIBfTv5AbZRALNr/AQKMPQP/XfyuMwlUJmceUdXvySTB1bvLeY6GKPcS Y9oSYt4dM1aYkQIwyQ9jL/jGRnxU8tAiN1FbbbPJoKQQwakcMoY5w3ut26VN0Qfd /Pfft/7pb1fWQs75sdizWkccvcPEEHCwlriG9aXoEn/pyZ/AhzFFD44eHvaJ/cNE 51I1paO+tcE= =osYf -----END PGP SIGNATURE----- |
From: Nino C. <nin...@ga...> - 2008-07-18 08:54:09
|
Can anyone help me to change 1000 limit in aggregate query to an upper number? Tanks Nino Ciurleo |
From: Bjoern W. <bjo...@rz...> - 2008-07-10 21:04:04
|
> data and need the time format to be HH:MM:SS Hey there, I have helped myself with regex replacement... It's fine now... -regards, bjoern |
From: Bjoern W. <bjo...@rz...> - 2008-07-10 16:03:38
|
Hey guys I need nfdump to print its time format when using %ts or %te (First seen, Last seen) without milliseconds, i.e. it should only print 2005-08-30 06:53:53 instead of 2005-08-30 06:53:53.370 Any suggestions where and how to implement that in the code for a quick and dirty recompilation? -best regards, bjoern PS: For anyone wondering why: I am using data mining techniques on this data and need the time format to be HH:MM:SS |
From: Peter H. <pet...@sw...> - 2008-07-05 09:50:03
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Daan, Daan van der Sanden wrote: | Hi, | | In both netflow_v5_v7.c and netflow_v9.c is an error in handling an overflow | of the SysUptime counter. However I can't test this it at the moment, because | I don't have any packet in which there is an overflow of SysUptime, but going | through the c-code I think it is wrong (unless I'm missing something | obvious). | | This is the part where the header gets read: | v5_header->SysUptime = ntohl(v5_header->SysUptime); | v5_header->unix_secs = ntohl(v5_header->unix_secs); | v5_header->unix_nsecs = ntohl(v5_header->unix_nsecs); | | /* calculate boot time in msec */ | boot_time = ((uint64_t)(v5_header->unix_secs)*1000 + | ((uint64_t(v5_header->unix_nsecs) / 1000000) ) - | (uint64_t)(v5_header->SysUptime); | | And here where the overflow correction takes place, when a flow record is | processed: | // Time issues | First = ntohl(v5_record->First); | Last = ntohl(v5_record->Last); | if ( First > Last ) | /* Last in msec, in case of msec overflow, between start and end */ | end_time = 0x100000000LL + Last + boot_time; | else | end_time = (uint64_t)Last + boot_time; | | /* start time in msecs */ | start_time = (uint64_t)First + boot_time; | | This is going wrong, because when SysUptime overflows Last is indead smaller | than First, but so will be SysUptime in the NetFlow header. So the unix | timestamp in the header is matched to the SysUptime value in the header. So | the end-time was allready correct and the start-time should be corrceted. | This way the flow will be exported as it were 50 days in the future. Indeed - you re right! The start-time needs to be corrected, instead of the end-time. I changed to codeto reflect this: if ( First > Last ) /* First in msec, in case of msec overflow, between start and end */ start_time = boot_time - 0x100000000LL + (uint64_t)First; else start_time = (uint64_t)First + boot_time; /* end time in msecs */ end_time = (uint64_t)Last + boot_time; Although a bug it does not harm much in current versions as it will affect only flows within a time range of +flow timeout around the SysUptime overflow, which is typically a few seconds up to a few minutes. | | Another option when the correction goes wrong is for example if both First and | Last are just before the overflow value (2^32), but the value of SysUptime in | the NetFlow header is overflown we again get the wrong value calculated. | Since now both the start_time and the end_time need to be corrected. True - in fact, this would mean, the overflow occurred after the flow ended but did not get exported so far. In case of this overflow, First and Last are > Sysuptime in absolute value. This could be corrected with the code snipped below, inserted after the the first correction: // if overflow happened after flow ended but before got exported if ( Last > v5_header->SysUptime ) { start_time -= 0x100000000LL; end_time -= 0x100000000LL; } | | I'm assuming that the SysUptime in the NetFlow header is always after Last (in | time, not neccesary in value). If I understood the netflow documentation | correctly this is always the case, since sysuptime header is the value of the | SysUptime counter at the moment the packet is sent. | | I hope it is clear, but I strongly belief that the overflow is not correctly | implemented. But please correct me if I'm wrong. Many thanks for this input. It's nice to see people digging into the details of the code! Any feedback is welcome to improve nfdump. - Peter | | Daan | | ------------------------------------------------------------------------- | Check out the new SourceForge.net Marketplace. | It's the best place to buy or sell services for | just about anything Open Source. | http://sourceforge.net/services/buy/index.php | _______________________________________________ | Nfdump-discuss mailing list | Nfd...@li... | https://lists.sourceforge.net/lists/listinfo/nfdump-discuss -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBSG9Dzv5AbZRALNr/AQL86AQAkY4WQXzSvDZ0lW3/ysDAMP2buH+fqKpi DHA/9dMJXQSy7x4TSb/hhEPoQuDT5qpWHsDER14Qldjq06Ldf9EGwri5kR9ILhgy KJtB22M0i8WZAJN4d17RD+tYPjeSbFteJQbM3kiwsmsUsZTaUFbXp9KkiPiNcYf6 ID01pM3OQ/g= =vl52 -----END PGP SIGNATURE----- |
From: Peter H. <pet...@sw...> - 2008-07-05 06:15:52
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bjoern Weiland wrote: | Nino Ciurleo wrote: |> try with: |> nfdump -r /and/dir/nfcapd.200407110845 -c 100 "proto tcp and ( src ip |> 172.16.17.18 or dst ip 172.16.17.19 )" | | That did it, thanks guys, simplest thing, actually. It works with single | quotes as well. Not only did I copy the command, so I indeed used | backticks, but the "proto" keyword was missing in the manpage example Indeed - it's a bug. I'll correct that. Sorry for the trouble. - Peter | | -regards, bjoern | | ------------------------------------------------------------------------- | Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! | Studies have shown that voting for your favorite open source project, | along with a healthy diet, reduces your potential for chronic lameness | and boredom. Vote Now at http://www.sourceforge.net/community/cca08 | _______________________________________________ | Nfdump-discuss mailing list | Nfd...@li... | https://lists.sourceforge.net/lists/listinfo/nfdump-discuss -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iQCVAwUBSG8RmP5AbZRALNr/AQKPnQP/V867nQgIs/SutUYS0P/id7178Yelp50S TCGgxnzurfRcZMDXEE0UAHQii2OQqfjAVlrhCmxBsJ+oS7MbSXGLKa/j5zGOaWQV n96oOtY1Qst8FNNr0CxrN1E7Uw014QXL5Zi41emNn1x6LARAyN5xWUEz30oAWDE7 YlF4om7TZRQ= =prUn -----END PGP SIGNATURE----- |