Re: [Nfdump-discuss] Reported flow counts after performing aggregation

[Karen M. <kpm...@gm...>]

Peter,



Sure, as an example I start with an nfcapd file nfcapd.201308142315 where
'nfdump -I -r nfcapd.201308142315' reports:


Flows: 70215

Packets: 1973930

Bytes: 1619354988

…


and 'nfdump -s srcip/flows -N -r nfcapd.201308142315':


Top 10 Src IP Addr ordered by flows:

Date first seen          Duration Proto       Src IP Addr    Flows(%)
Packets(%)       Bytes(%)         pps      bps   bpp

2013-08-14 16:10:25.417   512.418 any      172.16.128.169
11423(16.3)    14735(
0.7)  2572414( 0.2)       28    40161   174

2013-08-14 16:11:53.634   424.215 any      172.16.128.202     3716( 5.3)
4076( 0.2)   287191( 0.0)        9     5415    70

2013-08-14 16:13:45.503   312.309 any      172.16.128.203     2506( 3.6)
2597( 0.1)   182894( 0.0)        8     4684    70

…

Summary: total flows: 70215, total bytes: 1619354988, total packets:
1973930, avg bps: 21717906, avg pps: 3309, avg bpp: 820


Then I perform aggregation on the file and drop src and dest ports with
'nfdump -A "srcip,dstip,proto" -z -r nfcapd.201308142315 -w
aggr.201308142315'. Now 'nfdump -I -r aggr.201308142315' reports the same #
bytes and # packets but the # flows is the number of records in the file
after aggregation. I was hoping it would report the sum of the aggregated
flows, which should be the same as the original file (70215):


Flows: 13301

Packets: 1973930

Bytes: 1619354988

On the aggregated file 'nfdump -s srcip/flows -N -r aggr.201308142315'
reports a different set of top N because aggregated flows aren't taken into
account. In this example the top src ip has fallen into 5th place and the #
flows reported aren't aggregated flows:


Top 10 Src IP Addr ordered by flows:

Date first seen          Duration Proto       Src IP Addr    Flows(%)
Packets(%)       Bytes(%)         pps      bps   bpp

2013-08-14 16:11:38.755   443.184 any       172.16.125.79      403(
3.0)    22234(
1.1)  4436457( 0.3)       50    80083   199

2013-08-14 16:09:58.030   567.297 any       172.16.124.59      389(
2.9)    12701(
0.6)  3373682( 0.2)       22    47575   265

2013-08-14 16:10:12.285   576.477 any      172.16.124.146      332(
2.5)    11113(
0.6)  2600811( 0.2)       19    36092   234

2013-08-14 16:10:44.945   535.019 any      172.16.124.118      259( 1.9)
4874( 0.2)   827532( 0.1)        9    12373   169

2013-08-14 16:10:25.417   512.418 any      172.16.128.169      233(
1.8)    14735(
0.7)  2572414( 0.2)       28    40161   174

…

Summary: total flows: 13301, total bytes: 1619354988, total packets:
1973930, avg bps: 21717906, avg pps: 3309, avg bpp: 820


Hope that helps clarify my question. Note that I'm using nfdump 1.6.5 but
with your 2013-03-02 v1.6.9 fix (Fix bug statistics update with aggreagted
flow records).

Karen