[Nfdump-discuss] Fortinet netflow: values 69.000 times too large

[Brian C. <b.c...@po...>]

Using nfcapd/nfdump with Fortigate FGT1000D firewall (FortiOS v5.6.0), I 
get a very strange problem: the displayed packet counts and byte counts 
are exactly 69 times larger than they actually are (and as displayed by 
tshark)

Here's a specific example. The Fortigate is configured as per 
http://kb.fortinet.com/kb/documentLink.do?externalID=FD36460 like this:

config system netflow
     set collector-ip x.x.x.3
     set collector-port 9002
     set active-flow-timeout 10
     set template-tx-timeout 10
end

I captured the netflow packets using:
tshark -s 0 -i ens192 -w netflow-fortinet.pcap 'udp port 9002'

and decoded them using:
tshark -r netflow-fortinet.pcap -nnV -d udp.port==9002,cflow | less

Then I downloaded a test file:
wget 
https://nsrc.org/workshops/2017/caren-cndo/networking/cndo/en/presentations/Cabling_Installation_Hints.pdf

This test file is ~25MB (25058321 bytes)

Looking at the flows from nfdump:

$ nfdump -M /var/nfsen/profiles-data/live/firewall  -T  -r 
2017/06/15/nfcapd.201706151110 -c 20 -N 'host xx.xx.xx.132 and host 
128.223.157.25'
Date first seen          Duration Proto      Src IP Addr:Port          
Dst IP Addr:Port   Packets    Bytes Flows
2017-06-15 11:11:01.870   103.100 6 128.223.157.25:443   ->     
xx.xx.xx.132:58295  1196805 1794647289     1
2017-06-15 11:11:01.870   103.100 6 xx.xx.xx.132:58295 ->   
128.223.157.25:443     860913 49341003     1
Summary: total flows: 2, total bytes: 1843988292, total packets: 
2057718, avg bps: 143083475, avg pps: 19958, avg bpp: 896
Time window: 2017-06-15 09:46:27 - 2017-06-15 11:14:59
Total flows processed: 36699, Blocks skipped: 0, Bytes read: 2202532
Sys: 0.012s flows/second: 3058250.0  Wall: 0.013s flows/second: 2789949.8

Note the extremely large values for packets and bytes (1.79GB)

Looking at the captured packets decoded with tshark:

Cisco NetFlow/IPFIX
     Version: 9
     Count: 16
     SysUptime: 993931.010000000 seconds
     Timestamp: Jun 15, 2017 11:12:46.000000000 GMT
         CurrentSecs: 1497525166
     FlowSequence: 202736
     SourceId: 1
     FlowSet 1 [id=258] (1 flows)
         FlowSet Id: (Data) (258)
         FlowSet Length: 68
         [Template Frame: 16]
         Flow 1
*            Octets: 26009381**
*            Post Octets: 26009381
*            Packets: 17345**
*            Post Packets: 17345
             [Duration: 103.100000000 seconds (switched)]
                 StartTime: 993826.880000000 seconds
                 EndTime: 993929.980000000 seconds
             SrcPort: 443
             DstPort: 58295
             InputInt: 8
             OutputInt: 6
             Protocol: TCP (6)
*            [Expert Info (Warning/Malformed): Trying to fetch an 
unsigned integer with length 9]**
**                [Trying to fetch an unsigned integer with length 9]**
**                [Severity level: Warning]**
**                [Group: Malformed]**
*            ApplicationID: NBAR Application ID: 20:48 (type:id)
             Unknown Field Type: Type 65: Value (hex bytes): 0c 0c
             Forwarding Status
                 01.. .... = ForwardingStatus: Forward (1)
                 ..00 0000 = ForwardingStatusForwardCode: Forwarded 
(Unknown) (0)
             Flow End Reason: End of Flow detected (3)
             SrcAddr: 128.223.157.25
             DstAddr: xx.xx.xx.132
         Padding: 0000
     FlowSet 2 [id=258] (1 flows)
         FlowSet Id: (Data) (258)
         FlowSet Length: 68
         [Template Frame: 16]
         Flow 1
             Octets: 715087
             Post Octets: 715087
             Packets: 12477
             Post Packets: 12477
             [Duration: 103.100000000 seconds (switched)]
                 StartTime: 993826.880000000 seconds
                 EndTime: 993929.980000000 seconds
             SrcPort: 58295
             DstPort: 443
             InputInt: 6
             OutputInt: 8
             Protocol: TCP (6)
             [Expert Info (Warning/Malformed): Trying to fetch an 
unsigned integer with length 9]
                 [Trying to fetch an unsigned integer with length 9]
                 [Severity level: Warning]
                 [Group: Malformed]
             ApplicationID: NBAR Application ID: 20:48 (type:id)
             Unknown Field Type: Type 65: Value (hex bytes): 0c 0c
             Forwarding Status
                 01.. .... = ForwardingStatus: Forward (1)
                 ..00 0000 = ForwardingStatusForwardCode: Forwarded 
(Unknown) (0)
             Flow End Reason: End of Flow detected (3)
             SrcAddr: xx.xx.xx.132
             DstAddr: 128.223.157.25
         Padding: 0000

Those values are sensible. Now look at the ratio:

$ bc
scale=10
1196805/17345
*69.0000000000**
*1794647289/26009381
*69.0000000000*

I suspect the "Expert Info" warning from tshark is relevant: it had to 
process an unsigned integer of length 9.  If the value were out by a 
factor of 256 that would make sense; but 69 (0x45) seems most bizarre!

Any thoughts?

Regards,

Brian.