Thread: [Nfdump-discuss] vSphere 5.1 distributed switch to nfcapd with IPFIX
netflow collecting and processing tools
Brought to you by:
phaag
Menu
▾
▴
nfdump-discuss
|
From: David W. <da...@on...> - 2013-05-07 01:10:05
|
Hi,
I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5)
I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter.
I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in "received at".
NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time.
I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they become available to fix up this timestamp issue.
# nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 2
size = 72
first = 0 [1970-01-01 10:00:00]
last = 0 [1970-01-01 10:00:00]
msec_first = 0
msec_last = 0
src addr = 110.175.94.222
dst addr = 192.168.64.6
src port = 58464
dst port = 443
fwd status = 157
tcp flags = 0x00 ......
proto = 6
(src)tos = 0
(in)packets = 9
(in)bytes = 1500
input = 1678
output = 1799
ip router = 10.1.4.39
received at = 1367541600163 [2013-05-03 10:40:00.163]
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 2
size = 72
first = 0 [1970-01-01 10:00:00]
last = 0 [1970-01-01 10:00:00]
msec_first = 0
msec_last = 0
src addr = 101.163.67.76
dst addr = 192.168.64.6
src port = 2735
dst port = 443
fwd status = 255
tcp flags = 0x00 ......
proto = 6
(src)tos = 0
(in)packets = 1
(in)bytes = 40
input = 1678
output = 1799
ip router = 10.1.4.39
received at = 1367541600163 [2013-05-03 10:40:00.163]
Kind Regards,
David
|
|
From: Peter H. <ph...@us...> - 2013-05-07 07:08:43
|
Hmm .. obviously your VDS's don't send the date, as other IPFIX exporters do. I would need a packet dump of your VDS's traffic to the nfcapd collector. If you can send me the data off list, I'll have a look. - Peter On 7/5/13 2:44 AM, David Walsh wrote: > Hi, > I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5) > > I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter. > > I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in "received at". > > NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time. > > I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they become available to fix up this timestamp issue. > > > > # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw > > > Flow Record: > Flags = 0x06 FLOW, Unsampled > export sysid = 2 > size = 72 > first = 0 [1970-01-01 10:00:00] > last = 0 [1970-01-01 10:00:00] > msec_first = 0 > msec_last = 0 > src addr = 110.175.94.222 > dst addr = 192.168.64.6 > src port = 58464 > dst port = 443 > fwd status = 157 > tcp flags = 0x00 ...... > proto = 6 > (src)tos = 0 > (in)packets = 9 > (in)bytes = 1500 > input = 1678 > output = 1799 > ip router = 10.1.4.39 > received at = 1367541600163 [2013-05-03 10:40:00.163] > > > Flow Record: > Flags = 0x06 FLOW, Unsampled > export sysid = 2 > size = 72 > first = 0 [1970-01-01 10:00:00] > last = 0 [1970-01-01 10:00:00] > msec_first = 0 > msec_last = 0 > src addr = 101.163.67.76 > dst addr = 192.168.64.6 > src port = 2735 > dst port = 443 > fwd status = 255 > tcp flags = 0x00 ...... > proto = 6 > (src)tos = 0 > (in)packets = 1 > (in)bytes = 40 > input = 1678 > output = 1799 > ip router = 10.1.4.39 > received at = 1367541600163 [2013-05-03 10:40:00.163] > > Kind Regards, > David > ------------------------------------------------------------------------------ > Learn Graph Databases - Download FREE O'Reilly Book > "Graph Databases" is the definitive new guide to graph databases and > their applications. This 200-page book is written by three acclaimed > leaders in the field. The early access version is available now. > Download your free book today! http://p.sf.net/sfu/neotech_d2d_may > _______________________________________________ > Nfdump-discuss mailing list > Nfd...@li... > https://lists.sourceforge.net/lists/listinfo/nfdump-discuss > -- Be nice to your netflow data. Use NfSen and nfdump :) |
|
From: David W. <da...@on...> - 2013-05-14 00:59:18
|
FYI
I have finally got VMware looking at this for me. I'll reply to the list when I get more information. I am providing them with the logs of my vDS.
Cheers,
David
On 07/05/2013, at 10:44 AM, David Walsh <da...@on...> wrote:
> Hi,
> I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5)
>
> I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter.
>
> I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in "received at".
>
> NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time.
>
> I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they become available to fix up this timestamp issue.
>
>
>
> # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw
>
>
> Flow Record:
> Flags = 0x06 FLOW, Unsampled
> export sysid = 2
> size = 72
> first = 0 [1970-01-01 10:00:00]
> last = 0 [1970-01-01 10:00:00]
> msec_first = 0
> msec_last = 0
> src addr = 110.175.94.222
> dst addr = 192.168.64.6
> src port = 58464
> dst port = 443
> fwd status = 157
> tcp flags = 0x00 ......
> proto = 6
> (src)tos = 0
> (in)packets = 9
> (in)bytes = 1500
> input = 1678
> output = 1799
> ip router = 10.1.4.39
> received at = 1367541600163 [2013-05-03 10:40:00.163]
>
>
> Flow Record:
> Flags = 0x06 FLOW, Unsampled
> export sysid = 2
> size = 72
> first = 0 [1970-01-01 10:00:00]
> last = 0 [1970-01-01 10:00:00]
> msec_first = 0
> msec_last = 0
> src addr = 101.163.67.76
> dst addr = 192.168.64.6
> src port = 2735
> dst port = 443
> fwd status = 255
> tcp flags = 0x00 ......
> proto = 6
> (src)tos = 0
> (in)packets = 1
> (in)bytes = 40
> input = 1678
> output = 1799
> ip router = 10.1.4.39
> received at = 1367541600163 [2013-05-03 10:40:00.163]
>
> Kind Regards,
> David
> ------------------------------------------------------------------------------
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and
> their applications. This 200-page book is written by three acclaimed
> leaders in the field. The early access version is available now.
> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
> _______________________________________________
> Nfdump-discuss mailing list
> Nfd...@li...
> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
|
|
From: David W. <da...@on...> - 2013-11-08 03:39:43
|
Hi,
Here is an update on this issue…..
From VMware:
"This is with regards to the vDS issue. Just to keep you updated that engineering have isolated the code that seems to be causing the issue. We are working on a fix and I will share further updates as and when the same is available”
Hopefully it will be part of Update 2 of v5.1. I have not tested 5.5 yet.
On 14 May 2013, at 10:59 am, David Walsh <da...@on...> wrote:
> FYI
>
> I have finally got VMware looking at this for me. I'll reply to the list when I get more information. I am providing them with the logs of my vDS.
>
> Cheers,
> David
>
> On 07/05/2013, at 10:44 AM, David Walsh <da...@on...> wrote:
>
>> Hi,
>> I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5)
>>
>> I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter.
>>
>> I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in "received at".
>>
>> NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time.
>>
>> I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they become available to fix up this timestamp issue.
>>
>>
>>
>> # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw
>>
>>
>> Flow Record:
>> Flags = 0x06 FLOW, Unsampled
>> export sysid = 2
>> size = 72
>> first = 0 [1970-01-01 10:00:00]
>> last = 0 [1970-01-01 10:00:00]
>> msec_first = 0
>> msec_last = 0
>> src addr = 110.175.94.222
>> dst addr = 192.168.64.6
>> src port = 58464
>> dst port = 443
>> fwd status = 157
>> tcp flags = 0x00 ......
>> proto = 6
>> (src)tos = 0
>> (in)packets = 9
>> (in)bytes = 1500
>> input = 1678
>> output = 1799
>> ip router = 10.1.4.39
>> received at = 1367541600163 [2013-05-03 10:40:00.163]
>>
>>
>> Flow Record:
>> Flags = 0x06 FLOW, Unsampled
>> export sysid = 2
>> size = 72
>> first = 0 [1970-01-01 10:00:00]
>> last = 0 [1970-01-01 10:00:00]
>> msec_first = 0
>> msec_last = 0
>> src addr = 101.163.67.76
>> dst addr = 192.168.64.6
>> src port = 2735
>> dst port = 443
>> fwd status = 255
>> tcp flags = 0x00 ......
>> proto = 6
>> (src)tos = 0
>> (in)packets = 1
>> (in)bytes = 40
>> input = 1678
>> output = 1799
>> ip router = 10.1.4.39
>> received at = 1367541600163 [2013-05-03 10:40:00.163]
>>
>> Kind Regards,
>> David
>> ------------------------------------------------------------------------------
>> Learn Graph Databases - Download FREE O'Reilly Book
>> "Graph Databases" is the definitive new guide to graph databases and
>> their applications. This 200-page book is written by three acclaimed
>> leaders in the field. The early access version is available now.
>> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
>> _______________________________________________
>> Nfdump-discuss mailing list
>> Nfd...@li...
>> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
>
|
|
From: David W. <da...@on...> - 2013-11-13 03:15:47
|
Hi,
VMware have sent me a patch to test on a system to see if the issue is fixed. I installed the patch on a low-traffic ESXi host and moved a network gateway to it so it would throw the relevant net flow at my collector.
Wireshark dumps indicate that the correct date is being added to the “StartTime and EndTime fields” now. I’ve attached a screenshot from Wireshark. I captured the data with: tcpdump -n -i eth0 -s 1600 -w /tmp/vsphere.pcap 'port 2055’.
I can also confirm that other dumps from other ESX hosts without the patch enter 1970 in those fields.
However, when I view the dump with nfdump for that flow, it has the 1970 dates in it for “first” and “last”.
[root@nfsen ~]# nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/11/13/nfcapd.201311131100 -o raw | grep -A15 -B10 115.70.221.246
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 2
size = 72
first = 0 [1970-01-01 10:00:00]
last = 0 [1970-01-01 10:00:00]
msec_first = 0
msec_last = 0
src addr = X.X.X.X
dst addr = 115.70.221.246
src port = 80
dst port = 13845
fwd status = 66
tcp flags = 0x00 ......
proto = 6
(src)tos = 0
(in)packets = 4
(in)bytes = 2515
input = 10623
output = 8127
ip router = 10.1.4.39
received at = 1384304530019 [2013-11-13 11:02:10.019]
Does this mean that nfcapd/nfdump is not displaying the correct date in the first and last fields OR does it mean that the fields I see in the Wireshark dump, “StartTime and EndTime fields”, do not correlate to the “first” and “last” fields in the nfdump?
Regards,
David
On 8 Nov 2013, at 1:15 pm, David Walsh <da...@on...> wrote:
> Hi,
> Here is an update on this issue…..
>
> From VMware:
>
> "This is with regards to the vDS issue. Just to keep you updated that engineering have isolated the code that seems to be causing the issue. We are working on a fix and I will share further updates as and when the same is available”
>
> Hopefully it will be part of Update 2 of v5.1. I have not tested 5.5 yet.
>
> On 14 May 2013, at 10:59 am, David Walsh <da...@on...> wrote:
>
>> FYI
>>
>> I have finally got VMware looking at this for me. I'll reply to the list when I get more information. I am providing them with the logs of my vDS.
>>
>> Cheers,
>> David
>>
>> On 07/05/2013, at 10:44 AM, David Walsh <da...@on...> wrote:
>>
>>> Hi,
>>> I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5)
>>>
>>> I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter.
>>>
>>> I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in "received at".
>>>
>>> NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time.
>>>
>>> I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they become available to fix up this timestamp issue.
>>>
>>>
>>>
>>> # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw
>>>
>>>
>>> Flow Record:
>>> Flags = 0x06 FLOW, Unsampled
>>> export sysid = 2
>>> size = 72
>>> first = 0 [1970-01-01 10:00:00]
>>> last = 0 [1970-01-01 10:00:00]
>>> msec_first = 0
>>> msec_last = 0
>>> src addr = 110.175.94.222
>>> dst addr = 192.168.64.6
>>> src port = 58464
>>> dst port = 443
>>> fwd status = 157
>>> tcp flags = 0x00 ......
>>> proto = 6
>>> (src)tos = 0
>>> (in)packets = 9
>>> (in)bytes = 1500
>>> input = 1678
>>> output = 1799
>>> ip router = 10.1.4.39
>>> received at = 1367541600163 [2013-05-03 10:40:00.163]
>>>
>>>
>>> Flow Record:
>>> Flags = 0x06 FLOW, Unsampled
>>> export sysid = 2
>>> size = 72
>>> first = 0 [1970-01-01 10:00:00]
>>> last = 0 [1970-01-01 10:00:00]
>>> msec_first = 0
>>> msec_last = 0
>>> src addr = 101.163.67.76
>>> dst addr = 192.168.64.6
>>> src port = 2735
>>> dst port = 443
>>> fwd status = 255
>>> tcp flags = 0x00 ......
>>> proto = 6
>>> (src)tos = 0
>>> (in)packets = 1
>>> (in)bytes = 40
>>> input = 1678
>>> output = 1799
>>> ip router = 10.1.4.39
>>> received at = 1367541600163 [2013-05-03 10:40:00.163]
>>>
>>> Kind Regards,
>>> David
>>> ------------------------------------------------------------------------------
>>> Learn Graph Databases - Download FREE O'Reilly Book
>>> "Graph Databases" is the definitive new guide to graph databases and
>>> their applications. This 200-page book is written by three acclaimed
>>> leaders in the field. The early access version is available now.
>>> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
>>> _______________________________________________
>>> Nfdump-discuss mailing list
>>> Nfd...@li...
>>> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
>>
>
> ------------------------------------------------------------------------------
> November Webinars for C, C++, Fortran Developers
> Accelerate application performance with scalable programming models. Explore
> techniques for threading, error checking, porting, and tuning. Get the most
> from the latest Intel processors and coprocessors. See abstracts and register
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk_______________________________________________
> Nfdump-discuss mailing list
> Nfd...@li...
> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
|
|
From: David W. <da...@on...> - 2013-11-13 04:26:13
|
Hi, As another update, I just spoke with VMware who wanted to know how I was getting along. They advised me that the change they made was to IPfix tags 150 (FlowStatsSeconds) to get the current date in there. (As referenced by: http://www.iana.org/assignments/ipfix/ipfix.xhtml#ipfix-structured-data-types-semantics ) They are confident their end is done and will push this for inclusion into 5.1 update 2 (and 5.5 update1)….due Q1 2014…..unless I get back to them with other data. Cheers, David On 13 Nov 2013, at 1:15 pm, David Walsh <da...@on...> wrote: > Hi, > VMware have sent me a patch to test on a system to see if the issue is fixed. I installed the patch on a low-traffic ESXi host and moved a network gateway to it so it would throw the relevant net flow at my collector. > > Wireshark dumps indicate that the correct date is being added to the “StartTime and EndTime fields” now. I’ve attached a screenshot from Wireshark. I captured the data with: tcpdump -n -i eth0 -s 1600 -w /tmp/vsphere.pcap 'port 2055’. > I can also confirm that other dumps from other ESX hosts without the patch enter 1970 in those fields. > > > <Screen Shot 2013-11-13 at 12.05.31 pm.png> > > > However, when I view the dump with nfdump for that flow, it has the 1970 dates in it for “first” and “last”. > > > [root@nfsen ~]# nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/11/13/nfcapd.201311131100 -o raw | grep -A15 -B10 115.70.221.246 > > Flow Record: > Flags = 0x06 FLOW, Unsampled > export sysid = 2 > size = 72 > first = 0 [1970-01-01 10:00:00] > last = 0 [1970-01-01 10:00:00] > msec_first = 0 > msec_last = 0 > src addr = X.X.X.X > dst addr = 115.70.221.246 > src port = 80 > dst port = 13845 > fwd status = 66 > tcp flags = 0x00 ...... > proto = 6 > (src)tos = 0 > (in)packets = 4 > (in)bytes = 2515 > input = 10623 > output = 8127 > ip router = 10.1.4.39 > received at = 1384304530019 [2013-11-13 11:02:10.019] > > > Does this mean that nfcapd/nfdump is not displaying the correct date in the first and last fields OR does it mean that the fields I see in the Wireshark dump, “StartTime and EndTime fields”, do not correlate to the “first” and “last” fields in the nfdump? > > Regards, > David > > > > On 8 Nov 2013, at 1:15 pm, David Walsh <da...@on...> wrote: > >> Hi, >> Here is an update on this issue….. >> >> From VMware: >> >> "This is with regards to the vDS issue. Just to keep you updated that engineering have isolated the code that seems to be causing the issue. We are working on a fix and I will share further updates as and when the same is available” >> >> Hopefully it will be part of Update 2 of v5.1. I have not tested 5.5 yet. >> >> On 14 May 2013, at 10:59 am, David Walsh <da...@on...> wrote: >> >>> FYI >>> >>> I have finally got VMware looking at this for me. I'll reply to the list when I get more information. I am providing them with the logs of my vDS. >>> >>> Cheers, >>> David >>> >>> On 07/05/2013, at 10:44 AM, David Walsh <da...@on...> wrote: >>> >>>> Hi, >>>> I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5) >>>> >>>> I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter. >>>> >>>> I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in "received at". >>>> >>>> NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time. >>>> >>>> I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they become available to fix up this timestamp issue. >>>> >>>> >>>> >>>> # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw >>>> >>>> >>>> Flow Record: >>>> Flags = 0x06 FLOW, Unsampled >>>> export sysid = 2 >>>> size = 72 >>>> first = 0 [1970-01-01 10:00:00] >>>> last = 0 [1970-01-01 10:00:00] >>>> msec_first = 0 >>>> msec_last = 0 >>>> src addr = 110.175.94.222 >>>> dst addr = 192.168.64.6 >>>> src port = 58464 >>>> dst port = 443 >>>> fwd status = 157 >>>> tcp flags = 0x00 ...... >>>> proto = 6 >>>> (src)tos = 0 >>>> (in)packets = 9 >>>> (in)bytes = 1500 >>>> input = 1678 >>>> output = 1799 >>>> ip router = 10.1.4.39 >>>> received at = 1367541600163 [2013-05-03 10:40:00.163] >>>> >>>> >>>> Flow Record: >>>> Flags = 0x06 FLOW, Unsampled >>>> export sysid = 2 >>>> size = 72 >>>> first = 0 [1970-01-01 10:00:00] >>>> last = 0 [1970-01-01 10:00:00] >>>> msec_first = 0 >>>> msec_last = 0 >>>> src addr = 101.163.67.76 >>>> dst addr = 192.168.64.6 >>>> src port = 2735 >>>> dst port = 443 >>>> fwd status = 255 >>>> tcp flags = 0x00 ...... >>>> proto = 6 >>>> (src)tos = 0 >>>> (in)packets = 1 >>>> (in)bytes = 40 >>>> input = 1678 >>>> output = 1799 >>>> ip router = 10.1.4.39 >>>> received at = 1367541600163 [2013-05-03 10:40:00.163] >>>> >>>> Kind Regards, >>>> David >>>> ------------------------------------------------------------------------------ >>>> Learn Graph Databases - Download FREE O'Reilly Book >>>> "Graph Databases" is the definitive new guide to graph databases and >>>> their applications. This 200-page book is written by three acclaimed >>>> leaders in the field. The early access version is available now. >>>> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may >>>> _______________________________________________ >>>> Nfdump-discuss mailing list >>>> Nfd...@li... >>>> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss >>> >> >> ------------------------------------------------------------------------------ >> November Webinars for C, C++, Fortran Developers >> Accelerate application performance with scalable programming models. Explore >> techniques for threading, error checking, porting, and tuning. Get the most >> from the latest Intel processors and coprocessors. See abstracts and register >> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk_______________________________________________ >> Nfdump-discuss mailing list >> Nfd...@li... >> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss > > ------------------------------------------------------------------------------ > DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps > OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access > Free app hosting. Or install the open source package on any LAMP server. > Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! > http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk_______________________________________________ > Nfdump-discuss mailing list > Nfd...@li... > https://lists.sourceforge.net/lists/listinfo/nfdump-discuss |
|
From: David W. <da...@on...> - 2013-11-16 00:26:11
|
Hi,
Just as an FYI to those who watch this thread.
I tested vSphere 5.5 and it works with correct dates in nfdump/nfsen.
I advised VMware that although the test patch inserted a date, it did not
correlate with what nfdump expected to see and that they should scrap the
patch and look at their 5.5 code for a fix.
Cheers,
David
From: David Walsh [mailto:da...@on...]
Sent: Wednesday, 13 November 2013 2:26 PM
To: nfd...@li...
Subject: Re: [Nfdump-discuss] vSphere 5.1 distributed switch to nfcapd with
IPFIX *Update/Progress
Hi,
As another update, I just spoke with VMware who wanted to know how I was
getting along.
They advised me that the change they made was to IPfix tags 150
(FlowStatsSeconds) to get the current date in there.
(As referenced by:
<http://www.iana.org/assignments/ipfix/ipfix.xhtml#ipfix-structured-data-typ
es-semantics>
http://www.iana.org/assignments/ipfix/ipfix.xhtml#ipfix-structured-data-type
s-semantics )
They are confident their end is done and will push this for inclusion into
5.1 update 2 (and 5.5 update1)..due Q1 2014...unless I get back to them with
other data.
Cheers,
David
On 13 Nov 2013, at 1:15 pm, David Walsh <da...@on...> wrote:
Hi,
VMware have sent me a patch to test on a system to see if the issue is
fixed. I installed the patch on a low-traffic ESXi host and moved a network
gateway to it so it would throw the relevant net flow at my collector.
Wireshark dumps indicate that the correct date is being added to the
"StartTime and EndTime fields" now. I've attached a screenshot from
Wireshark. I captured the data with: tcpdump -n -i eth0 -s 1600 -w
/tmp/vsphere.pcap 'port 2055'.
I can also confirm that other dumps from other ESX hosts without the patch
enter 1970 in those fields.
<Screen Shot 2013-11-13 at 12.05.31 pm.png>
However, when I view the dump with nfdump for that flow, it has the 1970
dates in it for "first" and "last".
[root@nfsen ~]# nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh
-R 2013/11/13/nfcapd.201311131100 -o raw | grep -A15 -B10 115.70.221.246
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 2
size = 72
first = 0 [1970-01-01 10:00:00]
last = 0 [1970-01-01 10:00:00]
msec_first = 0
msec_last = 0
src addr = X.X.X.X
dst addr = 115.70.221.246
src port = 80
dst port = 13845
fwd status = 66
tcp flags = 0x00 ......
proto = 6
(src)tos = 0
(in)packets = 4
(in)bytes = 2515
input = 10623
output = 8127
ip router = 10.1.4.39
received at = 1384304530019 [2013-11-13 11:02:10.019]
Does this mean that nfcapd/nfdump is not displaying the correct date in the
first and last fields OR does it mean that the fields I see in the Wireshark
dump, "StartTime and EndTime fields", do not correlate to the "first" and
"last" fields in the nfdump?
Regards,
David
On 8 Nov 2013, at 1:15 pm, David Walsh <da...@on...> wrote:
Hi,
Here is an update on this issue...
>From VMware:
"This is with regards to the vDS issue. Just to keep you updated that
engineering have isolated the code that seems to be causing the issue. We
are working on a fix and I will share further updates as and when the same
is available"
Hopefully it will be part of Update 2 of v5.1. I have not tested 5.5 yet.
On 14 May 2013, at 10:59 am, David Walsh <da...@on...> wrote:
FYI
I have finally got VMware looking at this for me. I'll reply to the list
when I get more information. I am providing them with the logs of my vDS.
Cheers,
David
On 07/05/2013, at 10:44 AM, David Walsh <da...@on...> wrote:
Hi,
I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server.
(nfsen v 1.3.5)
I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list
on the 13/4/2013 by Peter.
I am receiving the net flow data and below is the output in raw form after I
applied the patch. You will notice that "first" and "last" are set on
1970-01-01 10:00:00. There is an up to date time in the last variable of the
packet in "received at".
NFsen can read the data and it is correct (I compare it to data we pull via
snmp) however NFsen /ndump are formatting the data with timestamps of
1970-01-01 10:00:00 instead of the actual time.
I notice this has been raised on various sites but I have not seen a fix. I
don't mind testing some patches if they become available to fix up this
timestamp issue.
# nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R
2013/05/03/nfcapd.201305031040 -c 100 -o raw
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 2
size = 72
first = 0 [1970-01-01 10:00:00]
last = 0 [1970-01-01 10:00:00]
msec_first = 0
msec_last = 0
src addr = 110.175.94.222
dst addr = 192.168.64.6
src port = 58464
dst port = 443
fwd status = 157
tcp flags = 0x00 ......
proto = 6
(src)tos = 0
(in)packets = 9
(in)bytes = 1500
input = 1678
output = 1799
ip router = 10.1.4.39
received at = 1367541600163 [2013-05-03 10:40:00.163]
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 2
size = 72
first = 0 [1970-01-01 10:00:00]
last = 0 [1970-01-01 10:00:00]
msec_first = 0
msec_last = 0
src addr = 101.163.67.76
dst addr = 192.168.64.6
src port = 2735
dst port = 443
fwd status = 255
tcp flags = 0x00 ......
proto = 6
(src)tos = 0
(in)packets = 1
(in)bytes = 40
input = 1678
output = 1799
ip router = 10.1.4.39
received at = 1367541600163 [2013-05-03 10:40:00.163]
Kind Regards,
David
----------------------------------------------------------------------------
--
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and
their applications. This 200-page book is written by three acclaimed
leaders in the field. The early access version is available now.
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
_______________________________________________
Nfdump-discuss mailing list
Nfd...@li...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
----------------------------------------------------------------------------
--
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and
register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231
<http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
_______________________________________________>
&iu=/4140/ostg.clktrk_______________________________________________
Nfdump-discuss mailing list
Nfd...@li...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
----------------------------------------------------------------------------
--
DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps
OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access
Free app hosting. Or install the open source package on any LAMP server.
Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native!
http://pubads.g.doubleclick.net/gampad/clk?id=63469471
<http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk
_______________________________________________>
&iu=/4140/ostg.clktrk_______________________________________________
Nfdump-discuss mailing list
Nfd...@li...
https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
|
|
From: Peter H. <ph...@us...> - 2013-11-13 16:20:06
|
Thanks for the update David. Could you please send me off list a packet trace - your vsphere.pcap for further analysis. Thanks - Peter On 13/11/13 4:15 AM, David Walsh wrote: > Hi, > VMware have sent me a patch to test on a system to see if the issue is > fixed. I installed the patch on a low-traffic ESXi host and moved a network > gateway to it so it would throw the relevant net flow at my collector. > > Wireshark dumps indicate that the correct date is being added to the “StartTime > and EndTime fields” now. I’ve attached a screenshot from Wireshark. I captured > the data with: tcpdump -n -i eth0 -s 1600 -w /tmp/vsphere.pcap 'port 2055’. > I can also confirm that other dumps from other ESX hosts without the patch enter > 1970 in those fields. > > > > > However, when I view the dump with nfdump for that flow, it has the 1970 dates > in it for “first” and “last”. > > > [root@nfsen ~]# nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R > 2013/11/13/nfcapd.201311131100 -o raw | grep -A15 -B10 115.70.221.246 > > Flow Record: > Flags = 0x06 FLOW, Unsampled > export sysid = 2 > size = 72 > first = 0 [1970-01-01 10:00:00] > last = 0 [1970-01-01 10:00:00] > msec_first = 0 > msec_last = 0 > src addr = X.X.X.X > dst addr = 115.70.221.246 > src port = 80 > dst port = 13845 > fwd status = 66 > tcp flags = 0x00 ...... > proto = 6 > (src)tos = 0 > (in)packets = 4 > (in)bytes = 2515 > input = 10623 > output = 8127 > ip router = 10.1.4.39 > received at = 1384304530019 [2013-11-13 11:02:10.019] > > > Does this mean that nfcapd/nfdump is not displaying the correct date in the > first and last fields OR does it mean that the fields I see in the Wireshark > dump, “StartTime and EndTime fields”, do not correlate to the “first” and “last” > fields in the nfdump? > > Regards, > David > > > > On 8 Nov 2013, at 1:15 pm, David Walsh <da...@on... > <mailto:da...@on...>> wrote: > >> Hi, >> Here is an update on this issue….. >> >> From VMware: >> >> "This is with regards to the vDS issue. Just to keep you updated that >> engineering have isolated the code that seems to be causing the issue. We are >> working on a fix and I will share further updates as and when the same is >> available” >> >> Hopefully it will be part of Update 2 of v5.1. I have not tested 5.5 yet. >> >> On 14 May 2013, at 10:59 am, David Walsh <da...@on... >> <mailto:da...@on...>> wrote: >> >>> FYI >>> >>> I have finally got VMware looking at this for me. I'll reply to the list >>> when I get more information. I am providing them with the logs of my vDS. >>> >>> Cheers, >>> David >>> >>> On 07/05/2013, at 10:44 AM, David Walsh <da...@on... >>> <mailto:da...@on...>> wrote: >>> >>>> Hi, >>>> I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. >>>> (nfsen v 1.3.5) >>>> >>>> I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list >>>> on the 13/4/2013 by Peter. >>>> >>>> I am receiving the net flow data and below is the output in raw form after I >>>> applied the patch. You will notice that "first" and "last" are set on >>>> 1970-01-01 10:00:00. There is an up to date time in the last variable of the >>>> packet in "received at". >>>> >>>> NFsen can read the data and it is correct (I compare it to data we pull via >>>> snmp) however NFsen /ndump are formatting the data with timestamps of >>>> 1970-01-01 10:00:00 instead of the actual time. >>>> >>>> I notice this has been raised on various sites but I have not seen a fix. I >>>> don't mind testing some patches if they become available to fix up this >>>> timestamp issue. >>>> >>>> >>>> >>>> # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R >>>> 2013/05/03/nfcapd.201305031040 -c 100 -o raw >>>> >>>> >>>> Flow Record: >>>> Flags = 0x06 FLOW, Unsampled >>>> export sysid = 2 >>>> size = 72 >>>> first = 0 [1970-01-01 10:00:00] >>>> last = 0 [1970-01-01 10:00:00] >>>> msec_first = 0 >>>> msec_last = 0 >>>> src addr = 110.175.94.222 >>>> dst addr = 192.168.64.6 >>>> src port = 58464 >>>> dst port = 443 >>>> fwd status = 157 >>>> tcp flags = 0x00 ...... >>>> proto = 6 >>>> (src)tos = 0 >>>> (in)packets = 9 >>>> (in)bytes = 1500 >>>> input = 1678 >>>> output = 1799 >>>> ip router = 10.1.4.39 >>>> received at = 1367541600163 [2013-05-03 10:40:00.163] >>>> >>>> >>>> Flow Record: >>>> Flags = 0x06 FLOW, Unsampled >>>> export sysid = 2 >>>> size = 72 >>>> first = 0 [1970-01-01 10:00:00] >>>> last = 0 [1970-01-01 10:00:00] >>>> msec_first = 0 >>>> msec_last = 0 >>>> src addr = 101.163.67.76 >>>> dst addr = 192.168.64.6 >>>> src port = 2735 >>>> dst port = 443 >>>> fwd status = 255 >>>> tcp flags = 0x00 ...... >>>> proto = 6 >>>> (src)tos = 0 >>>> (in)packets = 1 >>>> (in)bytes = 40 >>>> input = 1678 >>>> output = 1799 >>>> ip router = 10.1.4.39 >>>> received at = 1367541600163 [2013-05-03 10:40:00.163] >>>> >>>> Kind Regards, >>>> David >>>> ------------------------------------------------------------------------------ >>>> Learn Graph Databases - Download FREE O'Reilly Book >>>> "Graph Databases" is the definitive new guide to graph databases and >>>> their applications. This 200-page book is written by three acclaimed >>>> leaders in the field. The early access version is available now. >>>> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may >>>> _______________________________________________ >>>> Nfdump-discuss mailing list >>>> Nfd...@li... >>>> <mailto:Nfd...@li...> >>>> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss >>> >> >> ------------------------------------------------------------------------------ >> November Webinars for C, C++, Fortran Developers >> Accelerate application performance with scalable programming models. Explore >> techniques for threading, error checking, porting, and tuning. Get the most >> from the latest Intel processors and coprocessors. See abstracts and register >> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk_______________________________________________ >> Nfdump-discuss mailing list >> Nfd...@li... >> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss > > > > ------------------------------------------------------------------------------ > DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps > OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access > Free app hosting. Or install the open source package on any LAMP server. > Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! > http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Nfdump-discuss mailing list > Nfd...@li... > https://lists.sourceforge.net/lists/listinfo/nfdump-discuss > -- Be nice to your netflow data. Use NfSen and nfdump :) |
|
From: David W. <da...@on...> - 2013-06-21 01:09:28
|
Hi All,
I was having issues with the dates in IPFIX net flow packets from vSphere 5.1 vDistributed Switched at the beginning of May. Nfsen and NFdump display the date as 1970 00:00:00 and not the current time.
After dumping the netfow packets as per Peter's suggestion and analysing in Wireshark, and seeing an issue that someone else had raised on the VMware communities site, I opened a support case with VMware.
After a bit of back and forwards, I received a response from an engineer this morning.
Peter, would you be able to have a read and comment on his (VMware's point of view) answer please?
"
Hello David,
My name is XXXXX and I am a senior engineer assisting XXXXX on this SR. Our sincere apologies for the delay here. One of my colleagues in our Escalations Team got a chance to reproduce this.
As per the standard defined by Cisco for Netflow packet header, time is calculated in 'Unix Seconds' i.e. Seconds since 0000 Coordinated Universal Time (UTC) 1970.
Details:
http://www.cisco.com/en/US/docs/ios-xml/ios/netflow/configuration/12-4t/cfg-nflow-data-expt.html
In the example cited by you earlier (see attached screesnhot), the duration of the flow if 0 seconds:
[Duration: 0.000000000 seconds]
StartTime: Jan 22, 1970 09:06:47.000000000 EST
EndTime: Jan 22, 1970 09:06:47.000000000 EST
hence the start time and the end time are same. In our opinion this "time setting" is default as per Cisco's Implementation and would be noticed with any netflow enabled device.
I am not in office tomorrow however we can discuss this on Monday. Let me know if you have any questions in the interim.
"
This was the screenshot in Wireshark I sent them to help track down my issue.
On 07/05/2013, at 5:10 PM, Peter Haag <ph...@us...> wrote:
>
> Run something like:
>
> ./tcpdump -n -i <if> -s 1600 -w /tmp/vsphere.pcap 'port xxxxx'
>
> Let it run for a couple of minutes - at least twice the template repeat time and send me the pcap.
>
> Thanks
>
> - Peter
>
> On 7/5/13 2:44 AM, David Walsh wrote:
>> Hi,
>> I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5)
>>
>> I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter.
>>
>> I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in "received at".
>>
>> NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time.
>>
>> I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they become available to fix up this timestamp issue.
>>
>>
>>
>> # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw
>>
>>
>> Flow Record:
>> Flags = 0x06 FLOW, Unsampled
>> export sysid = 2
>> size = 72
>> first = 0 [1970-01-01 10:00:00]
>> last = 0 [1970-01-01 10:00:00]
>> msec_first = 0
>> msec_last = 0
>> src addr = 110.175.94.222
>> dst addr = 192.168.64.6
>> src port = 58464
>> dst port = 443
>> fwd status = 157
>> tcp flags = 0x00 ......
>> proto = 6
>> (src)tos = 0
>> (in)packets = 9
>> (in)bytes = 1500
>> input = 1678
>> output = 1799
>> ip router = 10.1.4.39
>> received at = 1367541600163 [2013-05-03 10:40:00.163]
>>
>>
>> Flow Record:
>> Flags = 0x06 FLOW, Unsampled
>> export sysid = 2
>> size = 72
>> first = 0 [1970-01-01 10:00:00]
>> last = 0 [1970-01-01 10:00:00]
>> msec_first = 0
>> msec_last = 0
>> src addr = 101.163.67.76
>> dst addr = 192.168.64.6
>> src port = 2735
>> dst port = 443
>> fwd status = 255
>> tcp flags = 0x00 ......
>> proto = 6
>> (src)tos = 0
>> (in)packets = 1
>> (in)bytes = 40
>> input = 1678
>> output = 1799
>> ip router = 10.1.4.39
>> received at = 1367541600163 [2013-05-03 10:40:00.163]
>>
>> Kind Regards,
>> David
>> ------------------------------------------------------------------------------
>> Learn Graph Databases - Download FREE O'Reilly Book
>> "Graph Databases" is the definitive new guide to graph databases and
>> their applications. This 200-page book is written by three acclaimed
>> leaders in the field. The early access version is available now.
>> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
>> _______________________________________________
>> Nfdump-discuss mailing list
>> Nfd...@li...
>> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
>>
>
> --
> Be nice to your netflow data. Use NfSen and nfdump :)
|
|
From: Peter H. <ph...@us...> - 2013-06-22 16:09:06
|
Hi David, On 6/21/13 W25 2:39, David Walsh wrote: > Hi All, > I was having issues with the dates in IPFIX net flow packets from vSphere 5.1 vDistributed Switched at the > beginning of May. Nfsen and NFdump display the date as 1970 00:00:00 and not the current time. > After dumping the netfow packets as per Peter's suggestion and analysing in Wireshark, and seeing an issue that someone > else had raised on the VMware communities site, I opened a support case with VMware. > > After a bit of back and forwards, I received a response from an engineer this morning. > > Peter, would you be able to have a read and comment on his (VMware's point of view) answer please? Hmm .. the comment is not very helpful. In the wireshark screenshot, you clearly can see, that the version is 10. This means it's IPFIX and not netflow of any version. Exactly at that point, netflow is different from IPFIX, in that IPFIX uses other timestamp models. Although, the reference is netflow v9 which is different. It seems to me, that the VMware reference is not IPFIX compatible. I need to check the pcap, when I'm back in the office. Sorry for the delay. Stay tuned. - Peter > > > " > Hello David, > > My name is XXXXX and I am a senior engineer assisting XXXXX on this SR. Our sincere apologies for the delay here. One of > my colleagues in our Escalations Team got a chance to reproduce this. > > As per the standard defined by Cisco for Netflow packet header, time is calculated in 'Unix Seconds' i.e. Seconds since > 0000 Coordinated Universal Time (UTC) 1970. > > Details: > > http://www.cisco.com/en/US/docs/ios-xml/ios/netflow/configuration/12-4t/cfg-nflow-data-expt.html > > In the example cited by you earlier (see attached screesnhot), the duration of the flow if 0 seconds: > > [Duration: 0.000000000 seconds] > StartTime: Jan 22, 1970 09:06:47.000000000 EST > EndTime: Jan 22, 1970 09:06:47.000000000 EST > > hence the start time and the end time are same. In our opinion this "time setting" is default as per Cisco's > Implementation and would be noticed with any netflow enabled device. > > I am not in office tomorrow however we can discuss this on Monday. Let me know if you have any questions in the interim. > " > > > This was the screenshot in Wireshark I sent them to help track down my issue. > > > > > > On 07/05/2013, at 5:10 PM, Peter Haag <ph...@us... <mailto:ph...@us...>> wrote: > >> >> Run something like: >> >> ./tcpdump -n -i <if> -s 1600 -w /tmp/vsphere.pcap 'port xxxxx' >> >> Let it run for a couple of minutes - at least twice the template repeat time and send me the pcap. >> >> Thanks >> >> - Peter >> >> On 7/5/13 2:44 AM, David Walsh wrote: >>> Hi, >>> I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5) >>> >>> I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter. >>> >>> I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that >>> "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in >>> "received at". >>> >>> NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting >>> the data with timestamps of 1970-01-01 10:00:00 instead of the actual time. >>> >>> I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they >>> become available to fix up this timestamp issue. >>> >>> >>> >>> # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw >>> >>> >>> Flow Record: >>> Flags = 0x06 FLOW, Unsampled >>> export sysid = 2 >>> size = 72 >>> first = 0 [1970-01-01 10:00:00] >>> last = 0 [1970-01-01 10:00:00] >>> msec_first = 0 >>> msec_last = 0 >>> src addr = 110.175.94.222 >>> dst addr = 192.168.64.6 >>> src port = 58464 >>> dst port = 443 >>> fwd status = 157 >>> tcp flags = 0x00 ...... >>> proto = 6 >>> (src)tos = 0 >>> (in)packets = 9 >>> (in)bytes = 1500 >>> input = 1678 >>> output = 1799 >>> ip router = 10.1.4.39 >>> received at = 1367541600163 [2013-05-03 10:40:00.163] >>> >>> >>> Flow Record: >>> Flags = 0x06 FLOW, Unsampled >>> export sysid = 2 >>> size = 72 >>> first = 0 [1970-01-01 10:00:00] >>> last = 0 [1970-01-01 10:00:00] >>> msec_first = 0 >>> msec_last = 0 >>> src addr = 101.163.67.76 >>> dst addr = 192.168.64.6 >>> src port = 2735 >>> dst port = 443 >>> fwd status = 255 >>> tcp flags = 0x00 ...... >>> proto = 6 >>> (src)tos = 0 >>> (in)packets = 1 >>> (in)bytes = 40 >>> input = 1678 >>> output = 1799 >>> ip router = 10.1.4.39 >>> received at = 1367541600163 [2013-05-03 10:40:00.163] >>> >>> Kind Regards, >>> David >>> ------------------------------------------------------------------------------ >>> Learn Graph Databases - Download FREE O'Reilly Book >>> "Graph Databases" is the definitive new guide to graph databases and >>> their applications. This 200-page book is written by three acclaimed >>> leaders in the field. The early access version is available now. >>> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may >>> _______________________________________________ >>> Nfdump-discuss mailing list >>> Nfd...@li... <mailto:Nfd...@li...> >>> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss >>> >> >> -- >> Be nice to your netflow data. Use NfSen and nfdump :) > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > > > > _______________________________________________ > Nfdump-discuss mailing list > Nfd...@li... > https://lists.sourceforge.net/lists/listinfo/nfdump-discuss > -- -- Be nice to your netflow data |
|
From: David W. <da...@on...> - 2013-07-18 00:52:04
|
Hi All,
I just had a call from VMware and they have identified a bug. They were not able to confirm with me that it was directly related to the dates but one has to assume it was as why would they bother ringing me.
Anyway, as of the 18th July, 2013, they will be filing an internal bug to their engineering team and will try sort out a fix. This may take weeks. I'll update the list as I get information.
cheers,
David
On 21/06/2013, at 10:39 AM, David Walsh <da...@on...> wrote:
> Hi All,
> I was having issues with the dates in IPFIX net flow packets from vSphere 5.1 vDistributed Switched at the beginning of May. Nfsen and NFdump display the date as 1970 00:00:00 and not the current time.
> After dumping the netfow packets as per Peter's suggestion and analysing in Wireshark, and seeing an issue that someone else had raised on the VMware communities site, I opened a support case with VMware.
>
> After a bit of back and forwards, I received a response from an engineer this morning.
>
> Peter, would you be able to have a read and comment on his (VMware's point of view) answer please?
>
>
> "
> Hello David,
>
> My name is XXXXX and I am a senior engineer assisting XXXXX on this SR. Our sincere apologies for the delay here. One of my colleagues in our Escalations Team got a chance to reproduce this.
>
> As per the standard defined by Cisco for Netflow packet header, time is calculated in 'Unix Seconds' i.e. Seconds since 0000 Coordinated Universal Time (UTC) 1970.
>
> Details:
>
> http://www.cisco.com/en/US/docs/ios-xml/ios/netflow/configuration/12-4t/cfg-nflow-data-expt.html
>
> In the example cited by you earlier (see attached screesnhot), the duration of the flow if 0 seconds:
>
> [Duration: 0.000000000 seconds]
> StartTime: Jan 22, 1970 09:06:47.000000000 EST
> EndTime: Jan 22, 1970 09:06:47.000000000 EST
>
> hence the start time and the end time are same. In our opinion this "time setting" is default as per Cisco's Implementation and would be noticed with any netflow enabled device.
>
> I am not in office tomorrow however we can discuss this on Monday. Let me know if you have any questions in the interim.
> "
>
>
> This was the screenshot in Wireshark I sent them to help track down my issue.
>
> <Wireshark 2013-06-04 at 4.34.20 PM.png>
>
>
>
>
> On 07/05/2013, at 5:10 PM, Peter Haag <ph...@us...> wrote:
>
>>
>> Run something like:
>>
>> ./tcpdump -n -i <if> -s 1600 -w /tmp/vsphere.pcap 'port xxxxx'
>>
>> Let it run for a couple of minutes - at least twice the template repeat time and send me the pcap.
>>
>> Thanks
>>
>> - Peter
>>
>> On 7/5/13 2:44 AM, David Walsh wrote:
>>> Hi,
>>> I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5)
>>>
>>> I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter.
>>>
>>> I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in "received at".
>>>
>>> NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time.
>>>
>>> I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they become available to fix up this timestamp issue.
>>>
>>>
>>>
>>> # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw
>>>
>>>
>>> Flow Record:
>>> Flags = 0x06 FLOW, Unsampled
>>> export sysid = 2
>>> size = 72
>>> first = 0 [1970-01-01 10:00:00]
>>> last = 0 [1970-01-01 10:00:00]
>>> msec_first = 0
>>> msec_last = 0
>>> src addr = 110.175.94.222
>>> dst addr = 192.168.64.6
>>> src port = 58464
>>> dst port = 443
>>> fwd status = 157
>>> tcp flags = 0x00 ......
>>> proto = 6
>>> (src)tos = 0
>>> (in)packets = 9
>>> (in)bytes = 1500
>>> input = 1678
>>> output = 1799
>>> ip router = 10.1.4.39
>>> received at = 1367541600163 [2013-05-03 10:40:00.163]
>>>
>>>
>>> Flow Record:
>>> Flags = 0x06 FLOW, Unsampled
>>> export sysid = 2
>>> size = 72
>>> first = 0 [1970-01-01 10:00:00]
>>> last = 0 [1970-01-01 10:00:00]
>>> msec_first = 0
>>> msec_last = 0
>>> src addr = 101.163.67.76
>>> dst addr = 192.168.64.6
>>> src port = 2735
>>> dst port = 443
>>> fwd status = 255
>>> tcp flags = 0x00 ......
>>> proto = 6
>>> (src)tos = 0
>>> (in)packets = 1
>>> (in)bytes = 40
>>> input = 1678
>>> output = 1799
>>> ip router = 10.1.4.39
>>> received at = 1367541600163 [2013-05-03 10:40:00.163]
>>>
>>> Kind Regards,
>>> David
>>> ------------------------------------------------------------------------------
>>> Learn Graph Databases - Download FREE O'Reilly Book
>>> "Graph Databases" is the definitive new guide to graph databases and
>>> their applications. This 200-page book is written by three acclaimed
>>> leaders in the field. The early access version is available now.
>>> Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
>>> _______________________________________________
>>> Nfdump-discuss mailing list
>>> Nfd...@li...
>>> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
>>>
>>
>> --
>> Be nice to your netflow data. Use NfSen and nfdump :)
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev_______________________________________________
> Nfdump-discuss mailing list
> Nfd...@li...
> https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
|
Thanks for helping keep SourceForge clean.
X