Re: [Nfdump-discuss] vSphere 5.1 distributed switch to nfcapd with IPFIX *Update/Progress
netflow collecting and processing tools
Brought to you by:
phaag
From: David W. <da...@on...> - 2013-11-16 00:26:11
|
Hi, Just as an FYI to those who watch this thread. I tested vSphere 5.5 and it works with correct dates in nfdump/nfsen. I advised VMware that although the test patch inserted a date, it did not correlate with what nfdump expected to see and that they should scrap the patch and look at their 5.5 code for a fix. Cheers, David From: David Walsh [mailto:da...@on...] Sent: Wednesday, 13 November 2013 2:26 PM To: nfd...@li... Subject: Re: [Nfdump-discuss] vSphere 5.1 distributed switch to nfcapd with IPFIX *Update/Progress Hi, As another update, I just spoke with VMware who wanted to know how I was getting along. They advised me that the change they made was to IPfix tags 150 (FlowStatsSeconds) to get the current date in there. (As referenced by: <http://www.iana.org/assignments/ipfix/ipfix.xhtml#ipfix-structured-data-typ es-semantics> http://www.iana.org/assignments/ipfix/ipfix.xhtml#ipfix-structured-data-type s-semantics ) They are confident their end is done and will push this for inclusion into 5.1 update 2 (and 5.5 update1)..due Q1 2014...unless I get back to them with other data. Cheers, David On 13 Nov 2013, at 1:15 pm, David Walsh <da...@on...> wrote: Hi, VMware have sent me a patch to test on a system to see if the issue is fixed. I installed the patch on a low-traffic ESXi host and moved a network gateway to it so it would throw the relevant net flow at my collector. Wireshark dumps indicate that the correct date is being added to the "StartTime and EndTime fields" now. I've attached a screenshot from Wireshark. I captured the data with: tcpdump -n -i eth0 -s 1600 -w /tmp/vsphere.pcap 'port 2055'. I can also confirm that other dumps from other ESX hosts without the patch enter 1970 in those fields. <Screen Shot 2013-11-13 at 12.05.31 pm.png> However, when I view the dump with nfdump for that flow, it has the 1970 dates in it for "first" and "last". [root@nfsen ~]# nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/11/13/nfcapd.201311131100 -o raw | grep -A15 -B10 115.70.221.246 Flow Record: Flags = 0x06 FLOW, Unsampled export sysid = 2 size = 72 first = 0 [1970-01-01 10:00:00] last = 0 [1970-01-01 10:00:00] msec_first = 0 msec_last = 0 src addr = X.X.X.X dst addr = 115.70.221.246 src port = 80 dst port = 13845 fwd status = 66 tcp flags = 0x00 ...... proto = 6 (src)tos = 0 (in)packets = 4 (in)bytes = 2515 input = 10623 output = 8127 ip router = 10.1.4.39 received at = 1384304530019 [2013-11-13 11:02:10.019] Does this mean that nfcapd/nfdump is not displaying the correct date in the first and last fields OR does it mean that the fields I see in the Wireshark dump, "StartTime and EndTime fields", do not correlate to the "first" and "last" fields in the nfdump? Regards, David On 8 Nov 2013, at 1:15 pm, David Walsh <da...@on...> wrote: Hi, Here is an update on this issue... >From VMware: "This is with regards to the vDS issue. Just to keep you updated that engineering have isolated the code that seems to be causing the issue. We are working on a fix and I will share further updates as and when the same is available" Hopefully it will be part of Update 2 of v5.1. I have not tested 5.5 yet. On 14 May 2013, at 10:59 am, David Walsh <da...@on...> wrote: FYI I have finally got VMware looking at this for me. I'll reply to the list when I get more information. I am providing them with the logs of my vDS. Cheers, David On 07/05/2013, at 10:44 AM, David Walsh <da...@on...> wrote: Hi, I have some vSphere 5.1 VDS's sending IPFIX net flow to our nfsen server. (nfsen v 1.3.5) I am running nfdump Version: 1.6.9 with the IPFIX patch posted on this list on the 13/4/2013 by Peter. I am receiving the net flow data and below is the output in raw form after I applied the patch. You will notice that "first" and "last" are set on 1970-01-01 10:00:00. There is an up to date time in the last variable of the packet in "received at". NFsen can read the data and it is correct (I compare it to data we pull via snmp) however NFsen /ndump are formatting the data with timestamps of 1970-01-01 10:00:00 instead of the actual time. I notice this has been raised on various sites but I have not seen a fix. I don't mind testing some patches if they become available to fix up this timestamp issue. # nfdump -M /opt/data/nfsen/profiles-data/live/netflow-vds-vsh -R 2013/05/03/nfcapd.201305031040 -c 100 -o raw Flow Record: Flags = 0x06 FLOW, Unsampled export sysid = 2 size = 72 first = 0 [1970-01-01 10:00:00] last = 0 [1970-01-01 10:00:00] msec_first = 0 msec_last = 0 src addr = 110.175.94.222 dst addr = 192.168.64.6 src port = 58464 dst port = 443 fwd status = 157 tcp flags = 0x00 ...... proto = 6 (src)tos = 0 (in)packets = 9 (in)bytes = 1500 input = 1678 output = 1799 ip router = 10.1.4.39 received at = 1367541600163 [2013-05-03 10:40:00.163] Flow Record: Flags = 0x06 FLOW, Unsampled export sysid = 2 size = 72 first = 0 [1970-01-01 10:00:00] last = 0 [1970-01-01 10:00:00] msec_first = 0 msec_last = 0 src addr = 101.163.67.76 dst addr = 192.168.64.6 src port = 2735 dst port = 443 fwd status = 255 tcp flags = 0x00 ...... proto = 6 (src)tos = 0 (in)packets = 1 (in)bytes = 40 input = 1678 output = 1799 ip router = 10.1.4.39 received at = 1367541600163 [2013-05-03 10:40:00.163] Kind Regards, David ---------------------------------------------------------------------------- -- Learn Graph Databases - Download FREE O'Reilly Book "Graph Databases" is the definitive new guide to graph databases and their applications. This 200-page book is written by three acclaimed leaders in the field. The early access version is available now. Download your free book today! http://p.sf.net/sfu/neotech_d2d_may _______________________________________________ Nfdump-discuss mailing list Nfd...@li... https://lists.sourceforge.net/lists/listinfo/nfdump-discuss ---------------------------------------------------------------------------- -- November Webinars for C, C++, Fortran Developers Accelerate application performance with scalable programming models. Explore techniques for threading, error checking, porting, and tuning. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60136231 <http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk _______________________________________________> &iu=/4140/ostg.clktrk_______________________________________________ Nfdump-discuss mailing list Nfd...@li... https://lists.sourceforge.net/lists/listinfo/nfdump-discuss ---------------------------------------------------------------------------- -- DreamFactory - Open Source REST & JSON Services for HTML5 & Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471 <http://pubads.g.doubleclick.net/gampad/clk?id=63469471&iu=/4140/ostg.clktrk _______________________________________________> &iu=/4140/ostg.clktrk_______________________________________________ Nfdump-discuss mailing list Nfd...@li... https://lists.sourceforge.net/lists/listinfo/nfdump-discuss |