Menu
▾
▴
Home
FlowViewer
Powered by flow-tools and SiLK
Advancing Network Traffic Situational Awareness
FAQ
FlowViewer provides a dynamic web front-end to two powerful open-source netflow data collector and analyzers, Mark Fullmer’s flow-tools suite and the Carnegie Mellon NetSA group’s netflow data capture/analyzer, SiLK. The inclusion of the underlying SiLK tool set enables existing FlowViewer users to continue to use the tool with the newer IPFIX netflow data protocol.
FlowViewer has been developed for NASA’s Earth Sciences Data and Information System (ESDIS) networks, and credit goes to NASA for their usual outstanding support of innovation.
FlowViewer provides users with the ability to quickly extract network management information of interest from voluminous quantities of stored netflow data. The user can configure a [Dashboard] of continuously updating FlowTrackings to maintain a situational awareness of his organization's network traffic. FlowViewer consists of three primary tools: FlowViewer, FlowGrapher and FlowTracker. The user is able to filter data (inclusion or exclusion) by device, IP address range, port, router interface, autonomous system (AS), specified time interval, protocols, TOS field, TCP flags, exporter, and next-hop. All generated reports and filters can be saved for future application. The user can switch between the tools preserving the previously specified filter. FlowViewer makes flow data analysis and tracking quick and easy.
[FlowViewer] enables the user to create text based reports from filtered netflow data. Several different reporting formats are provided. Each of these reports can be sorted by column heading.
[FlowGrapher] enables the user to graph the bandwidth used by a filtered subset of netflow data during a specified time period. Resulting reports include the graph and a textual listing of the largest flows.
[FlowTracker] enables the user to maintain a long-term history of a particular traffic subset. FlowTrackings consist of five graphs of traffic over suceesive longer time periods: Daily, Weekly, Monthly, Yearly, and Last 3 Years.
The user must install and configure either flow-tools, or SiLK. Users already running FlowViewer with flow-tools may opt to install SiLK in parallel to handle IPFIX exporters while leaving older exporters in place. The ability to collect and analyze IPFIX data requires SiLK (download SiLK) now at version 3.7.1. FlowViewer v4.0 continues to work with flow-tools for pre-IPFIX versions of netflow.
For more information including software requirements and installation instructions, please review the FAQ, User's Guide or the README file. Or, contact me directly at jloiacon@csc.com. For somewhat larger images, please see the Screenshots.
I see the requirement for Silk V3.0. I recently got Silk v2.5 up and running. Is it a futile effort to have this version of FlowViewer work with Silk v2.5? Reduced functionality, or simply won't work? Can someone please elaborate on this, or give some idea when Silk v3.0 will be available for open release? Thank You!
Yes - I tried v2.5 myself, but the limitations are (as I recall) that flow start and end times were not recoverable from netflow v9 and this wreaks havoc on FlowGrapher and FlowTracker. I can't recall whether the FlowViewer tool worked. I didn't try v2.5 on netflow v5 - probably because flow-tools works just fine there - but you might try it as it will be good prep for moving to v3.5 whenever.
Here's the (unfortunate) latest on SiLK:
"Due to changes in the oversight of the SEI that are outside of our control, major new releases of NetSA software are required to go through release review by the Office of the Secretary of Defense (OSD) before the software may be given to anyone who is not a federal government employee. Unfortunately, new releases of SiLK have been stuck in this process for a long time, and currently there is no estimate as to when this review will be completed."
Thanks for your timely reply. That's what I was afraid of.
I'm just starting to gather some experience with handling netflow data. My ASA firewall seems to be limited to Netflow v9 so I fell into Silk to handle that detail, saw the connection to Flowviewer and was hoping to implement, learn, and have that capability. I suppose there is still lots to learn, and useful data to be had, with Silk 2.5 though.
Ultimately, I'm pretty agnostic on what tool to use, mostly have been looking for open source (free) tools and chasing various ideas. Have tried nfdump & nfsen a bit. Mostly just collecting data at this point. Always looking for ideas.
No need to support IPv6.
Last edit: Junior 2013-06-13
But wait! No sooner do we bemoan the delay of the general public release of SiLK and presto: it's available (as of June 20, 2013)!
Download here: SiLK
Joe
With collector & monitor running I get the following periodic error messages:
/usr/local/www/apache24/cgi-bin/FlowViewer4.6/FlowMonitorFiles/FlowMonitorRRDtool/I.rrd
/usr/local/www/apache24/cgi-bin/FlowViewer4.6/FlowMonitorFiles/FlowMonitorRRDtool/INS.rrd
None of my monitors are called I.rrd or INS.rrd...would you have any pointers
Regards Luke
Hi Joe,
Great tool!
Question: My local Flowviewer box is running in UTC. Among other netflow sources, I've recently started rsyncing a flow-tools data repository from a box in Colorado that happens to be running in MDT (UTC-6). Not a typical configuration, I realize. Does this timezone discrepancy create an insurmountable problem for Flowviewer?
Flowviewer and Flowgrapher reports pulling data from the Colorado device look reasonable (although I haven't done a detailed analysis on the actual numbers), but Flowmonitor/RRD seem not to be picking up any of the data between hours 00 and 06 UTC, resulting in obvious gaps in the daily and weekly graphs. On the theory that RRD was looking only in the flow-tools daily data directory where it expects to find the data it's looking for, and consequently not finding it for the first 6 hours of each day due to the TZ lag, I wrote a script to copy flow-tools data files for hours 18-24 of the current MDT day into the following day's directory (which I create as part of the script). The script runs immediately following completion of each rsync. This file copy seems to have more-or-less solved the Flowmonitor/RRD output problem (i.e., daily and weekly graphs now look reasonable), but I consider this strategy to be only a diagnostic exercise -- not a good operational solution.
I see in the User's Guide that FV v4.6 uses the system TZ rather than setting TZ based on configuration files. Seems reasonable, but in my case it appears to mean that Flowmonitor will necessarily mis-interpret my Colorado data (or any non-UTC data, presumably).
Any advice on how I might tackle this problem (short of asking Colorado to start using UTC)? Thanks!
Hey Marty,
Good to hear from you. This could be a little tricky. First, I believe
anything that FlowViewer presents for Colorado will be off by 6 hours,
whether FlowGrapher or FlowMonitor.
In version 4.6 FlowViewer was updated to be able to work with SiLK's
timezone handling - but if I recall, this was to be able to better
accommodate those working completely in local time, and not to accommodate
multiple timezones in one deployment.
You might look at the SiLK toolset here and see if you can translate the
records to your system timezone, namely UTC.
Best!
Joe
From: "Martyn Noss" mjnoss@users.sf.net
To: "[flowviewer:wiki] " Home@wiki.flowviewer.p.re.sf.net
Date: 10/25/2016 02:13 PM
Subject: [flowviewer:wiki] Discussion for Home page
Hi Joe,
Great tool!
Question: My local Flowviewer box is running in UTC. Among other netflow
sources, I've recently started rsyncing a flow-tools data repository from
a box in Colorado that happens to be running in MDT (UTC-6). Not a typical
configuration, I realize. Does this timezone discrepancy create an
insurmountable problem for Flowviewer?
Flowviewer and Flowgrapher reports pulling data from the Colorado device
look reasonable (although I haven't done a detailed analysis on the actual
numbers), but Flowmonitor/RRD seem not to be picking up any of the data
between hours 00 and 06 UTC, resulting in obvious gaps in the daily and
weekly graphs. On the theory that RRD was looking only in the flow-tools
daily data directory where it expects to find the data it's looking for,
and consequently not finding it for the first 6 hours of each day due to
the TZ lag, I wrote a script to copy flow-tools data files for hours 18-24
of the current MDT day into the following day's directory (which I create
as part of the script). The script runs immediately following completion
of each rsync. This file copy seems to have more-or-less solved the
Flowmonitor/RRD output problem (i.e., daily and weekly graphs now look
reasonable), but I consider this strategy to be only a diagnostic exercise
-- not a good operational solution.
I see in the User's Guide that FV v4.6 uses the system TZ rather than
setting TZ based on configuration files. Seems reasonable, but in my case
it appears to mean that Flowmonitor will necessarily mis-interpret my
Colorado data (or any non-UTC data, presumably).
Any advice on how I might tackle this problem (short of asking Colorado to
start using UTC)? Thanks!
Sent from sourceforge.net because you indicated interest in
https://sourceforge.net/p/flowviewer/wiki/Home/
To unsubscribe from further messages, please visit
https://sourceforge.net/auth/subscriptions/
Hi Joe,
Thanks for the prompt reply. You've confirmed what I supposed, i.e., FlowViewer isn't designed to handle multiple timezones in a single deployment. Now that I know I'm not dealing with a configuration problem, I can turn to other strategies. Regarding using the SiLK toolset, Colorado is still using flow-tools so I'm restricted to flow-tools commands; not as powerful as SiLK, but your point remains the same. I'll rummage around in the man pages and see what I can find. Thanks again!
Marty
I need help from someone with a working Flowviewer and silk system i am struggling to graph the data but i can capture it though silk stores data in the root dir only
I need help from someone with a working Flowviewer and silk system i am struggling to graph the data but i can capture it though silk stores data in the root dir only
Did you use the 'rwflowpack_start' script in the /tools subdirectory?