Ubuntu 14.04LTS Server - Conversion from local time to UTC

[John Feeney]

I am having trouble with the time conversions on the FlowGrapher. I imagine that it is also happening with the text reports as well.

I am in the PST time zone (UTC-8 hours). SiLK is capturing data in UTC. When I fill out the report form, I have to convert PST time to UTC and enter that. What I get back is even worse, it gives the graph times as PST but there is a 16 hour difference. The graph shows +16 hours from what I put in the FlowGrapher report form.

So if I am looking for data collected at 5pm on Jan 26 PST, I have to convert to UTC and get 1am Jan 27 UTC. I put that in the report form, that is supposed to use PST, and I get the data I was looking for. The graph shows "Time: PST" but all the hours are 16 hours ahead of what the report says it is above in the report header.

I have looked at the $time_zone in the FlowViewer_Configuration.pm but according to the docs when this variable is blank, it uses the system time zone. It does get the PST from the system. It just does not seem to be doing the conversion correctly.

Note: The docs do say that it gets the system time zone from the "timelocal" function and Ubuntu does not seem to have this. FlowViewer does seem to figure out that it is in the PST time zone anyway.

I know it is off by a multiple of my time zone offset but I still am not quite sure what is going on here.

[Joe Loiacono]

Hi John,

Sorry about this. What version of FlowViewer are you using?

When you say "but all the hours are 16 hours ahead of what the report says" ... are you referring to the times for the individual flows listed beneath the graph?

Joe

[Joe Loiacono]

John - thanks for your help. I sent a direct email reply (but didn't have your email address). In case it doesn't make it ...

I've attached a FlowGrapher_Main.cgi.jf file. Could you save off the existing one, use this one (renaming it to drop off the suffix), make a run and forward the DEBUG_GRAPHER file?

Also - could you input the times you are interested in in PST (i.e., Jan. 27 5pm to 6 pm PST)? It should be converting these internally and I want to see why it isn't.

Thanks,

Joe

[John Feeney]

Thanks Joe,

Sorry for missing the version. I am using 4.5.

No, I was not talking about the individual flows listed beneath. I was refering to the time scale across the bottom of the graph. Sorry I was a little vague on terms. See the attached file in the first post. It is a screen shot of the graph.

In reference to the individual flows at the bottom of the graph, they seem to reflect what was put in the report form, the PST plus 8 hours.

[Joe Loiacono]

Can you do another run and post the $work_directory/DEBUG_GRAPHER file (sanitized if you like).

Also - could you show a few of the detail lines below the graph?

Thanks,

Joe

[John Feeney]

Ok. This is data from Jan 27 5pm to 6pm PST.

In FlowGrapher_Main.cgi
Using GDBM
Using NDBM
FORM{device_name}: Main
FORM{exporter}:
FORM{start_date}: 01/28/2015 start_date: 01/28/2015 FORM{end_date}: 01/28/2015 end_date: 01/28/2015
This device is exporting IPFIX
current_year_date: 01/01/2015 current_year_epoch: 1420070400 current_year_dst: 0
prior_year_date: 01/01/2014 prior_year_epoch: 1388534400 prior_year_dst: 0
next_year_date: 01/01/2016 next_year_epoch: 1451606400 next_year_dst: 0
Current Year: 01/01/2015 Epoch: 1420070400 DST: 0
Start Date: 01/28/2015 Epoch: 1422406800 DST: 0
selection_switches: --data-rootdir=/data --type=all --start-date=2015/01/27:22 --end-date=2015/01/28:02 --active=2015/01/28:01:00:00-2015/01/28:02:00:00
partitioning_switches:
from: start to: start filter_SiLK elapsed seconds: 0.005718 running: 0.005718
rwfilter_command: /usr/local/bin/rwfilter --site-config-file=/data/silk.conf --data-rootdir=/data --type=all --start-date=2015/01/27:22 --end-date=2015/01/28:02 --active=2015/01/28:01:00:00-2015/01/28:02:00:00 --pass=/usr/lib/cgi-bin/FlowViewer_4.5/Flow_Working/FlowGrapher_filtered_170607MM
from: start filter_SiLK to: start rwcount_SiLK elapsed seconds: 0.012036 running: 0.017754
silk_command: /usr/local/bin/rwcount --site-config-file=/data/silk.conf --bin-size=5 --start-time=2015/01/28:01:00:00 --epoch-slots /usr/lib/cgi-bin/FlowViewer_4.5/Flow_Working/FlowGrapher_filtered_170607MM > /usr/lib/cgi-bin/FlowViewer_4.5/Flow_Working/FlowGrapher_output_170607MM
from: start rwcount_SiLK to: start report_SiLK elapsed seconds: 0.004760 running: 0.022514
from: start report_SiLK to: start rwcut_SiLK elapsed seconds: 0.001153 running: 0.023667
from: start rwcut_SiLK to: start records_SiLK elapsed seconds: 0.005402 running: 0.029069
looked at: flows
passed: flows
from: start records_SiLK to: done_FLOWS elapsed seconds: 0.007635 running: 0.036704
from: done_FLOWS to: create_graph elapsed seconds: 0.020483 running: 0.057187
filter_hash: FG_FlowGrapher_save_170607MM filter_source: FG filter_source_file: /usr/lib/cgi-bin/FlowViewer_4.5/Flow_Working/FlowGrapher_save_170607MM
from: create_graph to: done elapsed seconds: 1.364145 running: 1.421332
run took: 2 seconds

[Joe Loiacono]

Thanks for your help John.

I've attached a FlowGrapher_Main.cgi.jf file. Could you save off the
existing one, use this one (renaming it to drop off the suffix), make a run
and forward the DEBUG_GRAPHER file?

Also - could you input the times you are interested in in PST (i.e., Jan.
27 5pm to 6 pm PST)? It should be converting these internally and I want to
see why it isn't.

Thanks,

Joe

[John Feeney]

Ok. I put in 4-5PM PST.

[Joe Loiacono]

John - here's the new version. Please use FlowGrapher_Main.cgi.jf2 and not FlowGrapher_Main.cgi.jf

Thanks for your help ...

Joe

[Joe Loiacono]

John,

Please try the following new scripts:

FlowGrapher_Main.cgi
FlowGrapher_Analyze.cgi
FlowViewer_Main.cgi

Thanks,

Joe

(now let's watch Seattle whup new England :-)

[John Feeney]

Joe,

I watched it with my brother. He is more of the NFL fan. I just like hanging out and drinking beer. It was an exciting last few minutes though. Sorry about Seattle. ;)

Sorry about the problems I have been sending your way. I really appreciate the time you are putting in. It is a cool program. I would eventually like to present it to my local Linux Users Group in OC CA.

I took the default time, about one hour behind the current time, and there was no data. Anyway, now I have to subtract 8 hrs. Here is the DEBUG_GRAPHER file. what I put in was 2:30 am to 3:30 am but the actual time is 11:24 am PST. So I Figured that where the data ends in the graph is about where it last collected something, at around 11:20 am.

It does keep collecting but I checked the /data directory and only one sensor seems to be collecting since the 29th. I have to check into that.

I have two active sensors, Main and MeshBridge, but only MeshBridge seems to be collecting since the 29th. But running the Graph for the Main sensor seems to get the MeshBridge data. That issue can wait but I just mention it in case it can be related.