Hello,
Just discovered LiveGraph and it looks mostly perfect for my application. I have been monitoring thermocouples and doing temperature plots. Written and played with lots of graphing software mostly under python. However, none have been really optimal for me, just a simple slow changing time series graph. What I have been doing as of late is, is just running a Gnuplot script on the data file every 3 mins or so, works, but its herky jerky and moves around the screen over time.
Anyway thanks for this, I am going to try it out, just wondering if its possible to let one of the data types in the CSV file be a time stamp. So I could plot temperature change as it relates to time. From what I can gather only strings that can be converted to integer work at the moment.
Thanks again so much for all your effort, this is a great new find, and believe me I have looked,
ED
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It truly is a perfect application for real time debugging. I export all my debug information periodically to another machine using csv format and Livegraph plotted it extremely well.
I was really impressed by the functionality provided by LiveGraph.
However I have the same problem. My X axis is Date Time series in format "11/11/1970 8:36:00 AM" which LiveGraph is unable to plot.
I look forward to the next version of LiveGraph with capability to plot Date time.
It would be a boom to real time debugging.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The problem with the time stamp is that LiveGraph needs a way of knowing whether to interpret the data values as numbers or time stamps. We would like to avoid extending the file format to include type information because we want to keep the format as simple as possible and to ensure compatibility with other CSV processing applications. We are currently thinking of ways to integrate time stamps, and we would be grateful for any suggestions from LiveGraph users! Please use this forum.
Thanks!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
A possibility to tell LiveGraph that a data column in the CSV file should be interpreted as time stamps is to introduce a new option in the transformation column in the Data Series Settings window.
The problem with this is that it would require serious changes to the program architecture:
At the moment, the data is first read into cache under the assumption that all data values are numbers (i.e. the text in the file is parsed into numbers immediately when read from the input stream). The transformations specified in the data series settings are applied at a latter stage - i.e. the metaphor is that the Data Series Settings define how to transform the data, not what the data is.
We will be grateful for any comments on this topic!
The way it looks in the moment, we will begin working on the time stamp issue in late September. Post your thought on the here.
Cheers))
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
As of versions 1.13 and 1.14 the following feature has been added:
A new type for the x-axis is available in the Graph Settings Window.
You can set LiveGraph to interpret the real values of any of the data series as seconds times 10 raised to some specified power P, where P can be any real number. For example, if P = -3, then the values of the specified series will be interpreted as seconds * 10^(-3), which means milliseconds. If you make use of that option, the labelling of the x-axis will be automatically formatted as timestamps (of the form "hh:mm:ss.xxx").
This means that you cannot put timestamps directly into the datafile. But if you convert the timestamps into seconds (or any fraction of seconds) and put the seconds into the datafile, you can easily plot against a timestamp-formatted x-axis.
Direct support of timestamps in the datafile is not planned in the near future due to serious changes it would require. However, the above approach should be suitable in most cases.
We hope that it is helpful!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That is an excellent solution. Thank you. It solves the problem nicely without complicating the interface. It took me a while to figure out that if I wanted my graph to start at 8:30 I need to start couting from 30,600, but it works as expected. Except LiveGraph says that the number is supposed to be in seconds, and I think it means miliseconds, like you state here. Once I put in the 0.001 it worked perfect. Oh, and when the scale is on the order or minutes (or longer), it would be nice to see 8:45:00 instead of 8:45:00.123. Some sort of auto-scaling of the precision displayed. Other than that, thanks again!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> That is an excellent solution. Thank you.
> It solves the problem nicely without complicating the interface.
Cheers.)
> It took me a while to figure out that if I wanted my graph to start
> at 8:30 I need to start couting from 30,600, but it works as expected.
Yes, tutorials are needed and work in in progress.
> Except LiveGraph says that the number is supposed to be in seconds,
> and I think it means miliseconds, like you state here.
> Once I put in the 0.001 it worked perfect.
Can you please clarify, I am not sure what you mean here. When you select X-axis type to be "Data series as seconds" in the Graph settings window, the "Power" input box at the bottom of that window becomes available. In there you need to put the 10-power of seconds that is used in your data file. I.e., if you use milliseconds, you need to put "-3" (since 1 millisecond = 10^-3 seconds). Did you do that?
> Oh, and when the scale is on the order or minutes (or longer),
> it would be nice to see 8:45:00 instead of 8:45:00.123.
> Some sort of auto-scaling of the precision displayed.
This is a good point. I'll discuss it with the team to see what should be the desired behaviour.
Cheers!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I tried using P= -3 & 30600 to be display the time from 8.30. However, I couldn't display the time correctly. It turned out to be 00:00:30:600. May I know where did i input wrongly? I am doing trial and error before displaying the graph based on my own data.
Seek for help
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Would this be a weird suggestion for those of us with data sets that are always generated at specific intervals?
Instead of having us massage the data in converting to the number of seconds...allow choosing "TEXT" as the format for a column. Then when we choose this column to be displayed we can hover our cursor over that flatline datapoint, and we will see what the date is for that x-axis data point.
I actually think the above suggestion will be useful in other scenarios...such as when I have a column of text error messages along with the other data. I will then be able to see these on the graph by use of datapoint selection.
An additional option of changing the X-axis label to be a Text column would be helpful for the time-date scenario as well.
Hope that makes sense?
Stuart
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hey, no suggestion is weird, certainly not yours. In fact, thank you.
There are a couple of points here:
> ...allow choosing "TEXT" as the format for a column. Then
> when we choose this column to be displayed we can hover
> our cursor over that flatline datapoint, and we will see what
> the date is for that x-axis data point.
At what coordinates would you suggest rendering each point of the "text" data line? Note that a good suggestion must be generic.
> I actually think the above suggestion will be useful in other
> scenarios...such as when I have a column of text error messages
> along with the other data. I will then be able to see these on the
> graph by use of datapoint selection.
Seems like a useful application.
> An additional option of changing the X-axis label to be a Text
> column would be helpful for the time-date scenario as well.
There is a related problem here: if you let the x-axis be read from a text data series, how would you then determine the x-coordinate values of all the data points you are rendering. You could, of course, plot against the dataset number, but see below..
The first suggestion will be difficult to realise in a general case. In many cases data points are very close to each other such that the mouse selects several of them and the feature becomes less useful.
But there are two bigger issues here:
I. The way LiveGraph is currently architectured. Currently, the data is parsed into numerical values the moment it is read and then always processed as such. Achieving the functionality you describe would be very useful, but time intensive, as it would require a lot of changes. As LiveGraph development is done by working people with families and our funding is zero, we have to be choosy about what we do implement and what we do not, and our current vision is to drive the core of V.2 to stability and to leave the beta status rather than adding new features. We aim to get to a stage where LiveGraph can work with plugins, such that the functions you are asking for can be realised more easily. Thus, although I think you request good features, I do not believe we will have time for them in the nearest future.
II. Philosophy. LiveGraph has two main objections:
- high frequency, low resource real-time plotting of very large data sets;
- simple point-and-click interface for efficient exploratory analysis of very large data sets.
As discussed above, allowing text data as x-axis would require a complication in the UI - i.e. it would have to restrict the x-axis style to dataset indices in some intuitive way. This would make using the GUI more complicated for some people. We have to ask the philosophical question - who should rather handle the complexity - the GUI user or the application developer. The LiveGraph philosophy in this case is that the complexity should be handled by the application developer.
In practical terms it means that YOU, as an application developer have some kind of code that produces a data file that contains timestamps. You call some function, get the timedate stamp and then convert it to some format before writing to a file. Basically, instead of converting it to HH:MM:SS you might just as well convert it to SSSSS. We believe that this is preferable to having a user deal with more complicated GUI settings.
There is no right or wrong here, its just a philosophy. But we are happy to discuss it if you think you have a good argument why we should change it.
Hope this helps!
Cheers..
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello,
Just discovered LiveGraph and it looks mostly perfect for my application. I have been monitoring thermocouples and doing temperature plots. Written and played with lots of graphing software mostly under python. However, none have been really optimal for me, just a simple slow changing time series graph. What I have been doing as of late is, is just running a Gnuplot script on the data file every 3 mins or so, works, but its herky jerky and moves around the screen over time.
Anyway thanks for this, I am going to try it out, just wondering if its possible to let one of the data types in the CSV file be a time stamp. So I could plot temperature change as it relates to time. From what I can gather only strings that can be converted to integer work at the moment.
Thanks again so much for all your effort, this is a great new find, and believe me I have looked,
ED
It truly is a perfect application for real time debugging. I export all my debug information periodically to another machine using csv format and Livegraph plotted it extremely well.
I was really impressed by the functionality provided by LiveGraph.
However I have the same problem. My X axis is Date Time series in format "11/11/1970 8:36:00 AM" which LiveGraph is unable to plot.
I look forward to the next version of LiveGraph with capability to plot Date time.
It would be a boom to real time debugging.
Hi there!
The problem with the time stamp is that LiveGraph needs a way of knowing whether to interpret the data values as numbers or time stamps. We would like to avoid extending the file format to include type information because we want to keep the format as simple as possible and to ensure compatibility with other CSV processing applications. We are currently thinking of ways to integrate time stamps, and we would be grateful for any suggestions from LiveGraph users! Please use this forum.
Thanks!
A possibility to tell LiveGraph that a data column in the CSV file should be interpreted as time stamps is to introduce a new option in the transformation column in the Data Series Settings window.
The problem with this is that it would require serious changes to the program architecture:
At the moment, the data is first read into cache under the assumption that all data values are numbers (i.e. the text in the file is parsed into numbers immediately when read from the input stream). The transformations specified in the data series settings are applied at a latter stage - i.e. the metaphor is that the Data Series Settings define how to transform the data, not what the data is.
We will be grateful for any comments on this topic!
The way it looks in the moment, we will begin working on the time stamp issue in late September. Post your thought on the here.
Cheers))
As of versions 1.13 and 1.14 the following feature has been added:
A new type for the x-axis is available in the Graph Settings Window.
You can set LiveGraph to interpret the real values of any of the data series as seconds times 10 raised to some specified power P, where P can be any real number. For example, if P = -3, then the values of the specified series will be interpreted as seconds * 10^(-3), which means milliseconds. If you make use of that option, the labelling of the x-axis will be automatically formatted as timestamps (of the form "hh:mm:ss.xxx").
This means that you cannot put timestamps directly into the datafile. But if you convert the timestamps into seconds (or any fraction of seconds) and put the seconds into the datafile, you can easily plot against a timestamp-formatted x-axis.
Direct support of timestamps in the datafile is not planned in the near future due to serious changes it would require. However, the above approach should be suitable in most cases.
We hope that it is helpful!
Is it not possible to use the linux time config seconds from 1972 ... ?
That is an excellent solution. Thank you. It solves the problem nicely without complicating the interface. It took me a while to figure out that if I wanted my graph to start at 8:30 I need to start couting from 30,600, but it works as expected. Except LiveGraph says that the number is supposed to be in seconds, and I think it means miliseconds, like you state here. Once I put in the 0.001 it worked perfect. Oh, and when the scale is on the order or minutes (or longer), it would be nice to see 8:45:00 instead of 8:45:00.123. Some sort of auto-scaling of the precision displayed. Other than that, thanks again!
Hi,
> That is an excellent solution. Thank you.
> It solves the problem nicely without complicating the interface.
Cheers.)
> It took me a while to figure out that if I wanted my graph to start
> at 8:30 I need to start couting from 30,600, but it works as expected.
Yes, tutorials are needed and work in in progress.
> Except LiveGraph says that the number is supposed to be in seconds,
> and I think it means miliseconds, like you state here.
> Once I put in the 0.001 it worked perfect.
Can you please clarify, I am not sure what you mean here. When you select X-axis type to be "Data series as seconds" in the Graph settings window, the "Power" input box at the bottom of that window becomes available. In there you need to put the 10-power of seconds that is used in your data file. I.e., if you use milliseconds, you need to put "-3" (since 1 millisecond = 10^-3 seconds). Did you do that?
> Oh, and when the scale is on the order or minutes (or longer),
> it would be nice to see 8:45:00 instead of 8:45:00.123.
> Some sort of auto-scaling of the precision displayed.
This is a good point. I'll discuss it with the team to see what should be the desired behaviour.
Cheers!
I tried using P= -3 & 30600 to be display the time from 8.30. However, I couldn't display the time correctly. It turned out to be 00:00:30:600. May I know where did i input wrongly? I am doing trial and error before displaying the graph based on my own data.
Seek for help
It turns out I wasn't using the tools correctly. I was entering 0.001 instead of -3. Sorry about the noise. Everything is worknig fine.
Thomas
Would this be a weird suggestion for those of us with data sets that are always generated at specific intervals?
Instead of having us massage the data in converting to the number of seconds...allow choosing "TEXT" as the format for a column. Then when we choose this column to be displayed we can hover our cursor over that flatline datapoint, and we will see what the date is for that x-axis data point.
I actually think the above suggestion will be useful in other scenarios...such as when I have a column of text error messages along with the other data. I will then be able to see these on the graph by use of datapoint selection.
An additional option of changing the X-axis label to be a Text column would be helpful for the time-date scenario as well.
Hope that makes sense?
Stuart
Hey, no suggestion is weird, certainly not yours. In fact, thank you.
There are a couple of points here:
> ...allow choosing "TEXT" as the format for a column. Then
> when we choose this column to be displayed we can hover
> our cursor over that flatline datapoint, and we will see what
> the date is for that x-axis data point.
At what coordinates would you suggest rendering each point of the "text" data line? Note that a good suggestion must be generic.
> I actually think the above suggestion will be useful in other
> scenarios...such as when I have a column of text error messages
> along with the other data. I will then be able to see these on the
> graph by use of datapoint selection.
Seems like a useful application.
> An additional option of changing the X-axis label to be a Text
> column would be helpful for the time-date scenario as well.
There is a related problem here: if you let the x-axis be read from a text data series, how would you then determine the x-coordinate values of all the data points you are rendering. You could, of course, plot against the dataset number, but see below..
The first suggestion will be difficult to realise in a general case. In many cases data points are very close to each other such that the mouse selects several of them and the feature becomes less useful.
But there are two bigger issues here:
I. The way LiveGraph is currently architectured. Currently, the data is parsed into numerical values the moment it is read and then always processed as such. Achieving the functionality you describe would be very useful, but time intensive, as it would require a lot of changes. As LiveGraph development is done by working people with families and our funding is zero, we have to be choosy about what we do implement and what we do not, and our current vision is to drive the core of V.2 to stability and to leave the beta status rather than adding new features. We aim to get to a stage where LiveGraph can work with plugins, such that the functions you are asking for can be realised more easily. Thus, although I think you request good features, I do not believe we will have time for them in the nearest future.
II. Philosophy. LiveGraph has two main objections:
- high frequency, low resource real-time plotting of very large data sets;
- simple point-and-click interface for efficient exploratory analysis of very large data sets.
As discussed above, allowing text data as x-axis would require a complication in the UI - i.e. it would have to restrict the x-axis style to dataset indices in some intuitive way. This would make using the GUI more complicated for some people. We have to ask the philosophical question - who should rather handle the complexity - the GUI user or the application developer. The LiveGraph philosophy in this case is that the complexity should be handled by the application developer.
In practical terms it means that YOU, as an application developer have some kind of code that produces a data file that contains timestamps. You call some function, get the timedate stamp and then convert it to some format before writing to a file. Basically, instead of converting it to HH:MM:SS you might just as well convert it to SSSSS. We believe that this is preferable to having a user deal with more complicated GUI settings.
There is no right or wrong here, its just a philosophy. But we are happy to discuss it if you think you have a good argument why we should change it.
Hope this helps!
Cheers..