Menu
▾
▴
metpx-commit
|
From: <nic...@us...> - 2006-09-15 15:08:04
|
Revision: 1134
http://svn.sourceforge.net/metpx/?rev=1134&view=rev
Author: nicholaslemay
Date: 2006-09-15 08:07:58 -0700 (Fri, 15 Sep 2006)
Log Message:
-----------
Modified calls to setMinMaxMedian. In some cases calls caused lost of data.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientGraphicProducer.py
trunk/sundew/lib/stats/FileStatsCollector.py
trunk/sundew/lib/stats/pickleMerging.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-09-18 19:04:20
|
Revision: 1136
http://svn.sourceforge.net/metpx/?rev=1136&view=rev
Author: nicholaslemay
Date: 2006-09-18 12:04:15 -0700 (Mon, 18 Sep 2006)
Log Message:
-----------
Added comments to PickleVersionChecker.py
Updated doc.
Modified Paths:
--------------
trunk/sundew/lib/stats/PickleVersionChecker.py
trunk/sundew/lib/stats/doc.txt
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-09-19 21:20:58
|
Revision: 1137
http://svn.sourceforge.net/metpx/?rev=1137&view=rev
Author: nicholaslemay
Date: 2006-09-19 14:20:49 -0700 (Tue, 19 Sep 2006)
Log Message:
-----------
Slight overhaul of almost all that is printed in graphics.
Modified Paths:
--------------
trunk/sundew/lib/stats/FileStatsCollector.py
trunk/sundew/lib/stats/StatsPlotter.py
trunk/sundew/lib/stats/generateGraphics.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-09-22 15:24:07
|
Revision: 1140
http://svn.sourceforge.net/metpx/?rev=1140&view=rev
Author: nicholaslemay
Date: 2006-09-22 08:23:59 -0700 (Fri, 22 Sep 2006)
Log Message:
-----------
- Completly modified generateAllGraphsForServer.py's structure. Is now much more modular.
- Now supports simultaneously ran process'.
Program now forks a different child process for every single graphic that needs to be produced.
Greatly improved speed, although it might slow down everything else being ran on the machine at the same time...
(Might need to investigate further)
- Added -i,--individual option that allows users to create graphs over different machine without combining
the data of each machine.
-Completly modified PickleVersionChecker approach.
Former version (md5sum) could not work with a large number of pickle files since parameter input is too limited.
Former version could not work with multi process since only one file was used, and needed to be written on by all process.
Former version created even more process by calling ms5sum,wich would probably slow down other user's process' even more.
New version implements the following :
It only verifies last modification date by using python only( = less process')
Files are looked up depending on client and machine(s) currently handled( = less lookup )
Each user/client couple now has it' own file containing dictionnaries that contain the info that pertain to their files only.
( Allows for all process to acces file version without collisions.)
Modified Paths:
--------------
trunk/sundew/lib/stats/PickleVersionChecker.py
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/pickleMerging.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-09-22 19:45:16
|
Revision: 1141
http://svn.sourceforge.net/metpx/?rev=1141&view=rev
Author: nicholaslemay
Date: 2006-09-22 12:45:10 -0700 (Fri, 22 Sep 2006)
Log Message:
-----------
Minor change.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientStatsPickler.py
trunk/sundew/lib/stats/pickleMerging.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-09-25 17:27:04
|
Revision: 1145
http://svn.sourceforge.net/metpx/?rev=1145&view=rev
Author: nicholaslemay
Date: 2006-09-25 10:26:53 -0700 (Mon, 25 Sep 2006)
Log Message:
-----------
Corrected small bug stemming from the last commit's changes.
Modified graphics labels so that long file names are no longer cut out.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientStatsPickler.py
trunk/sundew/lib/stats/StatsPlotter.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-09-28 19:11:26
|
Revision: 1151
http://svn.sourceforge.net/metpx/?rev=1151&view=rev
Author: nicholaslemay
Date: 2006-09-28 12:11:13 -0700 (Thu, 28 Sep 2006)
Log Message:
-----------
Added rsync for .conf files to FileStatsCollector.py.
Added --delete options for rsync.
FileStatsCollector logger is now info,not debug.
Modified Paths:
--------------
trunk/sundew/lib/stats/FileStatsCollector.py
trunk/sundew/lib/stats/doc.txt
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/pickleUpdater.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-02 17:13:50
|
Revision: 1157
http://svn.sourceforge.net/metpx/?rev=1157&view=rev
Author: nicholaslemay
Date: 2006-10-02 10:13:40 -0700 (Mon, 02 Oct 2006)
Log Message:
-----------
Minor changes.
Modified Paths:
--------------
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/pickleMerging.py
trunk/sundew/lib/stats/pickleUpdater.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-04 17:57:43
|
Revision: 1165
http://svn.sourceforge.net/metpx/?rev=1165&view=rev
Author: nicholaslemay
Date: 2006-10-04 10:57:35 -0700 (Wed, 04 Oct 2006)
Log Message:
-----------
Added first versions of stats/generateRRDGraphics.py and stats/transferPickleToRRD.py to library.
transferPickleToRRD.py is to be used as a complement to the current data pickling system. It will allow to preserve
pickled data in a space economical round robin database. It will also allow us to have quick access to the consolidated data of the last year.
( Could be modified to keep data for even longer, say 5 or 10 years if wanted. )
generateRRDGraphics.py is to be used to generate graphs based on the data saved in the rrd database. It is very much similar
to generateGraphics.py that's currently in place and that's using gnuplot to plot graphics. It does, however, add the possibility to use the efficient
rrd database instead of the pickle system that was in place. In turn, this permits to easily plot graphs for the past week or the past year.
Modified Paths:
--------------
trunk/sundew/lib/stats/pickleMerging.py
Added Paths:
-----------
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/transferPickleToRRD.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-05 19:35:38
|
Revision: 1175
http://svn.sourceforge.net/metpx/?rev=1175&view=rev
Author: nicholaslemay
Date: 2006-10-05 12:35:33 -0700 (Thu, 05 Oct 2006)
Log Message:
-----------
- Changed graphic output slightly.
- Changed list of defaults graphics being generated.
- Added get/set DatabaseTimeOfUpdate. Helpfull to verify if database can be updated according
to the parameters used. Times are to be stored in a pickle files named PXPaths.STATS/DATABASE-UPDATES/fileType/client_machine(s).
Each fileType/machine/client combo had to have its own file since the updates are made using multiple process wich would make
conflicting file access if only one file were used.
- Removed startTime option.
- Default startTime is now the time of the last update.
- For the first call, when no prior supdate times exist, startime will be 24 hours prior to call.
Modified Paths:
--------------
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/pickleUpdater.py
trunk/sundew/lib/stats/transferPickleToRRD.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-10 13:58:31
|
Revision: 1192
http://svn.sourceforge.net/metpx/?rev=1192&view=rev
Author: nicholaslemay
Date: 2006-10-10 06:58:19 -0700 (Tue, 10 Oct 2006)
Log Message:
-----------
Made a quick verification of each of those files' comments to make sure everything seemed up to date.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientGraphicProducer.py
trunk/sundew/lib/stats/ClientStatsPickler.py
trunk/sundew/lib/stats/FileStatsCollector.py
trunk/sundew/lib/stats/PickleVersionChecker.py
trunk/sundew/lib/stats/backwardReader.py
trunk/sundew/lib/stats/cpickleWrapper.py
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/pickleMerging.py
trunk/sundew/lib/stats/pickleUpdater.py
trunk/sundew/lib/stats/pickleViewer.py
trunk/sundew/lib/stats/pickledTimesViewer.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-10 15:48:21
|
Revision: 1193
http://svn.sourceforge.net/metpx/?rev=1193&view=rev
Author: nicholaslemay
Date: 2006-10-10 08:48:04 -0700 (Tue, 10 Oct 2006)
Log Message:
-----------
As a safety measure, added folder creation when it wasn't previously created, for every logger present in the library.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientGraphicProducer.py
trunk/sundew/lib/stats/ClientStatsPickler.py
trunk/sundew/lib/stats/DirectoryFileCollector.py
trunk/sundew/lib/stats/FileStatsCollector.py
trunk/sundew/lib/stats/StatsPlotter.py
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/generateGraphics.py
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/pickleMerging.py
trunk/sundew/lib/stats/pickleSynchroniser.py
trunk/sundew/lib/stats/pickleUpdater.py
trunk/sundew/lib/stats/transferPickleToRRD.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-10 20:05:25
|
Revision: 1196
http://svn.sourceforge.net/metpx/?rev=1196&view=rev
Author: nicholaslemay
Date: 2006-10-10 13:05:18 -0700 (Tue, 10 Oct 2006)
Log Message:
-----------
- All loggers now use the new bytes = True options.
These logs need a size based turning option. They don't need
a time-based turning system.
- Added exception trapping/logging in generateRRDGraphics.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientGraphicProducer.py
trunk/sundew/lib/stats/ClientStatsPickler.py
trunk/sundew/lib/stats/DirectoryFileCollector.py
trunk/sundew/lib/stats/FileStatsCollector.py
trunk/sundew/lib/stats/StatsPlotter.py
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/pickleSynchroniser.py
trunk/sundew/lib/stats/pickleUpdater.py
trunk/sundew/lib/stats/transferPickleToRRD.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-11 14:15:02
|
Revision: 1198
http://svn.sourceforge.net/metpx/?rev=1198&view=rev
Author: nicholaslemay
Date: 2006-10-11 07:14:58 -0700 (Wed, 11 Oct 2006)
Log Message:
-----------
Added symnbolic link creation option to StatsPlotter.
Added support for this option to related files.
generateAllGraphsForServer will create links by default since it's the program
set in a crontab to create graphics at a regular interval.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientGraphicProducer.py
trunk/sundew/lib/stats/StatsPlotter.py
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/generateGraphics.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-12 13:59:26
|
Revision: 1208
http://svn.sourceforge.net/metpx/?rev=1208&view=rev
Author: nicholaslemay
Date: 2006-10-12 06:59:20 -0700 (Thu, 12 Oct 2006)
Log Message:
-----------
- Rx graphics now have rx in their filename instead of tx. buildImageName method always used the default tx
value since there was no fileType specified in calls to the StatsPlotter constructor coming from
ClientGraphicProducer.
- Added all tx or rx names as default value for clientNames in generateRRdGraphics.py
- Modified graphics size, rx graphs were missing part of their titles.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientGraphicProducer.py
trunk/sundew/lib/stats/StatsPlotter.py
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/pickleUpdater.py
trunk/sundew/lib/stats/transferPickleToRRD.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-12 17:41:02
|
Revision: 1211
http://svn.sourceforge.net/metpx/?rev=1211&view=rev
Author: nicholaslemay
Date: 2006-10-12 10:40:54 -0700 (Thu, 12 Oct 2006)
Log Message:
-----------
Added --delete-before options to rsync commands being done in pickleSynchroniser.
Modified Paths:
--------------
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/pickleSynchroniser.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-13 14:16:08
|
Revision: 1215
http://svn.sourceforge.net/metpx/?rev=1215&view=rev
Author: nicholaslemay
Date: 2006-10-13 07:15:59 -0700 (Fri, 13 Oct 2006)
Log Message:
-----------
Fixed bugs caused by recent changes.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientGraphicProducer.py
trunk/sundew/lib/stats/StatsPlotter.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-13 19:14:30
|
Revision: 1219
http://svn.sourceforge.net/metpx/?rev=1219&view=rev
Author: nicholaslemay
Date: 2006-10-13 12:14:25 -0700 (Fri, 13 Oct 2006)
Log Message:
-----------
- Removed useless print out in MyDateLib
- Now uses bytecount total instead of means in graphic production and in transfer from pickle to rrd database.
- Added a few important things to generateRRDGraphics.py :
- Added --yearly --weekly --daily options so we can easily set em in crontabs.
- Added default ALL client options. It will now be possible to create all the needed graphs
for all the existing client of a specific machine akin to what generateAllGraphics.py does with gnuplot.
- Colors now reflect those that have been used in gnuplot. Latency=red, green=errors, bytecount=blue.
Modified Paths:
--------------
trunk/sundew/lib/stats/MyDateLib.py
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/transferPickleToRRD.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-18 17:43:24
|
Revision: 1233
http://svn.sourceforge.net/metpx/?rev=1233&view=rev
Author: nicholaslemay
Date: 2006-10-18 10:43:18 -0700 (Wed, 18 Oct 2006)
Log Message:
-----------
Modified output for graphics.
Total bytes per seconds is now displayed either in bytes KiloBytes,Megs or Giga
Added more sapces between graphs.
Default graph width is now 24 hours.
Modified Paths:
--------------
trunk/sundew/lib/stats/StatsPlotter.py
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/generateGraphics.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-20 19:29:34
|
Revision: 1235
http://svn.sourceforge.net/metpx/?rev=1235&view=rev
Author: nicholaslemay
Date: 2006-10-20 12:29:10 -0700 (Fri, 20 Oct 2006)
Log Message:
-----------
Removed all workarounds that were implemented as to be able to run program with the use of mirror machines.
Completly modified lanchGraphCreation so that it now reads a small test file to set up its parameters and that the executed commands
no longer require mirror machines.
Added parameter handling to picklecleaner as to be able to specify a certain number of days of files to keep.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientGraphicProducer.py
trunk/sundew/lib/stats/ClientStatsPickler.py
trunk/sundew/lib/stats/DirectoryFileCollector.py
trunk/sundew/lib/stats/FileStatsCollector.py
trunk/sundew/lib/stats/StatsPlotter.py
trunk/sundew/lib/stats/doc.txt
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/generateGraphics.py
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/pickleCleaner.py
trunk/sundew/lib/stats/pickleSynchroniser.py
trunk/sundew/lib/stats/pickleUpdater.py
trunk/sundew/lib/stats/transferPickleToRRD.py
Added Paths:
-----------
trunk/sundew/lib/stats/launchGraphCreation.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-10-27 19:22:13
|
Revision: 1255
http://svn.sourceforge.net/metpx/?rev=1255&view=rev
Author: nicholaslemay
Date: 2006-10-27 12:22:03 -0700 (Fri, 27 Oct 2006)
Log Message:
-----------
Added global workarounds. This will allow all machines to install the stats
library without having numerous version.
Modified Paths:
--------------
trunk/sundew/lib/stats/ClientGraphicProducer.py
trunk/sundew/lib/stats/ClientStatsPickler.py
trunk/sundew/lib/stats/DirectoryFileCollector.py
trunk/sundew/lib/stats/StatsPlotter.py
trunk/sundew/lib/stats/generateAllGraphsForServer.py
trunk/sundew/lib/stats/generateGraphics.py
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/howTo.txt
trunk/sundew/lib/stats/launchGraphCreation.py
trunk/sundew/lib/stats/pickleSynchroniser.py
trunk/sundew/lib/stats/pickleUpdater.py
trunk/sundew/lib/stats/transferPickleToRRD.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-11-01 21:30:42
|
Revision: 1259
http://svn.sourceforge.net/metpx/?rev=1259&view=rev
Author: nicholaslemay
Date: 2006-11-01 13:30:28 -0800 (Wed, 01 Nov 2006)
Log Message:
-----------
Modified RRAs to make it easier to draw proper graphics when using totals.
Added handling of filecount graphs.
Databases are now forcefully created at with a starting hour of 00:00:00 of the day of the creation to make
sure RRA's work as planned.
Modified graphic labels. Modified min max mean so they now reflect the ones being used with Gnuplot graphics.
Modified file naming.
Lowered number of simultaneously ran process' as to lower machine laods.
Removed numerous bugs.
Modified Paths:
--------------
trunk/sundew/lib/stats/generateRRDGraphics.py
trunk/sundew/lib/stats/transferPickleToRRD.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-11-06 20:03:54
|
Revision: 1264
http://svn.sourceforge.net/metpx/?rev=1264&view=rev
Author: nicholaslemay
Date: 2006-11-06 12:03:37 -0800 (Mon, 06 Nov 2006)
Log Message:
-----------
Brought all 3 doc files up to date.
Modified Paths:
--------------
trunk/sundew/lib/stats/doc.txt
trunk/sundew/lib/stats/howTo.txt
trunk/sundew/lib/stats/rrdToolDoc.txt
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-11-07 16:19:18
|
Revision: 1270
http://svn.sourceforge.net/metpx/?rev=1270&view=rev
Author: nicholaslemay
Date: 2006-11-07 08:19:13 -0800 (Tue, 07 Nov 2006)
Log Message:
-----------
Corrected error in database feeding method.
- Found that data drawn was always a minute too old. Data feeding is not handled
the same way as gnuplot. Ex: RRD draws the couple (1,8) between 0 and 1 instead of between
1 and 2.
Modified doc to warn about this behaviour.
Modified Paths:
--------------
trunk/sundew/lib/stats/rrdToolDoc.txt
trunk/sundew/lib/stats/transferPickleToRRD.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
|
From: <nic...@us...> - 2006-11-10 15:55:33
|
Revision: 1285
http://svn.sourceforge.net/metpx/?rev=1285&view=rev
Author: nicholaslemay
Date: 2006-11-10 07:55:30 -0800 (Fri, 10 Nov 2006)
Log Message:
-----------
Modified paths used for both utilities.
Source used when restoring database is no longer moved and is now copied.
Limit of existings backups was raised.
Modified Paths:
--------------
trunk/sundew/lib/stats/backupRRDDatabases.py
trunk/sundew/lib/stats/restoreRoundRobinDatabases.py
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
Thanks for helping keep SourceForge clean.
X