Activity for collectl
-
Laurence Oberman modified a comment on discussion Open Discussion
Happy New Year Please note, there will no longer be version updates on Sourceforge. All updates will be github only git clone https://github.com/sharkcz/collectl.git For problem reports please email loberman@redhat.com. Include a full description of the issue. Thanks Laurence
-
Happy New Year Please not, there will no longer be version updates on Sourceforge. All updates will be github only git clone https://github.com/sharkcz/collectl.git For problem reports please email loberman@redhat.com. Include a full description of the issue. Thanks Laurence
-
Hello Indeed if you have been watching sourceforge, I tried to keep it updated in multiple places but now have started primarily updating on github. git clone https://github.com/sharkcz/collectl.git cd coll* ./INSTALL For RHEL7 and higher: systemctl enable collectl systemctl start collectl Note that I do not have time to test on all different distros so if anything comes up let me know on Sourceforge or email me here. I still watch Sourceforge for problem reports. Regards Laurence Oberman On Tue,...
-
Hello Indeed if you have been watching sourceforge, I tried to keep it updated in multiple places but now have started primarily updating on github. git clone https://github.com/sharkcz/collectl.git cd coll* ./INSTALL For RHEL7 and higher: systemctl enable collectl systemctl start collectl Note that I do not have time to test on all different distros so if anything comes up let me know on Sourceforge or email me here. I still watch Sourceforge for problem reports. Regards Laurence Oberman
-
Mark Seger posted a comment on discussion Open Discussion
collectl has been moved to github since I retired and is being supported by Laurence Oberman at Redhat. CCing him. it's been a great run developing/supporting collectl especially in the high performance computing domain, but it was tie to hang up my keyboard and move on -mark On Fri, May 26, 2023 at 2:03 PM Ben Fulton benfulton10@users.sourceforge.net wrote: Hi, It looks like this project is being supported again? Is there any kind of roadmap or documentation update for it? The website says the last...
-
Ben Fulton posted a comment on discussion Open Discussion
Hi, It looks like this project is being supported again? Is there any kind of roadmap or documentation update for it? The website says the last update was in 2018.
-
windy posted a comment on discussion Open Discussion
thank you. I use it that works well on 4.3.8 with lustre 2.10.2 . will this pacth add to master version?
-
collectl released /collectl/collectl-4.3.8/collectl-4.3.8.src.tar.gz
-
Note that it is as always, also available to be cloned here https://github.com/sharkcz/collectl.git
-
Added rbd devices Added additional network devices names Updated version number (forgot to do that on the last release)
-
collectl-4.3.7.src.tar.gz Released with an fix to include nvme devices that have native multipathing. commit cd8a94b61f5700945af8f4e6a3fcd0e4ea18f1e2 (HEAD -> master, origin/master, origin/HEAD) Author: Laurence Oberman loberman@redhat.com Date: Wed Jan 25 17:21:32 2023 -0500 Add the nvme native multipath devices filter to the collectl.conf "nvme\d+c\d+" Also available here got clone https://github.com/sharkcz/collectl.git
-
-
Hmm, if your goal is to populate a database through multiple passes, can't you just tell collectl to generate plotable output? Then you have everying in formatted text output space separated so easily parsed. No need to convert anything after it runs. Maybe -mark On Fri, Dec 2, 2022, 11:02 AM Bill Torpey wallstprog@users.sourceforge.net wrote: On Dec 2, 2022, at 10:26 AM, Mark Seger markseger@users.sourceforge.net wrote: I think you may be confused about collectl data collection. When you collect...
-
Bill Torpey posted a comment on discussion Help
On Dec 2, 2022, at 10:26 AM, Mark Seger markseger@users.sourceforge.net wrote: I think you may be confused about collectl data collection. When you collect process data your get a lot more than is displayed. You can then play it back multiple times and choose to display different process options from same collection, see --helpsuboopts. Also, no need to run once to collectl process data and a second time for system disk or other data. Just collectl it all in one go and play back with --vmstat or...
-
Mark Bill has a special use case and he knows about the collectl service for sure that gathers everything. He is scripting this so I think you misunderstood. Regards Laurence On Fri, Dec 2, 2022 at 10:26 AM Mark Seger markseger@users.sourceforge.net wrote: I think you may be confused about collectl data collection. When you collect process data your get a lot more than is displayed. You can then play it back multiple times and choose to display different process options from same collection, see...
-
I think you may be confused about collectl data collection. When you collect process data your get a lot more than is displayed. You can then play it back multiple times and choose to display different process options from same collection, see --helpsuboopts. Also, no need to run once to collectl process data and a second time for system disk or other data. Just collectl it all in one go and play back with --vmstat or -sd or -sz. I was very careful to design collectl so you could collect everything...
-
Sorry if that wasn't clear: we run collectl once to get per-process cpu utilization (--procopts kw4096 -sZ -oDG), a second time to get per-process memory utilization (--procopts mkw4096 -sZ -oDG), a third time to get system-wide network utilization (-sN -oD), and fourth to get vmstat (--export vmstat -oD). All of that is from the same .raw file(s). Again, we did it that way just so we could run both collectl and our old monitoring tools and compare like-for-like. We'll be changing that to generate...
-
Mark Thank you, but things have evolved, we have huge systems with many CPUS and many disks and the GZ files are huge The decompress instream is slow. I think it's been so long for you, that you have not seen the growth. I have data from customers for example 4GB in size for a compressed raw file and like 20GB or mor uncompressed Regards Laurence On Thu, Dec 1, 2022 at 11:44 AM Mark Seger markseger@users.sourceforge.net wrote: bill - did I read it right when you said you run collectl once for each...
-
bill - did I read it right when you said you run collectl once for each cpu? By default collectl ALWAYS records data for all CPUs. laurence - maybe you already know this but if you're looking at disks stats ordered by disk, colmux will to that in real time and if on multiple systems will show you the busiest disks across all your systems. bill, if you're finding things go faster by first decompressing your files I suspect something may be wrong. I used to decompress before I found I could give gnuplot...
-
When I get data I use this baseline script to get text tables and sort them etc. !/bin/bash echo Working on $1 collectl -scmnd -oT -p $1 > scmnd.$1 collectl -sc --verbose -oT -p $1 > sc-verbose.$1 collectl -sD -oT -p $1 > sD.$1 collectl -sZ -oT -p $1 > sZ.$1 The graphing tool produces these graphs On Thu, Dec 1, 2022 at 10:55 AM Bill Torpey wallstprog@users.sourceforge.net wrote: Thanks Laurence, I'll check that out. FWIW, attached is the kind of output we get from vmstat. Attachments: vmstat.png...
-
Thanks Laurence, I'll check that out. FWIW, attached is the kind of output we get from vmstat.
-
Bill I use collectl -sD to get all the per disk detailed I/O related stuff and I work deeply in storage performance at Red Hat. You don't need to use the vmstat stuff. My tool created tables like this TOP 50 Disks ordered by IO/S per second Average dm-114-Writes 556.033453 dm-109-Writes 540.127098 dm-113-Writes 540.126867 dm-103-Writes 478.489872 dm-99-Writes 478.481306 dm-53-Writes 317.075587 dm-29-Writes 315.691747 dm-111-Writes 272.861905 dm-107-Writes 270.897558 dm-67-Writes 253.888760 dm-63-Writes...
-
What really speeds things up is uncompressing the files before parsing them. I never parse as .gz. My tool uses the plot format too. I will see if I can sanitize it to share it. The thing is it uses numpy, scipy, pandas and matplotlib because it also does stats analysis On Thu, Dec 1, 2022 at 10:36 AM Bill Torpey wallstprog@users.sourceforge.net wrote: Hi Mark (and Laurence): First off, thanks for all the help, and for a great tool! Please see replies inline below: Bill - I guess the one overall...
-
Hi Mark (and Laurence): First off, thanks for all the help, and for a great tool! Please see replies inline below: Bill - I guess the one overall question I'd ask is about your general use of collectl and why you have a handful of smallisk files you need to play back together and also way none of your files are compressed? We've recently switched to collectl for monitoring our QA performance testing environment, with a view to rolling out collectl in production. collectl is replacing a bunch of custom...
-
My maintenance responsibilities: Fix reported bugs Adapt collectl as the kernel evolves the data structures. Enhance where needed, for example just last week made ceph rbd device capture a default So far kernel updates have not broken collectl and even the latest Fedora is seamless. I hope that is in sync with what you Mark wanted when I took over maintenance. Regards Laurence On Thu, Dec 1, 2022 at 9:17 AM Laurence Oberman loberman@users.sourceforge.net wrote: In case you missed it you can now use...
-
In case you missed it you can now use git clone https://github.com/sharkcz/collectl.git then git pull to get updates The last couple of updates commit 2001864971f91999216daf091529f5aa47d69150 (HEAD -> master, origin/master, origin/HEAD) Author: Laurence Oberman loberman@redhat.com Date: Tue Nov 29 18:29:26 2022 -0500 Version 4.3.6 with fix in sub vmstatInit where error check was removed commit 625e2453842121519e9e5a38c283d02e72e2ed89 Author: Laurence Oberman loberman@redhat.com Date: Tue Nov 29 18:19:44...
-
Here at Red Hat our use case is two fold.. If a customer reports a performance problem or any issue needing a deep dive, Install collect, let it gather and then investigate various performance problems after the customer reports the timestamps. The historical capture is very important. We also use it live if needed to do monitoring on a live performance problem. collectl -scmnd -oT -i1 used a lot collectl -sc --verbose -oT -i1 also used a lot collectl -sD -oT -i1 --dskfilt xxxx I have also been using...
-
Bill - I guess the one overall question I'd ask is about your general use of collectl and why you have a handful of smallisk files you need to play back together and also way none of your files are compressed? I had designed collectl to very efficient in terms of cpu and storage. When I installed collectl on any system, I'd just turn it on and leave it running forever. For that reason, files are automatically purged after a week (configurable of course). When I want to diagnose a problem, I'd first...
-
Released /collectl/collectl-4.3.6/collectl-4.3.6.src.tar.gz 18 minutes ago
-
-
Right -- you're just removing the check for "-s". I've been running with that patch in my local copy for a week or so and have not seen any problems. Looks good to me! and Thanks!
-
Here is the commit message in my local tree. commit 625e2453842121519e9e5a38c283d02e72e2ed89 (HEAD -> master) Author: Laurence Oberman loberman@redhat.com Date: Tue Nov 29 18:19:44 2022 -0500 Removed the error check in sub vmstatInit It was erroneous and that option is seldom used. Reported-by: Bill Torpey On Tue, Nov 29, 2022 at 6:19 PM Laurence Oberman loberman@users.sourceforge.net wrote: Seems it is going to always be called and will check user switches. It's always going to log that message...
-
Seems it is going to always be called and will check user switches. It's always going to log that message as the test is straight forward because $userSubsys will not be empty Example here with DEBUG and why it happens [loberman@lobep17 collectl]$ time /usr/bin/collectl -p '.raw.gz' --export vmstat -oD 2>&1 | tail 20221018 23:58:50 1 0 2304K 5843M 6668K 98859M 88006M 9417M 0 0 0 15 1055 1848 2 0 97 0 20221018 23:59:00 1 0 2304K 5844M 6668K 98859M 88006M 9417M 0 0 0 34 1027 1852 2 0 97 0 20221018...
-
[loberman@lobep17 collectl]$ time /usr/bin/collectl -p '*.raw.gz' --export vmstat -oD 2>&1 | tail 20221018 23:58:40 0 0 2304K 5844M 6668K 98859M 88006M 9417M 0 0 0 32 1159 1953 2 0 97 0 20221018 23:58:50 1 0 2304K 5843M 6668K 98859M 88006M 9417M 0 0 0 15 1055 1848 2 0 97 0 20221018 23:59:00 1 0 2304K 5844M 6668K 98859M 88006M 9417M 0 0 0 34 1027 1852 2 0 97 0 20221018 23:59:10 1 0 2304K 5828M 6668K 98874M 88006M 9417M 0 0 0 79 1071 1831 2 0 97 0 20221018 23:59:20 0 0 2304K 5830M 6668K 98874M 88006M...
-
Thanks Bill I will reproduce this and fix this. If I have any issues reproducing will come back to the thread. Your Reproducer noted [btorpey@bt-brix7 collectl]$ ls -lh *.raw -rw-r--r-- 1 btorpey shared 102M Nov 22 09:36 csappia01-20221110-223734.raw -rw-r--r-- 1 btorpey shared 2.0G Nov 22 09:36 csappia01-20221111-000000.raw -rw-r--r-- 1 btorpey shared 2.0G Nov 22 09:36 csappia01-20221112-000000.raw -rw-r--r-- 1 btorpey shared 1.1G Nov 22 09:36 csappia01-20221113-000000.raw The result of running...
-
Here's the directory listing: [btorpey@bt-brix7 collectl]$ ls -lh *.raw -rw-r--r-- 1 btorpey shared 102M Nov 22 09:36 csappia01-20221110-223734.raw -rw-r--r-- 1 btorpey shared 2.0G Nov 22 09:36 csappia01-20221111-000000.raw -rw-r--r-- 1 btorpey shared 2.0G Nov 22 09:36 csappia01-20221112-000000.raw -rw-r--r-- 1 btorpey shared 1.1G Nov 22 09:36 csappia01-20221113-000000.raw The result of running the system collectl: [btorpey@bt-brix7 collectl]$ time /usr/bin/collectl -p '*.raw' --export vmstat -oD...
-
It would help to see the exact command in full. If memory serves, vmstat tries to force its own values for vmstat as it requires cpu, memory and disk data (I think) -mark On Tue, Nov 29, 2022, 2:36 PM Bill Torpey wallstprog@users.sourceforge.net wrote: Hi -- I'm back... Have run some tests, and my results seem to indicate that: Without quotes, wildcarding the "-p" param results in only the first file being processed (bash guarantees that filenames are returned in alpha order https://serverfault.com/questions/122737/in-bash-are-wildcard-expansions-guaranteed-to-be-in-order...
-
Hi -- I'm back... Have run some tests, and my results seem to indicate that: Without quotes, wildcarding the "-p" param results in only the first file being processed (bash guarantees that filenames are returned in alpha order) With quotes, collectl processes all files, subject to "--from" parameter It doesn't matter whether you use single or double quotes Presence/absence of "--from" parameter makes no difference. Except with "--export vmstat": collectl will NOT process multiple files -- at the...
-
Hi -- I'm back... Have run some tests, and my results seem to indicate that: Without quotes, wildcarding the "-p" param results in only the first file being processed (bash guarantees that filenames are returned in alpha order With quotes, collectl processes all files, subject to "--from" parameter It doesn't matter whether you use single or double quotes Presence/absence of "--from" parameter makes no difference. Except with "--export vmstat": collectl will NOT process multiple files -- at the end...
-
To be fair I didn't expect everyone to read the docs :) -mark On Wed, Nov 16, 2022, 12:01 PM Laurence Oberman loberman@users.sourceforge.net wrote: Nevermind The docs say that already Playback.html:need to quote the string if it contains spaces or wildcard characters. vmstatInit fails on second file when using wildcards https://sourceforge.net/p/collectl/discussion/696865/thread/e9f92856cf/?limit=25#1477 Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/collectl/discussion/696865/...
-
Nevermind The docs say that already Playback.html:need to quote the string if it contains spaces or wildcard characters.
-
Maybe I said that somewhere, but maybe I didn't. Great stroll down memory lane -mark On Wed, Nov 16, 2022, 11:46 AM Laurence Oberman loberman@users.sourceforge.net wrote: So the Quotes are needed, meaning we should modify documentation and there is no bug right ? vmstatInit fails on second file when using wildcards https://sourceforge.net/p/collectl/discussion/696865/thread/e9f92856cf/?limit=25#81c4 Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/collectl/discussion/696865/...
-
Bill I tested with collectl -scmnd -oT -p "lobep17-*.raw" 1> l 2>&1 and its picking them all up. If you confirm you are good with that I will update the documentation to reflect that Wildcards for -p need to be enclosed in "" Thanks Laurence
-
So the Quotes are needed, meaning we should modify documentation and there is no bug right ?
-
So having just installed a new power supply in my test box in my mjs account, I ran collect -f. For a few seconds and then ^c. It created 1 mjs..raw.gz file. Repeated and now I have 2 mjs... files. Running collectl -p mjs only plays back 1 file as you've also seen. BUT collectl -p"mjs" plays both back. A single quote also works fine. -mark On Wed, Nov 16, 2022, 11:28 AM Laurence Oberman loberman@users.sourceforge.net wrote: OK, but my tests how no expansion for the wildcard, only one file was used....
-
OK, but my tests how no expansion for the wildcard, only one file was used. Nevertheless if there is a bug provide a reproducer and I will investigate and fix it. collectl -scmnd -oT -p lobep17-*.raw 1> l 2>&1 This only parses the first file matched on the wildcard * [loberman@lobep17 Downloads]$ ls lobep17-*.raw lobep17-20221018-113317.raw lobep17-20221019-000000.raw lobep17-20221020-000000.raw lobep17-20221021-000000.raw
-
might want to try with some debugging switches, maybe -d1 or -d17? see beginning of collectl.pl for all options -mark On Wed, Nov 16, 2022 at 10:15 AM Bill Torpey wallstprog@users.sourceforge.net wrote: Weird -- not what I see: $ ls -lh .raw -rw-rw-r-- 1 btorpey shared 102M Nov 14 17:42 csappia01-20221110-223734.raw -rw-rw-r-- 1 btorpey shared 2.0G Nov 15 12:06 csappia01-20221111-000000.raw -rw-rw-r-- 1 btorpey shared 2.0G Nov 15 12:06 csappia01-20221112-000000.raw -rw-rw-r-- 1 btorpey shared 1.1G...
-
I disagree. I think the magic is you need to quote the filespec so it doesn't get expanded -mark On Wed, Nov 16, 2022, 10:57 AM Laurence Oberman loberman@users.sourceforge.net wrote: Wildcards are not expanded for -p, it will use the first file it matches on only, so to answer your question. Do not use wildcards for -p vmstatInit fails on second file when using wildcards https://sourceforge.net/p/collectl/discussion/696865/thread/e9f92856cf/?limit=25#fb0b Sent from sourceforge.net because you indicated...
-
I spent more time than I can remember getting multiple files processed with wild cards, particularly so they would generate correct plottable output. For a single file collectl always uses the first sample as the baseline to base second set of data on. BUT if multiple files you only want to skip timestamp on subsequent files if not immediately following previous ones or else you end up missing an interval and can mess up plotting. Sheesh... In other words processing multiple files, even for multiple...
-
Just the one file -- not all? Was there anything on stderr?
-
Wildcards are not expanded for -p, it will use the first file it matches on only, so to answer your question. Do not use wildcards for -p
-
so I dug a little deeper since over a few years since I've run collectl. As you can see, -d1 will show file opens (among other things). # this is just to make debugging time frames easier especially if user gets odd results. if ($debug & 1) { my $fromstamp=getDateTime($fromSecs); my $thrustamp=getDateTime($thruSecs); print "PlayBack From: $fromstamp Thru: $thrustamp\n"; } you'll see constructs like "if ($debug & xxx)" scattered heavily throughout the code which makes it possible to use any combination...
-
I never use wildcards for p but in the test I ran it picked up the first match on *.raw
-
when I wrote the vmstat module it was in response to someone I was trying to convince to use collectl. He said he'd still use vmstat since collectl couldn't generate output in his favorite format, so I added it. Almost never used it but was very useful for debugging questions of data integrity. Same thing with iostat. Never use that either except when someone accused vmstat of being wrong and by generating the same wrong output with iostat was able to get kernel developers attention. ;) -mark On...
-
OK, I have 4 files now in my test directory -rw-r--r-- 1 loberman loberman 827215610 Nov 16 10:36 lobep17-20221020-000000.raw -rw-r--r-- 1 loberman loberman 826461531 Nov 16 10:36 lobep17-20221019-000000.raw -rw-r--r-- 1 loberman loberman 436974087 Nov 16 10:36 lobep17-20221018-113317.raw -rw-rw-r-- 1 loberman loberman 292256261 Nov 16 10:05 lobep17-20221021-000000.raw [loberman@lobep17 Downloads]$ ls *.raw lobep17-20221018-113317.raw lobep17-20221019-000000.raw lobep17-20221020-000000.raw lobep17-20221021-000000.raw...
-
FWIW, simply removing the check fixes: sub vmstatInit { #error("-s not allowed with 'vmstat'") if $userSubsys ne ''; error("-f requires either --rawtoo or -P") if $filename ne '' && !$rawtooFlag && !$plotFlag; error("-P or --rawtoo require -f") if $filename eq '' && ($rawtooFlag || $plotFlag); $subsys=$userSubsys='cm'; } $ /shared/collectl/collectl -p 'csappia01-*.raw' --from 20221110:23:52:30-20221113:04:57:08 --export vmstat -oD | tail 20221113 04:56:59 0 0 0 45165M 332M 65719M 2472M 63521M 0 0...
-
It's true that there's other stuff in the directory, but it shouldn't matter? $ ls -lh total 5.2G -rw-rw-r-- 1 btorpey shared 102M Nov 14 17:42 csappia01-20221110-223734.raw -rw-r--r-- 1 btorpey shared 3.2M Nov 11 00:00 csappia01-20221110-223734.raw.gz -rw-rw-r-- 1 btorpey shared 2.0G Nov 15 12:06 csappia01-20221111-000000.raw -rw-r--r-- 1 btorpey shared 78M Nov 12 00:00 csappia01-20221111-000000.raw.gz -rw-rw-r-- 1 btorpey shared 2.0G Nov 15 12:06 csappia01-20221112-000000.raw -rw-r--r-- 1 btorpey...
-
Weird -- not what I see: $ ls -lh *.raw -rw-rw-r-- 1 btorpey shared 102M Nov 14 17:42 csappia01-20221110-223734.raw -rw-rw-r-- 1 btorpey shared 2.0G Nov 15 12:06 csappia01-20221111-000000.raw -rw-rw-r-- 1 btorpey shared 2.0G Nov 15 12:06 csappia01-20221112-000000.raw -rw-rw-r-- 1 btorpey shared 1.1G Nov 15 12:06 csappia01-20221113-000000.raw $ collectl -p 'csappia01-*.raw' --from 20221110:23:52:30-20221113:04:57:08 --export vmstat -oD | tail 20221110 23:59:52 0 0 0 114G 321M 1832M 370M 1725M 0 0...
-
OK its working fine for me. You must have another set of files its matching in there. [loberman@lobep17 Downloads]$ collectl -p lobep17-20221021-000000.raw --export vmstat -oD # procs ---------------memory (KB)--------------- --swaps-- -----io---- --system-- ----cpu----- #Date Time r b swpd free buff cache inact active si so bi bo in cs us sy id wa 20221021 00:00:10 0 0 26624K 9539M 6332K 96618M 63982M 14931M 0 0 0 150 3890 4423 2 0 96 0 20221021 00:00:20 0 0 26624K 9205M 6332K 96941M 64305M 14931M...
-
Hello Bill Thanks for reporting this. As soon as I get a chance will look into it. Having said that I never use the vmstat export function so there may be an issue that has been lurking in there for a while already.
-
When exporting with vmstat format using a wildcard filespec, the first file processes OK but the second fails with "Error: -s not allowed with 'vmstat'". Sample command: collectl -p 'collectl/csappia01-*.raw' --from 20221110:23:52:30-20221113:04:57:08 --export vmstat -oD Obviously the "-s" is not on the command line, but it appears to be coming from somewhere? Thanks in advance for any help...
-
Hello November 7 2022 If you want to clone collectl git clone https://github.com/sharkcz/collectl.git I have added the tar file for colplot to the git tree. You will always get the latest version with a git pull after the clone
-
-
Collectl was modified to day to include default support for rbd devices (Ceph disks) Testing will happen this weekend and then I will release a new version. Regards Laurence
-
For documentation there is a docs directory that will come along too
-
Hi Mark Everything will move My current git tree has [loberman@lobep17 collectl]$ ls ARTISTIC collectl.conf docs gexpr.ph hello.ph lexpr.ph proctree.ph RELEASE-collectl UNINSTALL vnet.ph client.pl colmux envrules.std GPL initd man1 README service vmstat.ph collectl COPYING formatit.ph graphite.ph INSTALL misc.ph README-WINDOWS statsd.ph vmsum.ph So if one clones it you get a copy of what was in the tar file
-
I think moving collectl to github would probably be a good idea, My main questions, and remember it's been awhile since I've even touched the code, is there are a lot of pieces. For example I always liked the plugin "vmstat" because it produced the same output as vmstat only using collectl data. The cool thing here is you could take collectl data and play it back though vmstat, something you can't do with the native tool. The other 2 biggies for me have always been colplot and colmux, especially...
-
Hello Folks, When I took over keeping collectl alive, I really did not get involved in any of the SourceForge Admin functions Mark used to do. So releasing fixes and updates means I am not able to update all the proper LINKS and messages etc. It would be a lot easier for me to use a git repo for this so I am letting you know that collectl will probably move to a git repo soon. It means you would use git clone and then got pull to get the updates. We don't provide RPMS anymore, just the INSTALL script...
-
As an fyi I resisted doing that kind of precision with regular data because it was t[ much of a pain getting column headers aligned in all cases. On the other hand since no header alignment fir plot format, -o 1/2 sets the precision for all numeric output. -mark On Tue, Aug 23, 2022, 5:02 PM Laurence Oberman loberman@users.sourceforge.net wrote: Bud Brown, my colleague has provided a patch for a new level of precision output. I will be adding this in the next release His note is as follows Patch:...
-
-
-
-
-
-
-
Bud Brown, my colleague has provided a patch for a new level of precision output. I will be adding this in the next release His note is as follows Patch: add --dskopts=F extended precision option, a bigger --dskopts=f The existing --dskopts=f option increases precision in some columns (Example #2 vs #1), to 1 digit. The proposed --dskopts=F option increases precision in all columns (Example #3 vs #2), to 2 digits similar to iostat output. In also removes the contraction of large iops and KB/s numbers....
-
Note that his comments should reflect diskopts not dskopt, that will be fixed when I release.
-
Bud Brown, my colleague has provided a patch for a new level of precision output. I will be adding this in the next release His note is as follows Patch: add --dskopt=F extended precision option, a bigger --dskopt=f The existing --dskopt=f option increases precision in some columns (Example #2 vs #1), to 1 digit. The proposed --dskopt=F option increases precision in all columns (Example #3 vs #2), to 2 digits similar to iostat output. In also removes the contraction of large iops and KB/s numbers....
-
Finally got around to this vmstat Swap si: Amount of memory swapped in from disk (/s). so: Amount of memory swapped to disk (/s). vmstat swap Time si so 16:02:27 256 77996 Collectl # <---Swap--> In Out 16:02:27 256M 78G @Manabu, Thank you Something is not right, thanks for reporting this, I will decide how best to fix and patch it. Captured data if good, it's the playback that needs checking like you reported.
-
Apologies, I have been buried doing my day job. I intend to make time to investigate this later this week. Regards Laurence
-
Manabu Gomi posted a comment on discussion Help
oh, I've missed reading message from Laurence and followed by Mark before I sent above. I appreciate your continued support.
-
Thank you for your comment ! I tried collectl --vmstat and compared with vmstat -S k 1. For example when vmstat said "49" (means 49KB) for swapin, then collectl --vmstat said "49152" (means 49152 B). This results show vmstat and collectl have same value internally. So I wonder if collectl -sm --verbose have the bug to handle unit for swapin and swapout.
-
I haven't forgot about having a sitdown with you. We have a grandbaby due the end of august and hope to spend more time in lexington afterwards -mark On Mon, Jul 4, 2022, 2:42 PM Laurence Oberman loberman@users.sourceforge.net wrote: Hello Only just saw this post. I will look into this and it merits a change I will patch it. Either way I will report back to this thread. Regards Laurence Size of Swap In/Out https://sourceforge.net/p/collectl/discussion/696865/thread/767bbbae5d/?limit=25#c3ae Sent...
-
Hello Only just saw this post. I will look into this and it merits a change I will patch it. Either way I will report back to this thread. Regards Laurence
-
Try "collectl -- vmstat" Collectl collects the data and formats the output like vmstat. Should at least makes easier to look at the two tools side-by-side -mark On Thu, Jun 30, 2022, 2:45 AM Manabu Gomi aojs53@users.sourceforge.net wrote: collectl -sm --verbose tells Swap In/Out size, and I wonder if there may be wrong size format. I've compared collectl output with vmstat output. First line for example, vmstat says 32KB for swap in, but collectl says 32"MB" for it. collectl outpu vmstat output <-----------Swap------------>...
-
collectl -sm --verbose tells Swap In/Out size, and I wonder if there may be wrong size format. I've compared collectl output with vmstat output. First line for example, vmstat says 32KB for swap in, but collectl says 32"MB" for it. collectl outpu vmstat output <-----------Swap------------> | ---swap--- Total Used Free In Out | si so 7999M 5877M 2122M 32M 0 | 32 0 7999M 5877M 2122M 64M 0 | 65 0 7999M 5876M 2123M 368M 0 | 376 0 7999M 5876M 2123M 0 0 | 0 0 7999M 5876M 2123M 48M 0 | 49 0 7999M 5876M...
-
-
-
-
commit 517c19c94ae623eb7ab373edff1563328d03eb2b Author: Laurence Oberman loberman@redhat.com Date: Fri Apr 1 11:52:02 2022 -0400 Patch to fix IB collection in systems with multiple mellanox cards suggested-by: Gary Skouson diff --git a/collectl b/collectl index 86850e42e9e5..71808794b049 100755 --- a/collectl +++ b/collectl @@ -2978,7 +2978,7 @@ for (; $count!=0 && !$doneFlag; $count--) # for OPA V4, /sys counters always 64 bits if ($HCAOpaV4[$i][$j]) { - my $proc="$SysIB/$HCAName[$i]0/ports/$j/counters";...
-
Hello Gary Thanks the patch looks fine, I will commit and add you as a suggested-by. I will update the version number and upload to sourceforge and let you know when its done. After taking over from Mark, I still need to get with him re: the way he used to do the updates. For now I am adding the tar files and alerting that there is a new version when I push the changes. Regards Laurence
-
Hello Thanks for the patch, I will review and commit if all is good. Best regards Laurence Oberman On Fri, Mar 25, 2022 at 8:15 AM Gary Skouson gskouson@users.sourceforge.net wrote: Here's a patch to fix a problem with IB. The use of HCAName isn't right and should be HCAId, especially if you have mellanox ethernet and IB in the same box, the '0' or $i isn't going to work right. For example on one system, we have bonded ethernet on a mellanox ethernet card and a separate IB card, so /sys/class/infiniband...
-
Hello Thanks for the patch, I will review and commit if all is good. Best regards Laurence Oberman
-
Gary Skouson posted a comment on discussion Open Discussion
Here's a patch to fix a problem with IB. The use of HCAName isn't right and should be HCAId, especially if you have mellanox ethernet and IB in the same box, the '0' or $i isn't going to work right. For example on one system, we have bonded ethernet on a mellanox ethernet card and a separate IB card, so /sys/class/infiniband shows two devices mlx5_2 and mlx5_bond_0. For the formatit.ph the logic isn't right. The $link comparison should be with "eq" not "=". I also think we need to chomp $link to...
-
I need to get with Mark and see how he did releases here. I did fix a couple of things in 4.3.2 Regards Laurence
-
For example, here's what I get from one sample of a particular process: proc:10385 stat 10385 (tpsdaemon) S 1 8783 8783 0 -1 1077944320 3273 0 0 0 0 0 0 0 20 0 15 0 985166519 2243244032 4006 18446744073709551615 4194304 4630248 140731205735328 140731205733344 140394182363493 0 0 0 2076278527 18446744073709551615 0 0 17 8 0 0 0 0 0 6728944 6731732 25190400 140731205741431 140731205741653 140731205741653 140731205746638 0 proc:10385 Tgid: 10385 proc:10385 Uid: 1118 1118 1118 1118 proc:10385 VmPeak:...
-
Thanks Mark! I assume that whatever collectl is collecting is in the raw file, so I unzipped that (using cat | gunzip) and grep'd out one of the pid's I was interested in to see what was being collected. I use the following DaemonCommands in /etc/collectl.conf, which I think is grabbing pretty much everything: DaemonCommands = -f /var/log/collectl -r00:00,7 -m -F60 -i 1:1:60 -scCdDmMnNstZ --procfilt ctpsdaemon,ctpsworker,fappia,ctimestensubd,ctimestenrepd,cdropcopy,cnsd,ccollectl
-
A quick look at a raw capture shows Slab shmem_inode_cache 2504 2664 680 24 4 : tunables 0 0 0 : slabdata 111 111 0 Shmem: 115332208 kB nr_shmem 28833052 smaps are per library, we don't get them right now if I am reading this correctly. 7ffc37de1000-7ffc37de3000 r-xp 00000000 00:00 0 [vdso] Size: 8 kB Rss: 4 kB Pss: 0 kB Shared_Clean: 4 kB Shared_Dirty: 0 kB Private_Clean: 0 kB Private_Dirty: 0 kB Referenced: 4 kB Anonymous: 0 kB AnonHugePages: 0 kB Swap: 0 kB KernelPageSize: 4 kB MMUPageSize: 4...
-
I think what Bill is looking is is if we gather /proc/PID/* meaning is there a structure in perl we may have the data in, and we are just making it available for output. I have not had a chance to have a full look at this but I dont think we grab it all. More later, will try investigate this weekend
-
I had to run a quick test to make sure I wasn't forgetting things and I'm happy to report I wasn't. When you tell collectl to collect stats, it actually takes a snapshot of most (if not all) the /proc structures related to what stats you're interested in. So lets say you want to look at memory with -sm, a LOT more is collected than displayed, one of the main reasons for me was that there is so much in /proc reporting all of it would defeat the purpose of trying to fit everything on a single line,...