Hey,

I sent the mail quoted below to the Nagios list, but after reading http://www.mail-archive.com/jfs-discussion@lists.sourceforge.net/msg00748.html - it seems to me that this is a JFS problem.

I'm ending up with a spool directory with only a few KB of content at any given time that grows over 200MB/month, and I don't have capacity to afford that, so I'm left with manually recreating the directory all the time.  The files in the directory (each under 1KB) are short-lived though, and have a very high turnover rate.

Is there any sort of way to fix this behavior?  This would affect anyone running Nagios on JFS.  My nagios configuration is not even very large, so I imagine this could be a much bigger issue for some.

Thanks for any advice,
--
Casey Allen Shobe

---------- Forwarded message ----------
From: Casey Allen Shobe <casey@shobe.info>
Date: 2010/1/26
Subject: Unexplainable disk space usage in nagios/var/spool/checkresults
To: nagios-users@lists.sourceforge.net


Hi!

I have Nagios installed on a 1GB partition, with everything installed under a nagios directory on that partition.  I did not expect Nagios to start consuming a large amount of space, and now that it is, I cannot determine why or in what files, which is baffling.

I've found the biggest culprit is the var/spool/checkresults directory within the Nagios installation.  a `du` command on the directory shows that it is using 287MB.  However, when I `ls -la` within that directory, I can't see any file using more than a single kilobyte, and there aren't that many files.  However, an `ls -ld` on the directory itself shows that the directory itself is using 286MB.  That doesn't make sense to me, and so I'm looking for input!  I wonder if the following approach would help correct things (but I'd like to understand why they're in this state first): Stop nagios, move the checkresults directory to checkresults.old, make a new checkresults directory, copy all the old files into it, remove the empty .old directory, start nagios.

The filesystem is JFS, if that makes any difference.

Here is raw output of what I'm seeing:

root@indhlcwww1:/mnt/data/share/nagios-3.2.0/var/spool/checkresults
# du -h
287M    .

root@indhlcwww1:/mnt/data/share/nagios-3.2.0/var/spool/checkresults
# ls -laSrh
total 287M
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cZar4eF.ok
-rw------- 1 nagios nagios    0 2010-01-04 19:15 cy8clT1.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cXaj1gN.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cvWmN4R.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cVvx4M8.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cvtm0Hd.ok
-rw------- 1 nagios nagios    0 2009-12-06 01:27 cvqSGDK
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cv6pJ1B.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 ctwW4SD.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cTpIKAl.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cTm3Vzw.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 ctCCihr.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 csRsM4M.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cryC1Dq.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cRR4Ubm.ok
-rw------- 1 nagios nagios    0 2009-11-27 23:24 cqqacdj
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cq75lkN.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cnxxq4z.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cnDr6c0.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cmxnujo.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cMPxHgo.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cMf26lf.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cLksJml.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 ckVnJcE.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cjjgyfh.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cjCc8ov.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 chokVz4.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cHN3b3M.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cHlsWuh.ok
-rw------- 1 nagios nagios    0 2009-12-11 19:19 checkyTpLfU
-rw------- 1 nagios nagios    0 2009-11-05 04:12 checkXbP1al
-rw------- 1 nagios nagios    0 2009-11-04 12:53 checkWx0m6z
-rw------- 1 nagios nagios    0 2009-11-05 04:12 checkwpqhVE
-rw------- 1 nagios nagios    0 2009-12-29 17:23 checkvMraRW
-rw------- 1 nagios nagios    0 2009-12-11 19:20 checkqWrl04
-rw------- 1 nagios nagios    0 2009-11-04 12:53 checkQq7MX0
-rw------- 1 nagios nagios    0 2009-12-29 17:23 checknzRtjN
-rw------- 1 nagios nagios    0 2009-11-05 04:12 checkM6htUZ
-rw------- 1 nagios nagios    0 2009-11-27 23:24 checkhVbOzY
-rw------- 1 nagios nagios    0 2009-11-17 12:43 checkEVDugp
-rw------- 1 nagios nagios    0 2009-11-04 12:53 checkd4A1MS
-rw------- 1 nagios nagios    0 2009-11-05 04:12 checkBXX71Z
-rw------- 1 nagios nagios    0 2009-11-27 23:24 checkBLCU9i
-rw------- 1 nagios nagios    0 2009-11-04 12:53 checkAaA6Qr
-rw------- 1 nagios nagios    0 2009-12-06 01:27 check8avYJJ
-rw------- 1 nagios nagios    0 2009-12-29 17:23 cGZFyHr
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cgBr4Y1.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cFxFlm3.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 cD2T5eV.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 caw2Ihv.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 c9iVtCP.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 c7QYuXc.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 c75znoT.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 c6NV5yW.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 c59UhZL.ok
-rw------- 1 nagios nagios    0 2010-01-26 15:23 c1m1KIr.ok
drwxr-xr-x 3 root   root      8 2009-10-20 01:44 ../
-rw------- 1 nagios nagios  249 2009-11-17 15:03 check47CLdy
-rw------- 1 nagios nagios  252 2009-11-17 15:03 checkAE7iMM
-rw------- 1 nagios nagios  277 2009-11-17 15:03 checkaNpheC
-rw------- 1 nagios nagios  278 2010-01-26 15:23 checkWDzCTc
-rw------- 1 nagios nagios  279 2010-01-26 15:23 checkpLyZS3
-rw------- 1 nagios nagios  280 2010-01-26 15:23 checkVSgE8U
-rw------- 1 nagios nagios  281 2010-01-26 15:23 check6txgYX
-rw------- 1 nagios nagios  285 2009-11-17 15:03 checkQXTDa5
-rw------- 1 nagios nagios  394 2010-01-26 15:23 cD2T5eV
-rw------- 1 nagios nagios  395 2010-01-26 15:23 cgBr4Y1
-rw------- 1 nagios nagios  397 2010-01-26 15:23 cZar4eF
-rw------- 1 nagios nagios  397 2010-01-26 15:23 ctwW4SD
-rw------- 1 nagios nagios  401 2010-01-26 15:23 ctCCihr
-rw------- 1 nagios nagios  401 2010-01-26 15:23 cjCc8ov
-rw------- 1 nagios nagios  403 2010-01-26 15:23 cHlsWuh
-rw------- 1 nagios nagios  404 2010-01-26 15:23 cRR4Ubm
-rw------- 1 nagios nagios  408 2009-12-01 08:38 cV0kCF9
-rw------- 1 nagios nagios  409 2010-01-26 15:23 cTpIKAl
-rw------- 1 nagios nagios  409 2010-01-26 15:23 cq75lkN
-rw------- 1 nagios nagios  412 2010-01-26 15:23 cTm3Vzw
-rw------- 1 nagios nagios  412 2009-12-11 19:20 ch4XVu5
-rw------- 1 nagios nagios  413 2010-01-26 15:23 c1m1KIr
-rw------- 1 nagios nagios  416 2010-01-26 15:23 c9iVtCP
-rw------- 1 nagios nagios  417 2010-01-26 15:23 chokVz4
-rw------- 1 nagios nagios  418 2010-01-26 15:23 cVvx4M8
-rw------- 1 nagios nagios  418 2010-01-26 15:23 cnxxq4z
-rw------- 1 nagios nagios  418 2010-01-26 15:23 cLksJml
-rw------- 1 nagios nagios  427 2010-01-26 15:23 cXaj1gN
-rw------- 1 nagios nagios  433 2010-01-26 15:23 ckVnJcE
-rw------- 1 nagios nagios  435 2010-01-26 15:23 cnDr6c0
-rw------- 1 nagios nagios  437 2010-01-26 15:23 cvWmN4R
-rw------- 1 nagios nagios  437 2010-01-26 15:23 c6NV5yW
-rw------- 1 nagios nagios  438 2010-01-26 15:23 cMPxHgo
-rw------- 1 nagios nagios  440 2010-01-26 15:23 cMf26lf
-rw------- 1 nagios nagios  442 2010-01-26 15:23 cmxnujo
-rw------- 1 nagios nagios  442 2010-01-26 15:23 c59UhZL
-rw------- 1 nagios nagios  445 2009-11-05 04:12 cdarn2Z
-rw------- 1 nagios nagios  447 2010-01-26 15:23 csRsM4M
-rw------- 1 nagios nagios  465 2010-01-26 15:23 cjjgyfh
-rw------- 1 nagios nagios  467 2010-01-26 15:23 cryC1Dq
-rw------- 1 nagios nagios  473 2010-01-26 15:23 cHN3b3M
-rw------- 1 nagios nagios  474 2010-01-26 15:23 c7QYuXc
-rw------- 1 nagios nagios  498 2010-01-26 15:23 cv6pJ1B
-rw------- 1 nagios nagios  499 2010-01-26 15:23 c75znoT
-rw------- 1 nagios nagios  504 2010-01-26 15:23 cFxFlm3
-rw------- 1 nagios nagios  506 2010-01-26 15:23 cvtm0Hd
-rw------- 1 nagios nagios  507 2010-01-26 15:23 caw2Ihv
drwxrwxr-x 2 nagios nagios 286M 2010-01-26 15:23 ./

root@indhlcwww1:/mnt/data/share/nagios-3.2.0/var/spool/checkresults
# ls -ldh
drwxrwxr-x 2 nagios nagios 286M 2010-01-26 15:24 ./

Thank you in advance for any advice!
--
Casey Allen Shobe
casey@shobe.info



--
Casey Allen Shobe
casey@shobe.info