[djatoka-devel] Restart still taking too long

[Phil C. <phi...@mo...>]

So this is a known issue we first came across when we were testing
djatoka, when restarting Tomcat tries to clear the temp files out of
the default temp directory, in this case /var/lib/tomcat55/temp, but
even though the directory isn't that large (I've seen ones that are
30Megs) they'll have a huge amount of files in them (I've seen over
100,000).  Tomcat does a 'rm -fr' on this directory, but rm really
calls out each file, thus the line becomes huge, and it takes forever
to come back (we've seen it take over 10 minutes) if it does at all.
I've turned down the amount that djatoka saves as cache, but this is
still an issue.  So, I get a message when Tomcat has crashed (this has
happened more often of late, details in next email) by monit., which
checks state every 2 minutes, and restarts it when it finds that it's
down.  It's at this point, when it's starting, that the server is
completely unavailable to the user - and I get alerted that the web
site is no longer available.  Generally I will go in, kill the rm
process, manually move the temp directory out of the way, create a new
temp directory with the correct permissions, and then restart tomcat.

Has anyone else encountered this?  I know we're somewhat atypical
since we have a large amount of images, and for that matter we're
discussing how much of a benefit we're getting with cached tiles.  One
of my ideas is to have the startup script do what I've been doing
manually, moving the existing directory, creating a new one and
setting the permissions, before actually starting Tomcat.  This still
feels like a work around, but I can't think of other options save for
turning off djatoka caching all together.  Is there any issues with
doing this?

Thanks

P

[08:59:11] [root@PRODUCTION /root !]# ps -fe | grep tom
root      7876     1  0 08:58 ?        00:00:00 /bin/sh
/etc/init.d/tomcat5.5 start
root      7906  7876  0 08:58 ?        00:00:00 find
/var/lib/tomcat5.5/temp/ -mindepth 1 -exec rm -fr {} +
root      7960  7906  0 08:59 ?        00:00:00 rm -fr
/var/lib/tomcat5.5/temp/cache1184270472-4524035889044291109.jpg
/var/lib/tomcat5.5/temp/cache426607110-209395681246992266.jpg
/var/lib/tomcat5.5/temp/cache-1033655268-3575920668857858540.jp2
/var/lib/tomcat5.5/temp/cache1951692115-4506731700190221933.jp2
/var/lib/tomcat5.5/temp/cache19963549364706905359200625835.jp2
/var/lib/tomcat5.5/temp/cache-528176946-5141609634088189850.jpg
/var/lib/tomcat5.5/temp/cache614655010-2236924771246762820.jpg
/var/lib/tomcat5.5/temp/cache-529404616-859568200065330511.jpg
/var/lib/tomcat5.5/temp/cache1461641178-126374537623619037.jp2
/var/lib/tomcat5.5/temp/cache1925339310-7999529852255043508.jp2
/var/lib/tomcat5.5/temp/cache-1813559268-1004634521019417393.jpg
/var/lib/tomcat5.5/temp/cache-376690266-44408426910478702.jpg
/var/lib/tomcat5.5/temp/cache-1845623557-7633926519616984307.jpg
/var/lib/tomcat5.5/temp/cache-889395029-6974374199862490878.jpg
/var/lib/tomcat5.5/temp/cache1122815355-4050730441240872578.jp2
/var/lib/tomcat5.5/temp/cache-236642438-1166979274653885320.jp2
/var/lib/tomcat5.5/temp/cache-803678129-1792946565933688717.jpg
/var/lib/tomcat5.5/temp/cache-1960088780-1809731222907048073.jpg
/var/lib/tomcat5.5/temp/cache4020300215470390422722455855.jp2
/var/lib/tomcat5.5/temp/cache-1230166449-8758583368926836588.jpg
/var/lib/tomcat5.5/temp/cache-1551198983-4012942121981001462.jp2
/var/lib/tomcat5.5/temp/cache-1489213184-6348253468856214072.jp2
/var/lib/tomcat5.5/temp/cache1766123464-7827911098371685319.jpg
/var/lib/tomcat5.5/temp/cache-625413588-7534013171343145351.jpg
/var/lib/tomcat5.5/temp/cache1373691925-2125720551882442542.jpg
/var/lib/tomcat5.5/temp/cache-1395135005-2895519933699674092.jpg
/var/lib/tomcat5.5/temp/cache1531266747-4519854669048070961.jp2
/var/lib/tomcat5.5/temp/cache-803309180-1127368665335618889.jpg
/var/lib/tomcat5.5/temp/cache-1232253866-7323697829008995828.jp2
/var/lib/tomcat5.5/temp/cache1669305335-8311441949794588176.jp2
/var/lib/tomcat5.5/temp/cache14661060996353724725560880346.jp2
/var/lib/tomcat5.5/temp/cache-14512478498941284313597377695.jp2
/var/lib/tomcat5.5/temp/cache-1586727473-7696723935179191417.jpg
/var/lib/tomcat5.5/temp/cache18990668214444486867253735672.jp2
/var/lib/tomcat5.5/temp/cache1536306170-389773011284292442.jpg
/var/lib/tomcat5.5/temp/cache-845753873-814466086204438185.jpg
/var/lib/tomcat5.5/temp/cache-2110940519417259212630122233.jp2
/var/lib/tomcat5.5/temp/cache1558230936-3541258274474335310.jp2
/var/lib/tomcat5.5/temp/cache3392935917906614044961299506.jp2
/var/lib/tomcat5.5/temp/cache-1005569674-3223338652273562320.jp2
/var/lib/tomcat5.5/temp/cache-1895310874-7769264953038019905.jpg
/var/lib/tomcat5.5/temp/cache2121937882-8719417294768800449.jp2
/var/lib/tomcat5.5/temp/cache882497774786640001082932696.jp2
/var/lib/tomcat5.5/temp/cache8383548137944959643571839282.jp2
/var/lib/tomcat5.5/temp/cache-312279762653529439335311.jp2
/var/lib/tomcat5.5/temp/cache-18216305062555376993601916047.jp2
/var/lib/tomcat5.5/temp/cache-563928439-4640798296425311576.jpg
/var/lib/tomcat5.5/temp/cache-2039561707-2749572114750607340.jp2
/var/lib/tomcat5.5/temp/cache-1972443969-1496261699603163227.jpg
/var/lib/tomcat5.5/temp/cache-1899473713-3120037167286243905.jpg
/var/lib/tomcat5.5/temp/cache-902359152070137796891946195.jp2
/var/lib/tomcat5.5/temp/cache1848176469-4679822454721226129.jp2
/var/lib/tomcat5.5/temp/cache-1193217293-6398016629160026108.jp2
/var/lib/tomcat5.5/temp/cache1082775484-4433907878005722079.jp2
/var/lib/tomcat5.5/temp/cache-9599853895156272328294593767.jp2
/var/lib/tomcat5.5/temp/cache572363197-1865627520249473075.jpg
/var/lib/tomcat5.5/temp/cache-886580918-5139537638173860585.jpg
/var/lib/tomcat5.5/temp/cache-452695571-3956263739857014424.jpg
/var/lib/tomcat5.5/temp/cache-13686205504043334501367849502.jp2
/var/lib/tomcat5.5/temp/cache1992787829-4395410432249625698.jp2
/var/lib/tomcat5.5/temp/cache-18421454165812008047076141132.jp2
/var/lib/tomcat5.5/temp/cache1539195452-8763600934507919226.jpg
/var/lib/tomcat5.5/temp/cache-1865121739-7318888375445417343.jp2
/var/lib/tomcat5.5/temp/cache-2097539610-4984804556387702492.jpg
/var/l