Re: [htdig-dev] news.txt file gone

[Gilles D. <gr...@sc...>]

According to Jesse op den Brouw:
> when accessing the htdig main website, file main=A9shtml does not
> contain the included news=A9txt file, allthough the string is
> definitely in the file itself=A9
> Can anyone verify this?

Yes, there seems to have been a hiccup in the news update script
(/home/groups/h/ht/htdig/scripts/news-get.sh) which runs at 3:42 PST
every morning.  It uses a wget query to fetch the URL
http://sourceforge.net/export/projnews.php?group_id=3D4593&limit=3D4&flat=
=3D0&show_summaries=3D1
and then checks that the file it got is not empty.  If it's not, it
copies it to maindocs/news.txt and cvs commits it.  Well, this morning,
the commited file was empty, so I can only assume the copy had failed.

Assuming it was either a temporary disk full condition (in which case
there's not much we can do - it's not full now) or a quota exceeding
(which I didn't hear about, and which I can't seem to find out about righ=
t
on their shell.sf.net server), I deleted some old database backup files
to cut down on our disk usage, in the hope that the problem will be less
likely to recur.  I also manually regenerated the news.txt file for today.

Not much else I can do.  Geoff, did you get any error messages from this
morning's script run?

> BTW: a few mirror do not use Server Side Includes, so the
> news isn't shown at all on theire pages=A9 Can we change this?

We could probably change the news-get.sh script to build a mail.html
file from the main.shtml and news.txt files, and commit that.  We'd also
need to change contents.html to point back to news.html.  This would
actually make the maindocs files a bit more like the htdig3/htdoc files,
only news.html would be regenerated on a daily basis.

The potential problem I see with this is the Last modified date will
keep changing every day, even if there's no new news, and news.html
will get copied to all cvs users who get daily updates of maindocs.
Can anyone suggest a way around this, without losing the Last modified
date altogether?  I suppose the script could compare the regenerated
news.txt against the previous one, and only regenerate main.html if
news.txt's content changes, or main.shtml has been changed since the
last time.

Another way might be to do away with main.shtml and news.txt from the
maindocs tree altogether, and just have a main.html file, as before the
SourceForge days.  Only difference is this time, the news section in
main.html would be between clear delimiters, and the news-get.sh script
would use these to automatically strip out and reinsert updated news
items from the one file.  It would recommit it only if it was different
from yesterday's file.  The script should also have proper tests to
ensure the newly generated file is indeed complete, to prevent the whole
thing from getting clobbered in the event of a disk space crunch, but
with that, it may be the best option from a maintenance point of view.
I'm leaning towards this latter approach.

Any ideas, concerns, other thoughts from the developers about this?

--=20
Gilles R. Detillieux              E-mail: <gr...@sc...>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/
Dept. Physiology, U. of Manitoba  Winnipeg, MB  R3E 3J7  (Canada)