Mailman

Here you are a patch to insert the hostname in the report title and Subject
(in case you manage more than one mailman instance and want to use the
same
SENDER for all of them)


---8<--------------------------------------------------------------------------

--- mmdsr.orig 2009-04-19 19:23:52.000000000 +0200
+++ mmdsr 2009-04-19 19:39:29.000000000 +0200
@@ -257,6 +257,12 @@
SENDER="INSERT.YOUR.SENDER.ADDRESS@HERE"
RCPTS="INSERT.YOUR.RECIPIENTS.ADDRESSES@HERE"

+###############################################################################
+# Specify hostname that will appear in the title (and in the Subject:
header)
+###############################################################################
+
+HOSTNAME=`hostname`
+

###############################################################################
# If you run this program in cron at 23:59:00, you need to sleep sixty
# seconds to make sure that you capture all the logs for the previous
day.
@@ -307,11 +313,11 @@
if [ "${RCPTS}x" != "x" ] ; then
echo "From: ${SENDER}" >> $TMP
echo "To: ${RCPTS}" >> $TMP
- echo "Subject: Mailman Daily Status Report: $DAY $YEAR" >> $TMP
+ echo "Subject: ${HOSTNAME}: Mailman Daily Status Report: $DAY $YEAR"
>> $TMP
echo "" >> $TMP
fi

-echo " Mailman Daily Status Report: $DAY $YEAR" >> $TMP
+echo " ${HOSTNAME}: Mailman Daily Status Report: $DAY $YEAR" >>
$TMP
echo "" >> $TMP
echo "" >> $TMP

It would be nice if this only generated output when it found a problem.
Less likely to be ignored. (Actually, what I often do in such scripts is
mail only on error, except even if no error 1x/week -- so I know that the
monitor is alive.)

Failing that, for low activity sites, if it could run less frequently than
daily (say, weekly) without getting confused.

Posted version 0.0.19 which corrects a minor error in the 'posts from
usenet' report.

The following changes have been made in version 0.18

# 0.0.18 Update by Mark Sapiro <mark@msapiro.net>
# Updated on: Tue May 13 08:58:26 PDT 2008
# Changes for Mailman 2.1.10 --
# Added 'bad' to the list of queues.
# Made listing of a queue directory conditional on existence.
# Sumarized unparseable messages and preserved queue entries
# in the Other Errors section.

File Added: mmdsr

Re-did the 0.0.17 change to use the prior sort key field separator by
default but to make it a user settable parameter.
File Added: mmdsr

I have added two files.

Version 0.0.16 corrects a couple of typo/minor issues. It is the version
that has been in the contrib directory of the distribution since Mailman
2.1.8.

Version 0.0.17 processes two generations of rotated logs (assuming they
are named xxx and xxx.1 and xxx.1 exists) so that data is not lost on the
day the logs rotate. It also changes a deprecated sort key option to one
that should work with all sort commands. This version will be in the
contrib directory of the Mailman 2.1.10 distribution.

File Added: mmdsr

File Added: mmdsr

I suggest incorporating the readme into the shell script, so there is only
the one file.

Logged In: YES
user_id=18417

Folks,

Okay, I just uploaded version 0.0.15, which addresses the most recent
comments here from Mark and Tom, and makes a few additional tweaks as
well. See the version comments at the top of the script.

Logged In: YES
user_id=1123998

The vette log summary lists posts held for moderation
individually under "Other Errors:". The following patch
(watch out for wrapped lines) summarizes them by list instead.

--- mmdsr 2005-09-29 21:31:33.000000000 -0700
+++ mmdsrx 2005-11-06 08:20:58.835895418 -0800
@@ -415,10 +415,15 @@
echo "------------------------------" >> $TMP
$GREP -i 'Posting to a moderated newsgroup' $TMPLOG |
$AWK '{ print $6 }' | $SORT | $UNIQ -c | $SORT -nr >> $TMP

+ echo "" >> $TMP
+ echo "Post to moderated list (by list):" >> $TMP
+ echo "------------------------------" >> $TMP
+ $GREP -i 'Post to moderated list' $TMPLOG |
$AWK '{ print $6 }' | $SORT | $UNIQ -c | $SORT -nr >> $TMP
+
echo "" >> $TMP
echo "Other Errors:" >> $TMP
echo "------------------------------" >> $TMP
- $EGREP -vi '(Post by non-member|suspicious
header|message approved|Discarded posting|bulk message
discarded|junk message discarded|Message has implicit
destination|Posting to a moderated newsgroup|Message
discarded, msgid)' $TMPLOG | $SED 's/^.* ([0-9]*) //' |
$SORT | $UNIQ -c | $SORT -nr >> $TMP
+ $EGREP -vi '(Post by non-member|suspicious
header|message approved|Discarded posting|bulk message
discarded|junk message discarded|Message has implicit
destination|Posting to a moderated newsgroup|Post to
moderated list|Message discarded, msgid)' $TMPLOG | $SED
's/^.* ([0-9]*) //' | $SORT | $UNIQ -c | $SORT -nr >> $TMP

else

Logged In: YES
user_id=1159458

ps output on solaris is full of whitespace but a further
echo get's rid of it.
The lines in the smtp log are sometimes broken up by a
newline (right before the msgid) which throws of the
summary. Piping it through sed first will rejoin the broken
lines.
Use $AWK instead of awk.

Patch inserted below:
--- mmdsr.orig 2005-10-19 09:42:30.000000000 +0200
+++ mmdsr 2005-10-19 09:44:23.000000000 +0200
@@ -203,7 +203,8 @@
# there is an easier cross-platform way to do it,
please let me know.

###############################################################################

-MYUID=`$PS -o user -p $$ | $TAIL -1`
+GRABUID=`$PS -o user -p $$ | $TAIL -1`
+MYUID=`echo $GRABUID`
RUNAS="mailman"


###############################################################################
@@ -254,7 +255,7 @@
$TOUCH $TMPLOG
echo "Log file: $LOG" >> $TMP
echo "==============================" >> $TMP
- $GREP -si "^$DAY [0-9][0-9:]* $YEAR" $LOGDIR/$LOG >>
$TMPLOG
+ $SED -e :a -e '$!N;s/\n //;ta' -e 'P;D' $LOGDIR/$LOG |
$GREP -si "^$DAY [0-9][0-9:]* $YEAR" >> $TMPLOG

if [ -f "$LOGDIR/${LOG}" ] ; then

@@ -264,7 +265,7 @@
echo "Hourly Summary of Posts" >> $TMP
echo "-----------------------" >> $TMP

- $SED -e 's/^[A-Z][a-z][a-z] *[0-9]* //' -e
's/:.*$//' $TMPLOG | $UNIQ -c | $SORT -n +1 | awk '{ printf(
"%8d %02d:00-%02d
:59\n", $1, $2, $2 ) }' >> $TMP
+ $SED -e 's/^[A-Z][a-z][a-z] *[0-9]* //' -e
's/:.*$//' $TMPLOG | $UNIQ -c | $SORT -n +1 | $AWK '{
printf( "%8d %02d:00-%02
d:59\n", $1, $2, $2 ) }' >> $TMP

echo "" >> $TMP
echo "Post Count by List" >> $TMP
@@ -295,7 +296,7 @@
echo "" >> $TMP
echo "Hourly Summary of Messages Sent" >> $TMP
echo "-------------------------------" >> $TMP
- $SED -e 's/^[A-Z][a-z][a-z] *[0-9]* //' -e
's/:.* for / /' -e 's/ recips,.*$//' $TMPLOG | awk '{
val=int($1); sum[val]+=$2
} END { for (i=0; i<24; i++) { printf "%8d
%02d:00-%02d:59\n", sum[i], i, i } }' >> $TMP
+ $SED -e 's/^[A-Z][a-z][a-z] *[0-9]* //' -e
's/:.* for / /' -e 's/ recips,.*$//' $TMPLOG | $AWK '{
val=int($1); sum[val]+=$
2 } END { for (i=0; i<24; i++) { printf "%8d
%02d:00-%02d:59\n", sum[i], i, i } }' >> $TMP

else

Logged In: YES
user_id=18417

Okay, I took Mark's comments and incorporated them. Adrian Wells also
pointed out a log file difference between Mailman 2.1.5 and 2.1.6 that
caused the summary of the "smtp" log to be munged.

I've now fixed these bugs, deleted the old file, and uploaded the new one

(version 0.0.12).

Thanks!

Logged In: YES
user_id=1123998

Oooops! I mistakenly thought I could add the mmdsr.patch as
a downloadable file, but I can't so here it is. Watch out
for wrapped lines ...

--- mmdsr.orig 2005-09-06 20:37:53.000000000 -0700
+++ mmdsr 2005-09-10 18:33:14.532393572 -0700
@@ -132,6 +132,7 @@

MMDIR="/usr/local/mailman"
TMPDIR="/tmp"
+LOGDIR="/var/log/mailman"


###############################################################################
# Maximum number of subdirectory entries to display in report
@@ -234,9 +235,9 @@
$TOUCH $TMPLOG
echo "Log file: $LOG" >> $TMP
echo "==============================" >> $TMP
- $GREP -si "^$DAY [0-9][0-9:]* $YEAR" logs/$LOG >> $TMPLOG
+ $GREP -si "^$DAY [0-9][0-9:]* $YEAR" $LOGDIR/$LOG >>
$TMPLOG

- if [ -f "logs/${LOG}" ] ; then
+ if [ -f "$LOGDIR/${LOG}" ] ; then

if [ "${LOG}" = "post" ] ; then

@@ -304,9 +305,9 @@
$TOUCH $TMPLOG
echo "Log file: $LOG" >> $TMP
echo "==============================" >> $TMP
- $GREP -si "^$DAY [0-9][0-9:]* $YEAR" logs/$LOG >> $TMPLOG
+ $GREP -si "^$DAY [0-9][0-9:]* $YEAR" $LOGDIR/$LOG >>
$TMPLOG

- if [ -f "logs/${LOG}" ] ; then
+ if [ -f "$LOGDIR/${LOG}" ] ; then

if [ "${LOG}" = "error" ] ; then

@@ -453,4 +454,4 @@
$CAT $TMP
fi

-$RM $TMP
+$RM -f $TMP $TMPLOG

Logged In: YES
user_id=1123998

I tried the script and other than the expected site specific
changes, I found two issues which are both addressed by the
patch in the mmdsr.patch file.

1) Mailman's log files are not necessarily in
$var_prefix/logs/ - they can be in any directory. The patch
adds a LOGDIR directory independant of MMDIR.

2) The $TMPLOG file is not removed at completion. The patch
removes it.

Logged In: YES
user_id=18417

Okay, I've deleted the old version of the mmdsr script that was attached,

and uploaded the latest version. This is what we're currently using to
monitor the lists on python.org, and we have found it very useful.

Any comments you may have will be appreciated.

Logged In: YES
user_id=18417

The UID variable in the current code was already replaced by MYUID,
because I got complaints on other platforms. But UID wasn't available to

me as a useful constant, so I had to use something else to obtain the
value.

The recommended patch from tgc99 does work, and I will be uploading a
new version of the code soon.

Logged In: YES
user_id=1175103

Use of variable named UID does work well with OS X (version
10.2.8). Apparently the variable UID is a constant already
in use. When trying to the run the script without
modification, I was receiving the following error message:

UID: readonly variable

This issue was resolved by changing the name of variable,
UID, to something else, such as MMUID. Works fine with this
change.

As an aside (& for what it is worth), the UID grab command
suggested by tgc99 on 2005-02-16 03:15 works on this system
(OS X - version 10.2.8)

Logged In: YES
user_id=1159458

The current UID grab command doesn't work on Solaris (2.6 &
8 tested).

I'd recommend this instead:
ps -o user -p $$|tail -1

This is tested and works on RH 6.2, RH 7.3, RHEL 2.1, RHEL3,
FC3, FreeBSD 4.9, Solaris 2.6, 8.