#36 Option to include *all* headers in logfile

[Vanguard]

Currently the only headers included in the
msgdeleted.log file are extremely minimal, like To, From,
and Subject, all of which are optional headers according
to RFC 2822 and are part of the DATA transfer to the
SMTP server which the sender can modify to be
anything they want. If e-mails are getting deleted off
the server using a spam-catch rule, the user has no
clue what were the real Received headers to track the
routing of that message. Also, I use SpamPal to detect
spam and Magic is configured to go through that proxy.
When SpamPal detects a message as spam, it adds
headers that are used by rules to detect that SpamPal
marked the message as spam. I use a rule in Magic to
delete that spam off the server. The logfile doesn't
show the headers so I cannot see SpamPal's explanation
as to why the message was considered spam. Could've
been a blacklisted source but the blacklist used is
unknown. Could've been due to web bugs, HTML tricks,
HTML-coded message with no alternative text MIME
part, Bayesian detection, and so on. I cannot monitor
how SpamPal is catching these spams because I cannot
see all the headers for that message that Magic then
deletes off the server to eliminate me having to
download them and then delete them locally using rules
in my e-mail client. Since users can configure the logfile
size (to prevent excessive size due to growth over
time), and since hard disk are getting rather huge these
days, adding a dozen more lines on average to show all
the headers wouldn't be considered excessive. An
option would let the user record just the simplistic
information that is now recorded in the logfile or allow
full headers to get recorded for each deleted item.
From the log for an account, when enabled, I already
see that Magic issues the "TOP <msgnum> 0" command
which has the POP3 server send all the headers. So
simply echo into the msgdeleted.log file the same
headers as reported by the POP3 server.

I can see a potential problem if Magic is keeping all
these headers in memory, that more memory would get
consumed to retain all instead of just some of the
headers for each message item. However, if the user is
not using any rules in Magic to delete from the server
then the messages will still be there to get yanked by
Magic (to preview the message) or by the e-mail client.
So once the rules have all been exercised then compact
the recorded data stored by Magic in memory to contain
just the simple and limited set of headers. Or, Magic
could keep one cache for item properties with just the
simple header info. A separate cache used on each mail
poll by account would be temporarily available to record
the output of the TOP command to get the headers.
That temp per-account cache would get released to
free up memory after the mail poll completes and after
all the rules have been exercised. Once the rules have
been ran, some of which may include the option to
delete from the server, the need for the full headers are
no longer needed.