Courier Mail Server

Doug Rintoul writes:

> I am getting this message when I use fetchmail to pull in a certain message 
> and send it on to courier. What causes this message and is there a way I can 
> increase the policy limit?

You probably have a mail loop somewhere.  This error is reported when: 

* A single header line exceeds 5000 characters. 

* There more more than a 100 Received: headers. 

* The size of all the header exceeds 100,000 bytes. 

These limits are perfectly reasonable.  If you're hitting it, you have 
something else that's broken. 

 


-- 
Sam

On Tuesday 13 March 2001 12:10 pm, you wrote:
> *This message was transferred with a trial version of CommuniGate(tm) Pro*
>
> Doug Rintoul writes:
> > I am getting this message when I use fetchmail to pull in a certain
> > message and send it on to courier. What causes this message and is there
> > a way I can increase the policy limit?
>
> You probably have a mail loop somewhere.  This error is reported when:
>
> * A single header line exceeds 5000 characters.
>
> * There more more than a 100 Received: headers.
>
> * The size of all the header exceeds 100,000 bytes.
>
> These limits are perfectly reasonable.  If you're hitting it, you have
> something else that's broken.

Thanks for the info. I am pretty sure that it is not caused by a loop. 
Looking at the communication between fetchmail and the server, it looks like 
the message header is a total of 6937 bytes, and the message body is 4394. So 
it looks like you first suggestion is the case here, ie. a single header line 
exceeds 5000 characters. I will try a modified version of Courier (I am 
assuming that courier/submit.C is what I would change since that is where the 
message is generated), to see if I can pull in the message. Or I may try 
another email client and examine the message after I pull it in.

Doug.

On Tuesday 13 March 2001 12:10 pm, you wrote:
> *This message was transferred with a trial version of CommuniGate(tm) Pro*
>
> Doug Rintoul writes:
> > I am getting this message when I use fetchmail to pull in a certain
> > message and send it on to courier. What causes this message and is there
> > a way I can increase the policy limit?
>
> You probably have a mail loop somewhere.  This error is reported when:
>
> * A single header line exceeds 5000 characters.
>
> * There more more than a 100 Received: headers.
>
> * The size of all the header exceeds 100,000 bytes.
>
> These limits are perfectly reasonable.  If you're hitting it, you have
> something else that's broken.

I was able to pull in the message with another email client and examine the 
message. It looks like the To: header is too long. The person sending the 
message sent it to 110 recipients and the total byte count is 5369. On 
average, each recipient's email address is 49 characters. This is because a 
long "friendly name" is included with each recipient's address. Question is, 
is this my problem or the sender's problem? What do I tell my clients? Keep 
their mailing lists below a certain number? It is not unusual to have my 
users send to two or three hundred people at a time, especially if it is a 
news letter.

Doug.

Sam Varshavchik writes:

> * A single header line exceeds 5000 characters.

> These limits are perfectly reasonable.  If you're hitting it, you have
> something else that's broken.
>
It's right with one exception: due to minor logical error in submit.C or
intentionally 5K length limit is applied not only to the single header line
but to the single header with properly wrapped (beginning with space or tab)
lines too. It leads to the bounces of absolutely RFC-compliant messages with
relatively short (< 10K) multiple recipient To: or Cc: headers.

For the time being I've applied attached patch but I'm not absolutely sure
that it will not entail any undesirable consequences, because single header
can now reach "headerlimit" (100000 bytes) length. Maybe Sam will suggest
more correct solution.

Alexei.

Alexei Batyr' writes:

> Sam Varshavchik writes: 
> 
>> * A single header line exceeds 5000 characters.
> 
>> These limits are perfectly reasonable.  If you're hitting it, you have
>> something else that's broken. 
>>
> It's right with one exception: due to minor logical error in submit.C or
> intentionally 5K length limit is applied not only to the single header line

It's intentional.  Nobody has any business sending mail headers that are 
more than 5000 bytes long, wrapped or unwrapped. 


-- 
Sam

Doug Rintoul writes:

> I was able to pull in the message with another email client and examine the 
> message. It looks like the To: header is too long. The person sending the 
> message sent it to 110 recipients and the total byte count is 5369. On 
> average, each recipient's email address is 49 characters. This is because a 
> long "friendly name" is included with each recipient's address. Question is, 
> is this my problem or the sender's problem? What do I tell my clients? Keep 
> their mailing lists below a certain number?

This mailing list has about 700 subscribers.  You don't see every one of 
them listed in the To: header. 

>                                              It is not unusual to have my 
> users send to two or three hundred people at a time, especially if it is a 
> news letter.

Then they can use a real mailing list manager, that does this right, instead 
of winging it on their own. 

-- 
Sam

Sam Varshavchik writes:
>
> It's intentional.  Nobody has any business sending mail headers that are
> more than 5000 bytes long, wrapped or unwrapped.
>
But they DO sending them, and they don't want to be innocent victims of my
mail server. There is very much people who never heard about mailing lists
but nevertheless need to send multiple recipient messages and used to send
them this way.

Same as other aspects of Courier's intolerance this one _must_ be
configurable.

Alexei.

On Tuesday 13 March 2001 04:57 pm, you wrote:
> Sam Varshavchik writes:
> > * A single header line exceeds 5000 characters.
> >
> > These limits are perfectly reasonable.  If you're hitting it, you have
> > something else that's broken.
>
> It's right with one exception: due to minor logical error in submit.C or
> intentionally 5K length limit is applied not only to the single header line
> but to the single header with properly wrapped (beginning with space or
> tab) lines too. It leads to the bounces of absolutely RFC-compliant
> messages with relatively short (< 10K) multiple recipient To: or Cc:
> headers.
>
> For the time being I've applied attached patch but I'm not absolutely sure
> that it will not entail any undesirable consequences, because single header
> can now reach "headerlimit" (100000 bytes) length. Maybe Sam will suggest
> more correct solution.
>
> Alexei.

This is absolutely what I saw. Thanks for the patch. The email in question 
had 110 recipients and the total size of the to header was about 5300 bytes. 

Doug.

Alexei Batyr' writes:

> Sam Varshavchik writes:
>>
>> It's intentional.  Nobody has any business sending mail headers that are
>> more than 5000 bytes long, wrapped or unwrapped. 
>>
> But they DO sending them, and they don't want to be innocent victims of my
> mail server. There is very much people who never heard about mailing lists
> but nevertheless need to send multiple recipient messages and used to send
> them this way.

Mail relays were also used to be completely open.  Things don't work like 
that anymore. 

And it's not like their mail disappears.  They get an informative bounce, 
giving them a chance to do it right.  At the very least - have them BCC the 
mail, so nobody had to plow through all the garbage in the headers. 

-- 
Sam

On Wednesday 14 March 2001 06:29 am, you wrote:

> Alexei Batyr' writes:
> > Sam Varshavchik writes:
> >> It's intentional.  Nobody has any business sending mail headers that are
> >> more than 5000 bytes long, wrapped or unwrapped.
> >
> > But they DO sending them, and they don't want to be innocent victims of
> > my mail server. There is very much people who never heard about mailing
> > lists but nevertheless need to send multiple recipient messages and used
> > to send them this way.
>
> Mail relays were also used to be completely open.  Things don't work like
> that anymore.
>
> And it's not like their mail disappears.  They get an informative bounce,
> giving them a chance to do it right.  At the very least - have them BCC the
> mail, so nobody had to plow through all the garbage in the headers.

Sam,

I would question that the bounce is informative. Many users have no idea what 
a message header is or what they could have done to make it exceed the policy 
limit. No information is in the bounce message as to what the limit is and 
how they can reduce it. I know we can't be pumping back detailed explanations 
every time something goes wrong. But in this case you listed for me three 
possibilities. The sender has no idea what those possibilities might be nor 
how to diagnose them. Something more informative might be helpful, like
"The To recipient list exceeds policy limit". Or is this message determined 
by some RFC?

Thanks so much for your work and for your quick responses. I have been  
impressed by the time and effort you put into this list and in maintaining 
Courier.

Doug.

Doug Rintoul writes:

> how they can reduce it. I know we can't be pumping back detailed explanations 
> every time something goes wrong. But in this case you listed for me three 
> possibilities. The sender has no idea what those possibilities might be nor 
> how to diagnose them. Something more informative might be helpful, like
> "The To recipient list exceeds policy limit". Or is this message determined 
> by some RFC?

Mail headers are not parsed to that extent.  This error message is reported 
when any header exceeds the maximum allowed size, be it a To: header, a Cc: 
header, or a Received: header. 

I'll change the error message to state "Message header size or recipient 
list exceeds the policy limit." 


-- 
Sam

On Wednesday 14 March 2001 10:22 am, you wrote:

> Doug Rintoul writes:
> > how they can reduce it. I know we can't be pumping back detailed
> > explanations every time something goes wrong. But in this case you listed
> > for me three possibilities. The sender has no idea what those
> > possibilities might be nor how to diagnose them. Something more
> > informative might be helpful, like "The To recipient list exceeds policy
> > limit". Or is this message determined by some RFC?
>
> Mail headers are not parsed to that extent.  This error message is reported
> when any header exceeds the maximum allowed size, be it a To: header, a Cc:
> header, or a Received: header.
>
> I'll change the error message to state "Message header size or recipient
> list exceeds the policy limit."

Sam,

That is definately clearer. Thanks.

Doug.

Sam,

trust me, I like multiline and multikilobyte To or Cc headers maybe even
less than you. But it's not a personal preference issue - as a corporate
sysadmin (of course in extreme but theoretically possible case) I could be
fired if some e-mails from some one big guy to a bunch of some other ones
will be systematically bounced. These guys know nothing and don't want to
know anything not only about mailing lists, but even about the fact that
e-mail message consists of headers and body.

Even to force them using Bcc instead of To I need to enter into
communication with their computer assistants etc. It's too time consuming
and not always possible. That's why I need to accept all that crap
(including non-RFC-compliant messages etc.) and would be very appreciated if
you could give us (I'm pretty sure it's not only my problem) some mechanism
for receiving and filtering all wrong messages. In this case bounces could
be much more informative, individualised and written in understandable for
senders language (unfortunately so far not a lot of Russian people can read
English messages).

For now I suggest to add to "header" bounce text something like that:
"If you have sent multiple recipient message, please move recipient list to
the Bcc: header field"

Alexei.

We are using courier-imap in a PHP webmail application.   The PHP IMAP 
module is used to access courier-imap.

Since the PHP IMAP module does not support a persistent connection, it must 
make a new connection, login, authenticate, and disconnect each time needs 
to access the mailbox files.

I've run a few tests and it appears the authentication step is the 
bottleneck in our system and limits how many connections/second I can 
make.  I could be wrong about this, but I want to disable authentication 
and run the same tests to see if it makes a difference.

Since the webmail users are already authenticated, doing another 
authentication with IMAP is not necessary.

I've started to hack through the authlib stuff in an attempt to figure out 
how to bypass the authorization stuff.  However, I'm wondering if there is 
a simple way to do this that I am missing.

Thanks,

Jeff Groves

On Wed, 14 Mar 2001 02:22:30 GMT, Sam Varshavchik
<mrsam@...> wrote:

>I'll change the error message to state "Message header size or recipient 
>list exceeds the policy limit." 

If I cannot control this policy to meet my own ideas of how things
should work, without needing to apply patches because the author is
opposed to certain ideas, I will be less inclined to invest time in
learning enough about this software to install and use it.

Egan

Egan wrote:
> 
> If I cannot control this policy to meet my own ideas of how things
> should work, without needing to apply patches because the author is
> opposed to certain ideas, I will be less inclined to invest time in
> learning enough about this software to install and use it.

Hey, it's free software for christ's sake. If you don't like the way it
is, then hack the code yourself, dammit. You're not in a position to ask
anyone for anything if you're not contributing anything yourself.

By the way, there's lot of alternatives. Exim, qmail, postfix, sendmail
etc. Make a choice from your own preferences, but don't bully the
authors for what they believe in.

Tomas

On Thu, 15 Mar 2001 20:31:47 +0100, Tomas Fasth <tomas@...>
wrote:

>By the way, there's lot of alternatives. Exim, qmail, postfix, sendmail
>etc. Make a choice from your own preferences, but don't bully the
>authors for what they believe in.

Few people will spend time coding software unless they expect people
to actually use it.  Otherwise, why bother?

Authors who want more people to use their software will accommodate
reasonable requests, instead of taking arbitrary, unyielding positions
and defending them to the death.

If a maximum header size must be set at some arbitrary value, then why
not 50,000 instead of 5,000?  That would put the limit high enough to
avoid the type of problems users are reporting.

No one is likely to have a recipient list exceeding 50,000 characters,
but it's easy to see how they could exceed 5,000.


>Hey, it's free software for christ's sake. If you don't like the way it
>is, then hack the code yourself

Why not just go all the way and fork the code while we're at it, if we
can't be flexible?


>You're not in a position to ask anyone for anything if you're not
>contributing anything yourself.

Thanks for the free advice.


Egan