fetchmail-devel — Development related discussion

Re: [fetchmail-devel] fetchmail socket problem

[Rob M. <rob...@gm...>]

On 10/28/07, aalap shah <aal...@gm...> wrote:
> hey sorry could not reply
> hey thanks for your help

However, for your continued failure to keep traffic on the list, I
will not be responding to any future emails from your email address.

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] fetchmail socket problem

[Rob M. <rob...@gm...>]

If you're incapable of replying to the list I will stop responding.

On 10/26/07, aalap shah <aal...@gm...> wrote:
> hey thanks really i installed fetchmail 6.3.8 and it all wroked
> one thing how can make my fetchmail not to send the mails on localhost
> port 25 i just want it to store all the mails in a file

See the MDA option in the man page, particularly the warnings.

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] fetchmail socket problem

[Rob M. <rob...@gm...>]

Keep the traffic on the list, and next time use the -users list.

On 10/26/07, Aalap Shah <aal...@gm...> wrote:
> hi, as u requested the output of
>  fetchmail --nosyslog --nodetach -vvv
> is
>
> darshak@darshak-desktop:~$ fetchmail --nosyslog --nodetach -vvv
> Enter password for aal...@po...:
> fetchmail: starting fetchmail 6.3.7 daemon

As I said, upgrade to 6.3.8

> fetchmail: 6.3.7 querying pop.gmail.com (protocol POP3) at Friday 26
> October 2007 01:50:48 PM IST: poll started
> Trying to connect to 66.249.91.109/110...connection failed.
> fetchmail: connection to pop.gmail.com:pop3 [66.249.91.109/110] failed:
> Connection timed out.

That'll never work - GMail provides POP3S, not POP3:

https://mail.google.com/support/bin/answer.py?answer=13287

<---SNIP--->
> it hangs after this i waited for 10  minutes but no outcome
> and the contents of my .fetchmailrc file is
>
> # Configuration created Thu Oct 25 01:31:45 2007 by fetchmailconf 1.52
> $Revision: 4740 $
> set postmaster "darshak-desktop"
> set bouncemail
> set no spambounce
> set properties ""
> set daemon 600
> poll pop.gmail.com with proto pop3 and options no dns
>         user 'aalap.shh' is 'darshak-desktop'

Add "ssl" to the last line.

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] fetchmail socket problem

[Rob M. <rob...@gm...>]

On 10/25/07, aalap shah <aal...@gm...> wrote:
> hi I am using fetchmail  to fetch mails from gmail account but I am facing 2
> problems.

This would be far more appropriate to the -users mailing list.

> The first one is that when i  try to do
> openssl s_client -connect smtp.gmail.com:995 -showcerts
> it shows
>  +OK Gpop ready for requests from 220.224.119.42 c25pf1139902ika
> where in when i saw certain configuration pages it shows that i should get
> something like
> +OK Gpop i16pf3941185wxd ready.
> I dont understand what is this

This is the greeting from the GMail POP service.

> and secondly I tried connecting to pop.gmail.com using fetchmail i got
> somekind of socket error no 2, I also created the certificates and edited
> the .fetchmailrc file . The error it shows is
>
> fetchmail: 6.3.7 querying pop.gmail.com (protocol POP3) at Thursday 25

Upgrade to 6.3.8.

> October 2007 11:32:29 PM IST: poll started
> Trying to connect to 66.249.91.109/995...connected.
> fetchmail: socket error while fetching from aal...@po...
> fetchmail: 6.3.7 querying pop.gmail.com (protocol POP3) at Thursday 25
> October 2007 11:33:37 PM IST: poll completed
> fetchmail: Query status=2 (SOCKET)
> fetchmail: normal termination, status 2

Socket errors (as has been discussed many times in the past) relate to
a communications failure with the remote host.

> please help
>
> the strace output after i input my password is

As specified in the FAQ under reporting a bug
(http://www.fetchmail.info/fetchmail-FAQ.html#G3) please provide the
output of:

fetchmail --nosyslog --nodetach -vvv

The content of your .fetchmailrc would also help.

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

[fetchmail-devel] fetchmail socket problem

[aalap s. <aal...@gm...>]

hi I am using fetchmail  to fetch mails from gmail account but I am facing 2
problems. The first one is that when i  try to do
openssl s_client -connect smtp.gmail.com:995 -showcerts
it shows
+OK Gpop ready for requests from 220.224.119.42 c25pf1139902ika
where in when i saw certain configuration pages it shows that i should get
something like
+OK Gpop i16pf3941185wxd ready.
I dont understand what is this

and secondly I tried connecting to pop.gmail.com using fetchmail i got
somekind of socket error no 2, I also created the certificates and edited
the .fetchmailrc file . The error it shows is

fetchmail: 6.3.7 querying pop.gmail.com (protocol POP3) at Thursday 25
October 2007 11:32:29 PM IST: poll started
Trying to connect to 66.249.91.109/995...connected.
fetchmail: socket error while fetching from aal...@po...
fetchmail: 6.3.7 querying pop.gmail.com (protocol POP3) at Thursday 25
October 2007 11:33:37 PM IST: poll completed
fetchmail: Query status=2 (SOCKET)
fetchmail: normal termination, status 2

please help

the strace output after i input my password is

socket(PF_INET, SOCK_STREAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(995), sin_addr=inet_addr("
66.249.91.109")}, 16) = 0
write(1, "Trying to connect to 66.249.91.1"..., 52Trying to connect to
66.249.91.109/995...connected.
) = 52
setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={300, 0}}, NULL) = 0
setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={0, 0}}, NULL) = 0
setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={300, 0}}, NULL) = 0
recv(3, "", 512, MSG_PEEK)              = 0
setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={0, 0}}, NULL) = 0
rt_sigaction(SIGALRM, {0x8054be0, [], 0}, {0x8054ff0, [], 0}, 8) = 0
setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={60, 0}}, NULL) = 0
close(3)                                = 0
setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={0, 0}}, NULL) = 0
rt_sigaction(SIGALRM, {0x8054ff0, [], 0}, {0x8054be0, [], 0}, 8) = 0
write(2, "fetchmail: ", 11fetchmail: )             = 11
write(2, "socket error while fetching from"..., 57socket error while
fetching from aal...@po...
) = 57
setitimer(ITIMER_REAL, {it_interval={0, 0}, it_value={0, 0}}, NULL) = 0
rt_sigaction(SIGALRM, {0x804e890, [], 0}, {0x8054ff0, [], 0}, 8) = 0
time(NULL)                              = 1193336722
stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=109, ...}) = 0
stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=109, ...}) = 0
stat64("/etc/localtime", {st_mode=S_IFREG|0644, st_size=109, ...}) = 0
write(1, "fetchmail: 6.3.7 querying pop.gm"..., 116fetchmail: 6.3.7 querying
pop.gmail.com (protocol POP3) at Thursday 25 October 2007 11:55:22 PM IST:
poll completed
) = 116
write(1, "fetchmail: Query status=2 (SOCKE"..., 35fetchmail: Query status=2
(SOCKET)
) = 35
unlink("/home/darshak/.fetchids")       = -1 ENOENT (No such file or
directory)
write(1, "fetchmail: normal termination, s"..., 40fetchmail: normal
termination, status 2
) = 40
unlink("/home/darshak/.fetchids")       = -1 ENOENT (No such file or
directory)
unlink("/home/darshak/.fetchmail.pid")  = 0
exit_group(2)                           = ?
Process 10133 detached

Re: [fetchmail-devel] SASL PLAIN support

[Petri A. <pa...@sc...>]

I only write some php/shell/whatever scripts so I cant help you with
libgsasl, sorry. hope you find programmers for sasl-project.

> >
> > Not your responsibility, but: What advantage does SASL PLAIN have over
> > USER/PASS? I see none, so who configures POP3 servers that way and why?
> >

I agree, but I cant configure servers out there.  ;)

Thanks for information and specialy maintaining this usefull tool for us,

Petri

Re: [fetchmail-devel] SASL PLAIN support

[Matthias A. <mat...@gm...>]

Petri Asikainen schrieb am 2007-10-13:

> This is more user question than developer, but technical so sent it to
> devel list:
> 
> I cant find anything about SASL PLAIN authentication method from
> fetchmail man pages. I also tried to search sourcecode but cant find
> anything.
> Is there "SASL PLAIN" authentication support in fetchmail?
> (http://www.ietf.org/rfc/rfc4616.txt)

Not at the moment - sorry.

I'm considering libgsasl support for the next major release though, but
that's unlikely to happen before well into 2008 as it requires major
internal restructuring and re-design of some parts of the system (unless
a full rewrite from scratch turns out to be less work that is).

A few daring and skilled programmers might speed things up though if a
good team effort can be achieved and they are willing to join an initial
planning phase.

I might talk about the plans a bit if there's sufficient interest.

> That would be nice and relly _usefull_ to include it. Some times (pop3)
> servers are configured to accept only SASL PLAIN and prevent USER/PASS
> authentication.

Not your responsibility, but: What advantage does SASL PLAIN have over
USER/PASS? I see none, so who configures POP3 servers that way and why?

Best regards

-- 
Matthias Andree

[fetchmail-devel] SASL PLAIN support

[Petri A. <pa...@sc...>]

This is more user question than developer, but technical so sent it to
devel list:

I cant find anything about SASL PLAIN authentication method from
fetchmail man pages. I also tried to search sourcecode but cant find
anything.
Is there "SASL PLAIN" authentication support in fetchmail?
(http://www.ietf.org/rfc/rfc4616.txt)

That would be nice and relly _usefull_ to include it. Some times (pop3)
servers are configured to accept only SASL PLAIN and prevent USER/PASS
authentication.

Cheers,

Petri Asikainen

Re: [fetchmail-devel] [Bug #11980] fetchmail hangs (imap4rev1) with MSExchange empty body

[KB S. <ma...@ya...>]

While a better fix is being investigated, in case someone else
is in the same boat, the following crude change
to imap.c WorksForMe(TM.)

-kb



       
____________________________________________________________________________________
Got a little couch potato? 
Check out fun summer activities for kids.
http://search.yahoo.com/search?fr=oni_on_mail&p=summer+activities+for+kids&cs=bz 

Re: [fetchmail-devel] [Bug #11980] fetchmail hangs (imap4rev1) with MSExchange empty body

[KB S. <ma...@ya...>]

Apologies -- I made a mistake in my original bug submit on the location
of the problem.

Attached is a gdb stack trace when fetchmail hangs (with some email
headers edited), hope that helps.

(gdb) r --nodetach --nosyslog -vvv
Starting program: /home/kbs/extsrc/fetchmail-6.3.8/fetchmail --nodetach
--nosyslog -vvv
Failed to read a valid object file image from memory.
fetchmail: starting fetchmail 6.3.8 daemon 
fetchmail: 6.3.8 querying Xx.zz (protocol IMAP) at Fri 14 Sep 2007 02:07:08 PM
PDT: poll started
Trying to connect to 16.232.83.0/993...connected.
fetchmail: Issuer Organization: RSA Data Security, Inc.
fetchmail: Unknown Issuer CommonName
fetchmail: Server CommonName: xx.zz
fetchmail: Subject Alternative Name: xx.zz
fetchmail: Subject Alternative Name: xx
fetchmail: Subject Alternative Name: aa
fetchmail: Subject Alternative Name: aa
fetchmail: Xx.zz key fingerprint:
72:6A:90:DB:D7:E9:64:B0:6E:FC:26:E4:F0:55:9B:5C
fetchmail: IMAP< * OK Microsoft Exchange Server 2007 IMAP4 service ready
fetchmail: IMAP> A0001 CAPABILITY
fetchmail: IMAP< * CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI IDLE
NAMESPACE LITERAL+
fetchmail: IMAP< A0001 OK CAPABILITY completed.
fetchmail: Protocol identified as IMAP4 rev 1
fetchmail: IMAP> A0002 AUTHENTICATE NTLM
fetchmail: IMAP< +
NTLM Request:
      Ident = NTLMSSP
      mType = 1
      Flags = 0000b207
       User = nnn
     Domain = zzz
fetchmail: IMAP>
TlRMTVNTUAABAAAAB7IAAAgACAAgAAAACAAIACgAAABzcmlyYW1za2FtZXJpY2Fz
fetchmail: IMAP< +
TlRMTVNTUAACAAAAEAAQADgAAAAFgoECr1SyxvugvkgAAAAAAAAAAKwArABIAAAABQLODgAAAA9BAE0ARQBSAEkAQwBBAFMAAgAQAEEATQBFAFIASQBDAEEAUwABAA4ARwAzAFcAMQAwADQANAAEACgAYQBtAGUAcgBpAGMAYQBzAC4AYwBwAHEAYwBvAHIAcAAuAG4AZQB0AAMAOABHADMAVwAxADAANAA0AC4AYQBtAGUAcgBpAGMAYQBzAC4AaABwAHEAYwBvAHIAcAAuAG4AZQB0AAUAFgBjAHAAcQBjAG8AcgBwAC4AbgBlAHQAAAAAAA==
NTLM Challenge:
      Ident = NTLMSSP
      mType = 2
     Domain = ZZZ
      Flags = 02818205
  Challenge = af 54 b2 c6 fb a0 be 48 
NTLM Response:
      Ident = NTLMSSP
      mType = 3
     LmResp = 0b 57 e0 45 66 7a e6 9b 74 bf 29 41 97 66 de 11 30 c9 8b dc 44 5d
2f 49 
     NTResp = b6 a8 61 df 09 c4 19 80 49 52 5e c2 76 a1 37 ee e9 7a 81 b4 73 d3
2f ae 
     Domain = zzz
       User = nnn
        Wks = nnn
       sKey = 
      Flags = 02818205
fetchmail: IMAP>
TlRMTVNTUAADAAAAGAAYAEAAAAAYABgAWAAAABAAEABwAAAAEAAQAIAAAAAQABAAkAAAAAAAAABgAAAABYKBAgtX4EVmeuabdL8pQZdm3hEwyYvcRF0vSbaoYd8JxBmASVJewnahN+7peoG0c9MvrmEAbQBlAHIAaQBjAGEAcwBzAHIAaQByAGEAbQBzAGsAcwByAGkAcgBhAG0AcwBrAA==
fetchmail: IMAP< A0002 OK AUTHENTICATE completed.
fetchmail: selecting or re-polling default folder
fetchmail: IMAP> A0003 SELECT "INBOX"
fetchmail: IMAP< * 13 EXISTS
fetchmail: IMAP< * 0 RECENT
fetchmail: IMAP< * FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
fetchmail: IMAP< * OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted
 \Draft
$MDNSent)] Permanent flags
fetchmail: IMAP< * OK [UNSEEN 1] Is the first unseen message
fetchmail: IMAP< * OK [UIDVALIDITY 2529] UIDVALIDITY value
fetchmail: IMAP< * OK [UIDNEXT 32334] The next unique identifier value
fetchmail: IMAP< A0003 OK [READ-WRITE] SELECT completed.
fetchmail: 13 messages waiting after first poll
fetchmail: IMAP> A0004 EXPUNGE
fetchmail: IMAP< A0004 OK EXPUNGE completed.
fetchmail: 13 messages waiting after expunge
fetchmail: IMAP> A0005 SEARCH UNSEEN NOT DELETED
fetchmail: IMAP< * SEARCH 1 2 3 4 5 6 7 8 9 10 11 12 13
fetchmail: 1 is unseen
fetchmail: 2 is unseen
fetchmail: 3 is unseen
fetchmail: 4 is unseen
fetchmail: 5 is unseen
fetchmail: 6 is unseen
fetchmail: 7 is unseen
fetchmail: 8 is unseen
fetchmail: 9 is unseen
fetchmail: 10 is unseen
fetchmail: 11 is unseen
fetchmail: 12 is unseen
fetchmail: 13 is unseen
fetchmail: IMAP< A0005 OK SEARCH completed.
fetchmail: 1 is first unseen
13 messages for nnn@zzz at Xx.zz.
fetchmail: IMAP> A0006 FETCH 1:13 RFC822.SIZE
fetchmail: IMAP< * 1 FETCH (RFC822.SIZE 1046)
fetchmail: IMAP< * 2 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 3 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 4 FETCH (RFC822.SIZE 2408)
fetchmail: IMAP< * 5 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 6 FETCH (RFC822.SIZE 12634)
fetchmail: IMAP< * 7 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 8 FETCH (RFC822.SIZE 2009)
fetchmail: IMAP< * 9 FETCH (RFC822.SIZE 3815)
fetchmail: IMAP< * 10 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 11 FETCH (RFC822.SIZE 8798)
fetchmail: IMAP< * 12 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 13 FETCH (RFC822.SIZE 10551)
fetchmail: IMAP< A0006 OK FETCH completed.
fetchmail: IMAP> A0007 FETCH 1 RFC822.HEADER
fetchmail: IMAP< * 1 FETCH (RFC822.HEADER {964}
reading message nnn@zzz@Xx.zz:1 of 13 (964 header octets)
About to rewrite From: zzz
Rewritten version is From: zzz

About to rewrite To: xxx
Rewritten version is To: xxx

Trying to connect to 127.0.0.1/25...connected.
fetchmail: SMTP< 220 kbs-deskbottom.zz ESMTP Postfix (Debian/GNU)
fetchmail: SMTP> EHLO kbs-deskbottom.zz
fetchmail: SMTP< 250-kbs-deskbottom.zz
fetchmail: SMTP< 250-PIPELINING
fetchmail: SMTP< 250-SIZE 10240000
fetchmail: SMTP< 250-VRFY
fetchmail: SMTP< 250-ETRN
fetchmail: SMTP< 250-STARTTLS
fetchmail: SMTP< 250-ENHANCEDSTATUSCODES
fetchmail: SMTP< 250-8BITMIME
fetchmail: SMTP< 250 DSN
fetchmail: forwarding to localhost
fetchmail: SMTP> MAIL FROM:<zzz> BODY=8BITMIME SIZE=1046
fetchmail: SMTP< 250 2.1.0 Ok
fetchmail: SMTP> RCPT TO:<kbs@localhost>
fetchmail: SMTP< 250 2.1.5 Ok
fetchmail: SMTP> DATA
fetchmail: SMTP< 354 End data with <CR><LF>.<CR><LF>
#
fetchmail: IMAP< )
fetchmail: IMAP< A0007 OK FETCH completed.
fetchmail: IMAP> A0008 FETCH 1 BODY.PEEK[TEXT]
fetchmail: IMAP< * 1 FETCH (BODY[TEXT] NIL)
 (82 body octets)*
Program received signal SIGINT, Interrupt.
0xb7f46410 in ?? ()
(gdb) where
#0  0xb7f46410 in ?? ()
#1  0xbfc94438 in ?? ()
#2  0x00000005 in ?? ()
#3  0x080a71b0 in ?? ()
#4  0xb7d01783 in read () from /lib/tls/i686/cmov/libc.so.6
#5  0xb7e0ce9a in BIO_new_socket () from /usr/lib/i686/cmov/libcrypto.so.0.9.8
#6  0xb7e0af21 in BIO_read () from /usr/lib/i686/cmov/libcrypto.so.0.9.8
#7  0xb7ecb6d3 in ssl3_read_n () from /usr/lib/i686/cmov/libssl.so.0.9.8
#8  0xb7ecbc97 in ssl3_read_bytes () from /usr/lib/i686/cmov/libssl.so.0.9.8
#9  0xb7ec8789 in ssl3_peek () from /usr/lib/i686/cmov/libssl.so.0.9.8
#10 0xb7ed904b in SSL_peek () from /usr/lib/i686/cmov/libssl.so.0.9.8
#11 0x0804baa3 in SockRead (sock=6, 
    buf=0xbfc94618 "A0008 OK FETCH completed.\r\n", len=8191) at socket.c:456
#12 0x0805a074 in readbody (sock=6, ctl=0x8095650, forward=1 '\001', len=55)
    at transact.c:1365
#13 0x080558ce in fetch_messages (mailserver_socket=6, ctl=0x8095650, 
    count=13, msgsizes=0x8086d08, maxfetch=0, fetches=0xbfc9a734, 
    dispatches=0xbfc9a730, deletions=0xbfc9a744) at driver.c:686
#14 0x08056f0a in do_session (ctl=0x8095650, proto=0x8081f80, maxfetch=0)
    at driver.c:1415
#15 0x0805750a in do_protocol (ctl=0x8095650, proto=0x8081f80)
    at driver.c:1626
#16 0x0806cc49 in doIMAP (ctl=0x8095650) at imap.c:1302
#17 0x080505e9 in query_host (ctl=0x8095650) at fetchmail.c:1481
#18 0x0804e0ed in main (argc=4, argv=0xbfc9a9d4) at fetchmail.c:740



      ____________________________________________________________________________________
Fussy? Opinionated? Impossible to please? Perfect.  Join Yahoo!'s user panel and lay it on us. http://surveylink.yahoo.com/gmrs/yahoo_panel_invite.asp?a=7 

[fetchmail-devel] [Bug #11980] fetchmail hangs (imap4rev1) with MSExchange empty body

[<ad...@be...>]

Bug #11980, was updated on 2007-Sep-12 15:49
Here is a current snapshot of the bug.

Project: Community Fetchmail
Category: None
Status: Open
Resolution: None
Bug Group: None
Priority: 5
Submitted by: kbs
Assigned to : none
Summary: fetchmail hangs (imap4rev1) with MSExchange empty body

Details: The MSExchange IMAP server sometimes returns a protocol
line reading

fetchmail: IMAP< * 1 FETCH (BODY[TEXT] NIL)

for empty body messages.

This causes imap.c to hang in this section of the code:
[...]
    /* looking for FETCH response */
    do {
	int	ok;

	if ((ok = gen_recv(sock, buf, sizeof(buf))))
	    return(ok);
    } while
	(!strstr(buf+4, "FETCH") || sscanf(buf+2, "%d", &num) != 1);
[...]

as it appears to always expect to find a number.

Here are more details, I've stripped off some of the
emails and host names etc.

kbs@kbs-deskbottom:/tmp$ fetchmail -V
This is fetchmail release 6.3.6+NTLM+SDPS+SSL+NLS.

Copyright (C) 2002, 2003 Eric S. Raymond
Copyright (C) 2004 Matthias Andree, Eric S. Raymond, Rob F. Funk, Graham Wilson
Copyright (C) 2005-2006 Matthias Andree, Sunil Shetye
Fetchmail comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. For details,
please see the file COPYING in the source or documentation directory.

Fallback MDA: (none)
Linux kbs-deskbottom 2.6.18-5-686 #1 SMP Thu Aug 30 02:19:07 UTC 2007 i686 GNU/Linux
Taking options from command line and /home/kbs/.fetchmailrc
Poll interval is 120 seconds
Idfile is /home/kbs/.fetchids
Fetchmail will forward misaddressed multidrop messages to kbs.
Options for retrieving from XXX:
  True name of server is XX
  Protocol is IMAP.
  All available authentication methods will be tried.
  SSL encrypted sessions enabled.
  Server nonresponse timeout is 60 seconds.
  Default mailbox selected.
  Only new messages will be retrieved (--all off).
  Fetched messages will not be kept on the server (--keep off).
  Old messages will not be flushed before message retrieval (--flush off).
  Oversized messages will not be flushed before message retrieval (--limitflush off).
  Rewrite of server-local addresses is enabled (--norewrite off).
  Carriage-return stripping is disabled (stripcr off).
  Carriage-return forcing is disabled (forcecr off).
  Interpretation of Content-Transfer-Encoding is enabled (pass8bits off).
  MIME decoding is disabled (mimedecode off).
  Idle after poll is disabled (idle off).
  Nonempty Status lines will be kept (dropstatus off)
  Delivered-To lines will be kept (dropdelivered off)
  Fetch message size limit is 100 (--fetchsizelimit 100).
  Do binary search of UIDs during 3 out of 4 polls (--fastuidl 4).
  Messages will be SMTP-forwarded to: localhost (default)
  Single-drop mode: 1 local name recognized.
  No UIDs saved from this host.


Output from fetchmail --nosyslog --nodetach -vvv 

fetchmail: starting fetchmail 6.3.6 daemon 
fetchmail: 6.3.6 querying XX (protocol IMAP) at Wed 12 Sep 2007 02:40:32 PM PDT: poll started
Trying to connect to 16.232.83.0/993...connected.
fetchmail: Issuer Organization: RSA Data Security, Inc.
fetchmail: Unknown Issuer CommonName
fetchmail: Server CommonName: xx
fetchmail: Subject Alternative Name: xx
fetchmail: Subject Alternative Name: xx
fetchmail: Subject Alternative Name: xx
fetchmail: Subject Alternative Name: xx
fetchmail: XX key fingerprint: 72:6A:90:DB:D7:E9:64:B0:6E:FC:26:E4:F0:55:9B:5C
fetchmail: IMAP< * OK Microsoft Exchange Server 2007 IMAP4 service ready
fetchmail: IMAP> A0001 CAPABILITY
fetchmail: IMAP< * CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI IDLE NAMESPACE LITERAL+
fetchmail: IMAP< A0001 OK CAPABILITY completed.
fetchmail: Protocol identified as IMAP4 rev 1
fetchmail: IMAP> A0002 AUTHENTICATE NTLM
fetchmail: IMAP< +
NTLM Request:
      Ident = NTLMSSP
      mType = 1
      Flags = 0000b207
       User = xx
     Domain = xx
fetchmail: IMAP> TlRMTVNTUAABAAAAB7IAAAgACAAgAAAACAAIACgAAABzcmlyYW1za2FtZXJpY2Fz
fetchmail: IMAP< + TlRMTVNTUAACAAAAEAAQADgAAAAFgoECvWaPexRO6vgAAAAAAAAAAKwArABIAAAABQLODgAAAA9BAE0ARQBSAEkAQwBBAFMAAgAQAEEATQBFAFIASQBDAEEAUwABAA4ARwAzAFcAMQAwADQANAAEACgAYQBtAGUAcgBpAGMAYQBzAC4AYwBwAHEAYwBvAHIAcAAuAG4AZQB0AAMAOABHADMAVwAxADAANAA0AC4AYQBtAGUAcgBpAGMAYQBzAC4AaABwAHEAYwBvAHIAcAAuAG4AZQB0AAUAFgBjAHAAcQBjAG8AcgBwAC4AbgBlAHQAAAAAAA==
NTLM Challenge:
      Ident = NTLMSSP
      mType = 2
     Domain = xx
      Flags = 02818205
  Challenge = bd 66 8f 7b 14 4e ea f8 
NTLM Response:
      Ident = NTLMSSP
      mType = 3
     LmResp = c4 b3 6d 24 c8 77 dc 53 c5 3c cb cd 0b 96 41 d1 2b 3e f5 92 e7 13 8a c5 
     NTResp = 6f 8c f4 a7 57 b9 91 9f b9 dc 60 21 ac 57 79 89 64 3f a1 41 27 89 e2 af 
     Domain = xx
       User = xx
        Wks = xx
       sKey = 
      Flags = 02818205
fetchmail: IMAP> TlRMTVNTUAADAAAAGAAYAEAAAAAYABgAWAAAABAAEABwAAAAEAAQAIAAAAAQABAAkAAAAAAAAABgAAAABYKBAsSzbSTId9xTxTzLzQuWQdErPvWS5xOKxW+M9KdXuZGfudxgIaxXeYlkP6FBJ4nir2EAbQBlAHIAaQBjAGEAcwBzAHIAaQByAGEAbQBzAGsAcwByAGkAcgBhAG0AcwBrAA==
fetchmail: IMAP< A0002 OK AUTHENTICATE completed.
fetchmail: selecting or re-polling default folder
fetchmail: IMAP> A0003 SELECT "INBOX"
fetchmail: IMAP< * 9 EXISTS
fetchmail: IMAP< * 0 RECENT
fetchmail: IMAP< * FLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)
fetchmail: IMAP< * OK [PERMANENTFLAGS (\Seen \Answered \Flagged \Deleted \Draft $MDNSent)] Permanent flags
fetchmail: IMAP< * OK [UNSEEN 1] Is the first unseen message
fetchmail: IMAP< * OK [UIDVALIDITY 2529] UIDVALIDITY value
fetchmail: IMAP< * OK [UIDNEXT 32216] The next unique identifier value
fetchmail: IMAP< A0003 OK [READ-WRITE] SELECT completed.
fetchmail: 9 messages waiting after first poll
fetchmail: IMAP> A0004 EXPUNGE
fetchmail: IMAP< A0004 OK EXPUNGE completed.
fetchmail: 9 messages waiting after expunge
fetchmail: IMAP> A0005 SEARCH UNSEEN NOT DELETED
fetchmail: IMAP< * SEARCH 1 2 3 4 5 6 7 8 9
fetchmail: 1 is unseen
fetchmail: 2 is unseen
fetchmail: 3 is unseen
fetchmail: 4 is unseen
fetchmail: 5 is unseen
fetchmail: 6 is unseen
fetchmail: 7 is unseen
fetchmail: 8 is unseen
fetchmail: 9 is unseen
fetchmail: IMAP< A0005 OK SEARCH completed.
fetchmail: 1 is first unseen
9 messages for xx@xx at XX.
fetchmail: IMAP> A0006 FETCH 1:9 RFC822.SIZE
fetchmail: IMAP< * 1 FETCH (RFC822.SIZE 1046)
fetchmail: IMAP< * 2 FETCH (RFC822.SIZE 897)
fetchmail: IMAP< * 3 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 4 FETCH (RFC822.SIZE 860)
fetchmail: IMAP< * 5 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 6 FETCH (RFC822.SIZE 1699)
fetchmail: IMAP< * 7 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 8 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< * 9 FETCH (RFC822.SIZE 555)
fetchmail: IMAP< A0006 OK FETCH completed.
fetchmail: IMAP> A0007 FETCH 1 RFC822.HEADER
fetchmail: IMAP< * 1 FETCH (RFC822.HEADER {964}
reading message xx@xx@XX:1 of 9 (964 header octets)
About to rewrite From: yy

Rewritten version is From: yy


About to rewrite To: zz

Rewritten version is To: zz


Trying to connect to 127.0.0.1/25...connected.
fetchmail: SMTP< 220 aa ESMTP Postfix (Debian/GNU)
fetchmail: SMTP> EHLO aa
fetchmail: SMTP< 250-aa
fetchmail: SMTP< 250-PIPELINING
fetchmail: SMTP< 250-SIZE 10240000
fetchmail: SMTP< 250-VRFY
fetchmail: SMTP< 250-ETRN
fetchmail: SMTP< 250-STARTTLS
fetchmail: SMTP< 250-ENHANCEDSTATUSCODES
fetchmail: SMTP< 250-8BITMIME
fetchmail: SMTP< 250 DSN
fetchmail: forwarding to localhost
fetchmail: SMTP> MAIL FROM:<zz> BODY=8BITMIME SIZE=1046
fetchmail: SMTP< 250 2.1.0 Ok
fetchmail: SMTP> RCPT TO:<kbs@localhost>
fetchmail: SMTP< 250 2.1.5 Ok
fetchmail: SMTP> DATA
fetchmail: SMTP< 354 End data with <CR><LF>.<CR><LF>
fetchmail: IMAP< )
fetchmail: IMAP< A0007 OK FETCH completed.
fetchmail: IMAP> A0008 FETCH 1 BODY.PEEK[TEXT]
fetchmail: IMAP< * 1 FETCH (BODY[TEXT] NIL)
 (82 body octets)fetchmail: timeout after 60 seconds.
fetchmail: socket error while fetching from aa@bb@XX
fetchmail: 6.3.6 querying XX (protocol IMAP) at Wed 12 Sep 2007 02:41:38 PM PDT: poll completed
fetchmail: Query status=2 (SOCKET)
fetchmail: Deleting fetchids file.
fetchmail: sleeping at Wed 12 Sep 2007 02:41:38 PM PDT for 120 seconds

Finally, I took some tcpdumps (unfortunately, because the IMAP side is over SSL, it wasn't that useful.) However, fetchmail appears to write the FETCH message itself over to the SMTP socket, so
clearly something is going wrong here.

Here is the extracted tcpdump session on the connection that fetchmail makes with the local SMTP host.
220 zz ESMTP Postfix (Debian/GNU)
EHLO zz
250-zz
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
MAIL FROM:<nn@zz> BODY=8BITMIME SIZE=1046
250 2.1.0 Ok
RCPT TO:<kbs@localhost>
250 2.1.5 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
MIME-Version: 1.0
Received: from xx [xx.xx.xx.xx]
.by zz with IMAP (fetchmail-6.3.6)
.for <kbs@localhost> (single-drop); Wed, 12 Sep 2007 14:46:38 -0700 (PDT)
Received: from zz ([xx.xx.xx.xx]) by
 nn ([xx.xx.xx.xx]) with mapi; Wed, 12 Sep 2007
 16:30:59 +0000
From: zz
To: xx
Date: Wed, 12 Sep 2007 16:30:57 +0000
Subject: xx
Thread-Topic: xx
Thread-Index: Acf1WkwmkO1XV4CUR+qFqV1I8ZO9Gg==
Message-ID:
 <B77B101B97D2664095BB678BDDAF0DC0EDD34464@nn>
Accept-Language: en-US
Content-Language: en-US
X-MS-Exchange-Organization-AuthAs: Internal
X-MS-Exchange-Organization-AuthMechanism: 03
X-MS-Exchange-Organization-AuthSource: nn
X-MS-Has-Attach:
X-Auto-Response-Suppress: DR, OOF, AutoReply
X-MS-Exchange-Organization-SCL: -1
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

A0008 OK FETCH completed.


For detailed info, follow this link:
http://developer.berlios.de/bugs/?func=detailbug&bug_id=11980&group_id=1824

Re: [fetchmail-devel] STARTTLS for ODMR

[Matthias A. <mat...@gm...>]

Jakob Hirsch schrieb:

> In general, you are right, of course. But the code is merely 60 lines 
> and contains no heavy logic. The only SSL-specific stuff is a single 
> call to SSLOpen(), the rest is more or less the same, with only some 
> protocol specifics, esp. the rereading of the server's capabilities. I'm 
> not sure if creating a new function that has to fulfill all the needs is 
> that good, but I didn't look that close at the code. pop3.c contains 
> just the same code, btw.

Apparently my attempt to subliminally delegate this part of the cleanup
failed, but at least I tried. :-)

>> If it's still possible to add it manually, I don't mind, although I
>> wonder if that makes the patch suitable for inclusion into 6.3.
> 
> Sure, "fetchdomains" works as before. This is only the first hunk of the 
> patch, so it could be easily removed.

It's an incompatible change, so if we can find a configuration option to
suppress adding that, I'm more comfortable with that because it avoids
another "incompatible change" changelog entry :-)

>>> - What's with the formatting in odmr.c?? Mixed spaces and tabs, that's
>>> quite horrible... if you don't mind, I'll clean that up (to whatever the
>>> preferred format is).
>> Well, my canonical style (I'm using vim) is an indentation width of 4
>> columns per level and using a tabwidth of 8, I.e. it's zero or more tabs
> 
> Oh. Ok. Any reason for that? (Not that I want to start a discussion on 
> tabs vs. spaces.)
> And how do you tell vim (which I happen to use, too) to use this style?

Historical practice with unknown origin. I don't even know if
smartindent/cindent/indentexpr would be even easier to use (-8

I think the crucial vim magic is the first 'set' line below
(tabstop=8 is the default in my vim 7):

" --------------------------------------------------------
" ~/.vimrc excerpt

" crucial part:
set shiftwidth=4 softtabstop=4 tabstop=8

" And beyond that, I use - among others:
set nocompatible
set bs=2 fo=cqrt ls=2 shm=at tw=72 ww=<,>,h,l
set showmatch ruler
set grepprg=grep\ -nH\ $*
set hidden
set backup
set autoindent
set iskeyword+=:

filetype plugin on
filetype indent on

if has("syntax")
        so ${VIMRUNTIME}/syntax/syntax.vim
endif
" end of ~/.vimrc excerpt
" --------------------------------------------------------

Re: [fetchmail-devel] Downloading remote server certificate for a TLS session?

[Rob M. <rob...@gm...>]

On 8/23/07, Jakob Hirsch <jh...@pl...> wrote:
>
> The message "connect: Connection refused" you got with openssl sounds
> more like it couldn't even connect to the server. It may be worth
> tracing this problem with strace and tcpdump.
> Is your main server publically reachable?

It is, and it works fine (from the same host) with fetchmail or
telnet.  I'll have a closer look tomorrow night, but I'm pretty sure
it's not a network problem :)

> 0.9.8e is 6 months old, so I guess (or hope) it will be absorbed in the
> next version of $DISTRIBUTION.

But lots of people won't/don't upgrade working systems if they don't
have to.  Heck, I've got a Mandrake 9.0 box that's barely been touched
since it was installed.

> As Matthias pointed out, the fingerprint is suitable for that.
> Don't get me wrong, I'm not against a "dump the server's cert" feature
> in fetchmail, it could be handy. But I'm not sure that what you want to
> do with it is The Right Thing. But then again, I didn't follow the
> discussion which lead you to start this thread.

I'm beginning to lean towards the view that the fingerprint is fine.
I do think however that failed certificate verification should be
handled the same way a failed password verification is - emailed
notification upon the first occurrence.

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] Downloading remote server certificate for a TLS session?

[Jakob H. <jh...@pl...>]

Quoting Rob MacGregor:

>> I don't know anything about your server, but it works fine here:
>>
>> $ openssl s_client -connect koi:pop3 -starttls pop3
>> CONNECTED(00000003)
>> ...
>> +OK Dovecot ready. <...>
> Ok, I've just tried it on another host and it works there, just not on
> my main host - which works fine with fetchmail's TLS support.

The message "connect: Connection refused" you got with openssl sounds 
more like it couldn't even connect to the server. It may be worth 
tracing this problem with strace and tcpdump.
Is your main server publically reachable?

>> Right, but I don't hink that fetchmail is the right tool for that. On
>> the other hand, it's not that hard to print the certificate.
> From my results above it looks like the s_client functionality isn't
> 100%, and it's use relies on people having the very latest version for
> IMAP support (from your comment below).

0.9.8e is 6 months old, so I guess (or hope) it will be absorbed in the 
next version of $DISTRIBUTION.

> History on the -users mailing list suggests that we really need to
> make it easy for people to solve these problems.  If it's simple
> enough to provide all the required information from within fetchmail
> then that's got to make their lives (and by extension mine :>) easier.

As Matthias pointed out, the fingerprint is suitable for that.
Don't get me wrong, I'm not against a "dump the server's cert" feature 
in fetchmail, it could be handy. But I'm not sure that what you want to 
do with it is The Right Thing. But then again, I didn't follow the 
discussion which lead you to start this thread.

> It does however assume that everybody is using at least 0.9.8e :)  I
> suppose I'm mostly concerned about the poor souls who're using RPM
> based distributions.  It isn't always possible to upgrade one package

Sure, even Fedora 7 uses 0.9.8b. But the same applies to fetchmail. Most 
people use the distribution's package, I guess. But upgrading fetchmail 
is admittedly quite easier, because of much less dependencies (if any).

Re: [fetchmail-devel] STARTTLS for ODMR

[Jakob H. <jh...@pl...>]

Quoting Matthias Andree:

>> Most of the code is copied from imap.c. I inserted it into the generic
>> smtp code, so it could probably be used to deliver fetched mail via TLS
>> (if somebody ever wanted to do that).
> Without looking at the code, is it possible to factor out the common
> code into tls.c? Copy & paste is always prone to leaving a bug in one

In general, you are right, of course. But the code is merely 60 lines 
and contains no heavy logic. The only SSL-specific stuff is a single 
call to SSLOpen(), the rest is more or less the same, with only some 
protocol specifics, esp. the rereading of the server's capabilities. I'm 
not sure if creating a new function that has to fulfill all the needs is 
that good, but I didn't look that close at the code. pop3.c contains 
just the same code, btw.

>> You may notice that the patch removes adding our hostname to the
>> ctl->domainlist for ODMR. I think this was plain wrong and bothered me
>> everytime I set up fetchmail with odmr. ODMR's default is a simple
>> "ATRN" without any arguments, which means "fetch mail for all domains
>> associated with the account", so we should stick with that.
> If it's still possible to add it manually, I don't mind, although I
> wonder if that makes the patch suitable for inclusion into 6.3.

Sure, "fetchdomains" works as before. This is only the first hunk of the 
patch, so it could be easily removed.

>> Additional notes:
>> - The logging looks broken, with -v you get mixed ODMR and SMTP line
>> prefixes, but that's not related to my patch. Fixing this would could
>> probably easily be done by changing smtp_mode and using it as an index
>> to an array { "SMTP", "LMTP", "ODMR", ... }.
> Sunil knows that part of the code better than I do, but we can fix that
> if we catch all occurrences and switch() statements related to
> smtp_mode. (I wonder if that's a reasonable design anyways, but I'm
> loathe to change it in 6.3.)

AFAICS smtp_mode is used in logging, to get the char, or in comparisons 
with the macroes SMTP_MODE and LMTP_MODE. Should be not too hard to change.

>> - What's with the formatting in odmr.c?? Mixed spaces and tabs, that's
>> quite horrible... if you don't mind, I'll clean that up (to whatever the
>> preferred format is).
> Well, my canonical style (I'm using vim) is an indentation width of 4
> columns per level and using a tabwidth of 8, I.e. it's zero or more tabs

Oh. Ok. Any reason for that? (Not that I want to start a discussion on 
tabs vs. spaces.)
And how do you tell vim (which I happen to use, too) to use this style?

> Thanks for your efforts again.

Well, it's definitely not much compared to your efforts. So, many thanks 
back :)

Re: [fetchmail-devel] Downloading remote server certificate for a TLS session?

[Rob M. <rob...@gm...>]

On 8/23/07, Matthias Andree <mat...@gm...> wrote:
>
> Yes, older openssl s_client versions don't support as many protocols.

Even 0.9.8e, the most recent, appears to be less than 100% - it won't
talk TLS to my domain host's IMAP/POP servers (but fetchmail will).

<---SNIP--->
> Generally, there are two approaches of trusting the server's
> certificate:
>
> 1. The canonical and recommended one: verify the recognized
>    Certification Authority's signature on the server certificate (that
>    works for major CAs as their certificates are usually shipped with
>    the OS or available as add-on, for instance, in FreeBSD's ca-roots
>    port).  /etc/ssl/certs contains certificates of CAs we trust
>    (recognize) and is thus the configuration directory for the "trusted
>    CAs".
>
>    That is the usual way of doing things, and reasonable sites using
>    self-signed certificates provide their root CA certificates for
>    download with a web browser and usually offer a phone number you can
>    call to verify the fingerprint.
>    At least that's how my former and current universities and the DFN
>    (at a very coarse look, they provide the Internet backbone to German
>    Universities) do that.

Sadly, not all do.  I'll need to check, but I'm pretty sure one of my
current mail hosts uses a self signed certificate and no way of
downloading the CA certificate.

> 2. The less durable one: verify the server's certificate instead.
>
>    The recommendation of downloading the certificate and stuffing it
>    into /etc/ssl/certs however is just a very cumbersome alternative of
>    specifying the sslfingerprint which fetchmail already prints at -v
>    verbose level. Let's not tell users to use openssl s_client to
>    download certificates, but let's just point them to the
>    sslfingerprint option and tell them that they need a recent fetchmail.
>
>    The only technical difference is we're using a hash of the
>    certificate and are currently relying on MD5, but I'm not aware that
>    attacks are publicly known to generate a message with a specific hash
>    in a reasonable amount of time.

I'm not aware of any easy way of generating a matching MD5, with valid
content, easily.

I suppose this raises a feature request then - when a certificate
fails to verify fetchmail should email the user specified in the poll
command (or the postmaster if multiple users are listed) to tell them.

> The wording above is perhaps not as clear as it could be if I revised
> this text several times, but let's just see where it's unclear and
> revise the critical parts. (And if that's going to evolve into a "SSL
> certificate management for fetchmail" section, that's exactly what I'm
> aiming at :-))

I don't have any problems understanding it, but then I have a
reasonable understanding of how SSL works :)  I'll come back to it
later and have another read to see if I can spot anything the average
user of fetchmail may not understand.

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] STARTTLS for ODMR

[Matthias A. <mat...@gm...>]

Jakob Hirsch schrieb am 2007-08-23:

> here's a patch to add STARTTLS to the ODMR protocol. (I was a little
> surprised yesterday when I noticed that it isn't already there...)

Thanks.

> Most of the code is copied from imap.c. I inserted it into the generic
> smtp code, so it could probably be used to deliver fetched mail via TLS
> (if somebody ever wanted to do that).

Without looking at the code, is it possible to factor out the common
code into tls.c? Copy & paste is always prone to leaving a bug in one
place while the other has been fixed.  I should like the crucial code to
be in ONE place when reasonably possible even if that requires using
callback hooks or function address arguments, so that we don't have to
patch two or three places when a bug is found in once, and "copying"
breaks that, and SSL/TLS code makes delicate decisions -- let's avoid
any avoidable embarrassment in that area.

> You may notice that the patch removes adding our hostname to the
> ctl->domainlist for ODMR. I think this was plain wrong and bothered me
> everytime I set up fetchmail with odmr. ODMR's default is a simple
> "ATRN" without any arguments, which means "fetch mail for all domains
> associated with the account", so we should stick with that.

If it's still possible to add it manually, I don't mind, although I
wonder if that makes the patch suitable for inclusion into 6.3.

> Additional notes:
> - The logging looks broken, with -v you get mixed ODMR and SMTP line
> prefixes, but that's not related to my patch. Fixing this would could
> probably easily be done by changing smtp_mode and using it as an index
> to an array { "SMTP", "LMTP", "ODMR", ... }.

Sunil knows that part of the code better than I do, but we can fix that
if we catch all occurrences and switch() statements related to
smtp_mode. (I wonder if that's a reasonable design anyways, but I'm
loathe to change it in 6.3.)

> - What's with the formatting in odmr.c?? Mixed spaces and tabs, that's
> quite horrible... if you don't mind, I'll clean that up (to whatever the
> preferred format is).

Well, my canonical style (I'm using vim) is an indentation width of 4
columns per level and using a tabwidth of 8, I.e. it's zero or more tabs
followed by zero or four spaces. Any reasonable editor should be capable
of writing that, and if not, use "all spaces" and run the code through
unexpand --first-only before diffing. Thanks.

I hope to get around to do some bugfixing later tonight (European time)
so that I can release 6.3.9 soon and then move on to 6.4.

Thanks for your efforts again.

-- 
Matthias Andree

Re: [fetchmail-devel] FAQ re-write

[Rob M. <rob...@gm...>]

On 8/23/07, Matthias Andree <mat...@gm...> wrote:
> On Wed, 22 Aug 2007, Rob MacGregor wrote:
<---SNIP--->
> Well, the MoinMoin wiki engine has a command-line tool to export static
> HTML pages, Timo Sirainen is using that for snapshotting the Dovecot
> Wiki contents. (Dovecot is a fast combined IMAP/POP3+SSL server
> supporting mbox and Maildir, among other useful features.)

That would certainly make life easier.

> If the Wiki can't export to HTML, perhaps we can stick to using a set
> of HTML pages - it's easier to control quality that way, too: we don't
> need to eyeball it constantly to avoid other people posting stupid
> things (such as enclosing mda placeholders in single quotes, injecting
> into sendmail with variable destination address or something).
> I'm not at all against the concept of a Wiki, but I think some kind of
> "peer review" to avoid stupidities should be inherent once we deal with
> security sensitive stuff.

If we go the wiki route then it would have to support (at least) 2
levels of users - admins (those you trust not to stuff up the
documentation) and everybody else (who have read only access).

Looking at the MoinMoin wiki documentation it does support that, plus
email notifications about page edits.

As for peer review, presumably such discussions could take place via
email, or even via Talk pages in the wiki?

> The HTML document is in SVN, and I think we (that means my asking Graham
> Wilson) can arrange commit access for you, or we can just revive the
> BerliOS SVN stuff to host the web site.

Well, getting up and running with SVN wouldn't be terribly difficult,
so that's always an option.

I'll admit that I'm looking for a solution that involves the minimal
amount of effort post-edit - I'm lazy but at least I know it ;)

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] Downloading remote server certificate for a TLS session?

[Matthias A. <mat...@gm...>]

On Thu, 23 Aug 2007, Rob MacGregor wrote:

> From my results above it looks like the s_client functionality isn't
> 100%, and it's use relies on people having the very latest version for
> IMAP support (from your comment below).

Yes, older openssl s_client versions don't support as many protocols.

> History on the -users mailing list suggests that we really need to
> make it easy for people to solve these problems.  If it's simple
> enough to provide all the required information from within fetchmail
> then that's got to make their lives (and by extension mine :>) easier.

I know at least two very annoying bugs in older versions of fetchmail (I
don't recall the version numbers offhand, but it's in the NEWS file):

a. fetchmail not initializing the default certificate path in OpenSSL.

   Consequence: OpenSSL doesn't look at /etc/ssl/certs (or whatever the
   default path is).

   Workaround: use --sslcertpath /etc/ssl/certs

b. fetchmail complaining if --sslcertck is NOT given, but a matching
   --sslfingerprint is.

   AFAIK no workaround except short of filtering them out from the log.

Generally, there are two approaches of trusting the server's
certificate:

1. The canonical and recommended one: verify the recognized
   Certification Authority's signature on the server certificate (that
   works for major CAs as their certificates are usually shipped with
   the OS or available as add-on, for instance, in FreeBSD's ca-roots
   port).  /etc/ssl/certs contains certificates of CAs we trust
   (recognize) and is thus the configuration directory for the "trusted
   CAs".

   That is the usual way of doing things, and reasonable sites using
   self-signed certificates provide their root CA certificates for
   download with a web browser and usually offer a phone number you can
   call to verify the fingerprint.
   At least that's how my former and current universities and the DFN
   (at a very coarse look, they provide the Internet backbone to German
   Universities) do that. As to the major freemailers I have acocunts
   with, they have their server certificates signed by Thawte, Verisign,
   Equifax or whoever else that up-to-date distributions usually
   recognize as trusted.

   I'm not sure if it's always possible to download the CA's certificate
   with openssl s_client.

2. The less durable one: verify the server's certificate instead.

   The recommendation of downloading the certificate and stuffing it
   into /etc/ssl/certs however is just a very cumbersome alternative of
   specifying the sslfingerprint which fetchmail already prints at -v
   verbose level. Let's not tell users to use openssl s_client to
   download certificates, but let's just point them to the
   sslfingerprint option and tell them that they need a recent fetchmail.

   The only technical difference is we're using a hash of the
   certificate and are currently relying on MD5, but I'm not aware that
   attacks are publicly known to generate a message with a specific hash
   in a reasonable amount of time.

Reason why I call 2 less durable is that CA certificates are usually
valid for five years or longer and to a certain extent managed by the
distributor, whereas server certificates often expire sooner, causing
issues once a year or so.

The wording above is perhaps not as clear as it could be if I revised
this text several times, but let's just see where it's unclear and
revise the critical parts. (And if that's going to evolve into a "SSL
certificate management for fetchmail" section, that's exactly what I'm
aiming at :-))

Best regards,

-- 
Matthias Andree

Re: [fetchmail-devel] FAQ re-write

[Matthias A. <mat...@gm...>]

On Wed, 22 Aug 2007, Rob MacGregor wrote:

> As for turning Wiki pages into HTML and/or PDF - neither look like
> standard features of either the Berlios of PB Wikis.  Both conversions
> however should be easy enough to automate/script (I've already written
> some scripts to turn XML/HTML into PDF, based on HTMLDoc ISTR), though
> I'm willing to be proven wrong :)

The (X)HTML -> PDF part is the easy one, there are several automated
ways to achieve that, HTMLDOC being one of the easiest and with very
reasonable result.

Well, the MoinMoin wiki engine has a command-line tool to export static
HTML pages, Timo Sirainen is using that for snapshotting the Dovecot
Wiki contents. (Dovecot is a fast combined IMAP/POP3+SSL server
supporting mbox and Maildir, among other useful features.)

If the Wiki can't export to HTML, perhaps we can stick to using a set
of HTML pages - it's easier to control quality that way, too: we don't
need to eyeball it constantly to avoid other people posting stupid
things (such as enclosing mda placeholders in single quotes, injecting
into sendmail with variable destination address or something).
I'm not at all against the concept of a Wiki, but I think some kind of
"peer review" to avoid stupidities should be inherent once we deal with
security sensitive stuff.

The HTML document is in SVN, and I think we (that means my asking Graham
Wilson) can arrange commit access for you, or we can just revive the
BerliOS SVN stuff to host the web site.

Best

-- 
Matthias Andree

Re: [fetchmail-devel] Downloading remote server certificate for a TLS session?

[Rob M. <rob...@gm...>]

On 8/22/07, Jakob Hirsch <jh...@pl...> wrote:
> Rob MacGregor wrote:
>
> I don't know anything about your server, but it works fine here:
>
> $ openssl s_client -connect koi:pop3 -starttls pop3
> CONNECTED(00000003)
> ...
> +OK Dovecot ready. <...>

Ok, I've just tried it on another host and it works there, just not on
my main host - which works fine with fetchmail's TLS support.

> Right, but I don't hink that fetchmail is the right tool for that. On
> the other hand, it's not that hard to print the certificate.

From my results above it looks like the s_client functionality isn't
100%, and it's use relies on people having the very latest version for
IMAP support (from your comment below).

History on the -users mailing list suggests that we really need to
make it easy for people to solve these problems.  If it's simple
enough to provide all the required information from within fetchmail
then that's got to make their lives (and by extension mine :>) easier.

If openssl's client support was better established then I'd probably
agree with you though.

> But it looks like the openssl people are working on it: 0.9.8e also
> contains STARTTLS support for imap (and even handles smtp properly).

Just tried it with one provider and it works, useful.

It does however assume that everybody is using at least 0.9.8e :)  I
suppose I'm mostly concerned about the poor souls who're using RPM
based distributions.  It isn't always possible to upgrade one package
without upgrading dozens of others - something people who're just
trying to get one package working may not want to/be able to do.

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] Downloading remote server certificate for a TLS session?

[Jakob H. <jh...@pl...>]

Rob MacGregor wrote:

>> You are probably looking for
>> openssl s_client -connect your.server:pop3s
>> (or ...:pop3 -starttls pop3)
> I just tried "openssl s_client -connect my.mail.server:pop3 -starttls
> pop3" and all I get is:
> 
> connect: Connection refused
> connect:errno=61

I don't know anything about your server, but it works fine here:

$ openssl s_client -connect koi:pop3 -starttls pop3
CONNECTED(00000003)
...
+OK Dovecot ready. <...>

>> starttls works only with pop3 and smtp, though (and the smtp support is
>> lousy...).
> The recent thread was about an IMAP server that supported TLS, but
> there was no IMAPS support, and my own test suggests that the s_client
> protocol support is, well, poor.  If we're going to encourage the use

Right, but I don't hink that fetchmail is the right tool for that. On
the other hand, it's not that hard to print the certificate.

But it looks like the openssl people are working on it: 0.9.8e also
contains STARTTLS support for imap (and even handles smtp properly).

[fetchmail-devel] STARTTLS for ODMR

[Jakob H. <jh...@pl...>]

Hi,

here's a patch to add STARTTLS to the ODMR protocol. (I was a little
surprised yesterday when I noticed that it isn't already there...)

Most of the code is copied from imap.c. I inserted it into the generic
smtp code, so it could probably be used to deliver fetched mail via TLS
(if somebody ever wanted to do that).

I did run some tests, but not really thorough. Comments welcome.

You may notice that the patch removes adding our hostname to the
ctl->domainlist for ODMR. I think this was plain wrong and bothered me
everytime I set up fetchmail with odmr. ODMR's default is a simple
"ATRN" without any arguments, which means "fetch mail for all domains
associated with the account", so we should stick with that.

Additional notes:
- The logging looks broken, with -v you get mixed ODMR and SMTP line
prefixes, but that's not related to my patch. Fixing this would could
probably easily be done by changing smtp_mode and using it as an index
to an array { "SMTP", "LMTP", "ODMR", ... }.

- What's with the formatting in odmr.c?? Mixed spaces and tabs, that's
quite horrible... if you don't mind, I'll clean that up (to whatever the
preferred format is).


Regards,
Jakob

Re: [fetchmail-devel] Downloading remote server certificate for a TLS session?

[Rob M. <rob...@gm...>]

On 8/22/07, Jakob Hirsch <jh...@pl...> wrote:
>
> You are probably looking for
> openssl s_client -connect your.server:pop3s
> (or ...:pop3 -starttls pop3)

I just tried "openssl s_client -connect my.mail.server:pop3 -starttls
pop3" and all I get is:

connect: Connection refused
connect:errno=61

But it's a TLS capable server.  Running the relevant command against
the pop3s service works fine.

> starttls works only with pop3 and smtp, though (and the smtp support is
> lousy...).

The recent thread was about an IMAP server that supported TLS, but
there was no IMAPS support, and my own test suggests that the s_client
protocol support is, well, poor.  If we're going to encourage the use
of SSL/TLS (which we do by the fact that fetchmail will use TLS if
it's available) then we need to make it easy for people to import
certificates.

-- 
                 Please keep list traffic on the list.

Rob MacGregor
      Whoever fights monsters should see to it that in the process he
        doesn't become a monster.                  Friedrich Nietzsche

Re: [fetchmail-devel] Downloading remote server certificate for a TLS session?

[Jakob H. <jh...@pl...>]

Rob MacGregor wrote:

> I was thinking about the recent thread that Anne Wilson started about
> TLS certificate problems.  While there's a fairly easy way of getting
> SSL certificates, there doesn't seem to be any way to use OpenSSL to
> get a certificate from a TLS session.

You are probably looking for
openssl s_client -connect your.server:pop3s
(or ...:pop3 -starttls pop3)

starttls works only with pop3 and smtp, though (and the smtp support is
lousy...).