Menu
▾
▴
fetchmail-devel
|
From: Joe M <joe...@gm...> - 2013-11-12 06:49:08
|
I am attaching a patch to add a --startnum option to fetchmail. This enables fetchmail to start downloading from the specified message number instead of from the first message. This helps with flaky SMTP connections where the connection is closed after a certain number of messages are downloaded. This is my first patch to fetchmail. Please let me know if there is a better way of going about this. Below is further explanation of my issue. ----- Forwarded message from Joe M <joe...@gm...> ----- Date: Mon, 11 Nov 2013 15:23:40 -0600 To: fet...@li... Subject: fetchmail downloading with message number as a parameter Hello, I am using fetchmail to download from my hotmail account. It downloads a hundred or so emails and the connection dies. It does this every time. I have the "keep" option set so it keeps downloading the first emails again and again. I am trying to figure out if I can tell fetchmail to download a certain message number or range of message numbers. Any thoughts or other alternative ideas that I can try, please? Thanks Joe ----- End forwarded message ----- |
|
From: Joe M <joe...@gm...> - 2013-11-12 07:00:22
|
Joe M wrote: > I am attaching a patch to add a --startnum option to fetchmail. Attached is an updated patch which adds this option to fetchmail.man Any comments, please? Thanks Joe |
|
From: Joe M <joe...@gm...> - 2013-11-12 07:41:03
|
> Joe M wrote: > > I am attaching a patch to add a --startnum option to fetchmail. > > Attached is an updated patch which adds this option to fetchmail.man Attached is the updated patch fixing a compilation issue and it seems to be working when I test it. Below is output from my test run: FETCHMAILHOME=/home/j/etc/mail/username/ fetchmail --fetchmailrc ~/etc/mail/username/fetchmailrc --verbose -B 30 --startnum 302 fetchmail: 6.3.26 querying pop3.live.com (protocol POP3) at Tue 12 Nov 2013 12:27:44 AM CST: poll started Trying to connect to 65.54.51.39/995...connected. fetchmail: Server certificate: fetchmail: Issuer Organization: GlobalSign nv-sa fetchmail: Issuer CommonName: GlobalSign Organization Validation CA - G2 fetchmail: Subject CommonName: *.hotmail.com fetchmail: Subject Alternative Name: *.hotmail.com fetchmail: Subject Alternative Name: *.live.com fetchmail: Subject Alternative Name: *.outlook.com fetchmail: Subject Alternative Name: hotmail.com fetchmail: pop3.live.com key fingerprint: 86:60:F6:38:1C:84:A6:AC:94:92:51:2F:67:9A:7D:76 fetchmail: POP3< +OK snt0-pop60 POP3 server ready fetchmail: POP3> CAPA fetchmail: POP3< -ERR unrecognized command fetchmail: unrecognized command fetchmail: Repoll immediately on use...@ho...@pop3.glbdns2.microsoft.com Trying to connect to 65.54.51.39/995...connected. fetchmail: Server certificate: fetchmail: Issuer Organization: GlobalSign nv-sa fetchmail: Issuer CommonName: GlobalSign Organization Validation CA - G2 fetchmail: Subject CommonName: *.hotmail.com fetchmail: Subject Alternative Name: *.hotmail.com fetchmail: Subject Alternative Name: *.live.com fetchmail: Subject Alternative Name: *.outlook.com fetchmail: Subject Alternative Name: hotmail.com fetchmail: pop3.live.com key fingerprint: 86:60:F6:38:1C:84:A6:AC:94:92:51:2F:67:9A:7D:76 fetchmail: POP3< +OK snt0-pop125 POP3 server ready fetchmail: POP3> USER use...@ho... fetchmail: POP3< +OK password required fetchmail: POP3> PASS * fetchmail: POP3< +OK mailbox has 2521 messages fetchmail: POP3> STAT fetchmail: POP3< +OK 2521 503162348 2521 messages for use...@ho... at pop3.live.com (503162348 octets). fetchmail: POP3> LIST 302 fetchmail: POP3< +OK 302 10944 fetchmail: POP3> RETR 302 fetchmail: POP3< +OK reading message use...@ho...@pop3.glbdns2.microsoft.com:302 of 2521 (10944 octets) #*****************************************.******************************************* ***.**********************************.************************************.*************************.***************************.***************************.*********************** *.**************************.************* not flushed fetchmail: POP3> LIST 303 fetchmail: POP3< +OK 303 5596 fetchmail: POP3> RETR 303 fetchmail: POP3< +OK reading message use...@ho...@pop3.glbdns2.microsoft.com:303 of 2521 (5596 octets) #************************************.*****************************************.******* **********************************.********************************.****************** Thanks Joe |
|
From: Matthias A. <mat...@gm...> - 2013-11-12 10:20:03
|
Am 12.11.2013 06:51, schrieb Joe M: > Joe M wrote: >> I am attaching a patch to add a --startnum option to fetchmail. > > Attached is an updated patch which adds this option to fetchmail.man > > Any comments, please? > > Thanks > Joe Joe, thanks a bunch for your contribution. I am only wondering what limitations of either hotmail or your SMTP sink this is trying to work around that are not already solved -- and if it will remain useful at all for 7.x when make UIDL support mandatory. The code is there, on the "master" branch in Git. The URL is https://gitorious.org/fetchmail/fetchmail/ Would you be able to check if the code is fit for fetchmail's master branch? Thank you. Best regards Matthias |
|
From: Matthias A. <mat...@gm...> - 2013-11-12 10:28:30
|
Am 12.11.2013 10:20, schrieb Matthias Andree: > Am 12.11.2013 06:51, schrieb Joe M: >> Joe M wrote: >>> I am attaching a patch to add a --startnum option to fetchmail. >> >> Attached is an updated patch which adds this option to fetchmail.man >> >> Any comments, please? >> >> Thanks >> Joe > > Joe, > > thanks a bunch for your contribution. > > I am only wondering what limitations of either hotmail or your SMTP sink > this is trying to work around that are not already solved -- and if it > will remain useful at all for 7.x when make UIDL support mandatory. The > code is there, on the "master" branch in Git. > > The URL is https://gitorious.org/fetchmail/fetchmail/ > > Would you be able to check if the code is fit for fetchmail's master branch? > > Thank you. > > Best regards > Matthias Joe, As an update, I just polled my pop3.live.com accounts and figured that --uidl works for me on that account. Note that using --keep without --uidl is a dangerous endeavor, meaning that if you use LAST or a locally recorded message number, this setup is prone to mail loss or skipping, should messages before the designated start number disappear or should a downloaded message fail to be forwarded. That comment above does not rule out my taking your patch, it may be useful to re-setup things if the UID database should ever get damaged (although I have not had reports of corruption of the .fetchids file in a long time), we may however need to extend your patch so that if you say "--startnum 1001", fetchmail could optionally record the first 1000 POP3 UIDs as seen. Hope that passes for a comment for the nonce :-) Thanks again! Best regards Matthias |
|
From: Joe M <joe...@gm...> - 2013-11-15 15:59:38
|
I inadvertently forwarded the below email to Mr. Matthias instead of the list. Hence, sending to the list now. ---------- Forwarded message ---------- From: Joe M <joe...@gm...> Date: Thu, Nov 14, 2013 at 8:36 PM Subject: Re: [fetchmail-devel] Fwd: fetchmail downloading with message number as a parameter To: Matthias Andree <mat...@gm...> Hello Matthias, > As an update, I just polled my pop3.live.com accounts and figured that > --uidl works for me on that account. Note that using --keep without > --uidl is a dangerous endeavor, meaning that if you use LAST or a > locally recorded message number, this setup is prone to mail loss or > skipping, should messages before the designated start number disappear > or should a downloaded message fail to be forwarded. This seems to have happened and I did not get your email. > That comment above does not rule out my taking your patch, it may be > useful to re-setup things if the UID database should ever get damaged > (although I have not had reports of corruption of the .fetchids file in > a long time), we may however need to extend your patch so that if you > say "--startnum 1001", fetchmail could optionally record the first 1000 > POP3 UIDs as seen. > > > Hope that passes for a comment for the nonce :-) Thanks for responding. It makes sense to use --uidl. I was downloading from multiple email accounts with different fetchmailrc (but, did not change the FETCHMAILHOME) and I guess the .fetchids was getting overwritten or something like that. Hence, I thought that there was no such functionality in fetchmail. > Thanks again! Thank you and Sorry for the delay. Thanks again, Joe |
|
From: Joe M <joe...@gm...> - 2013-11-15 16:02:23
|
Hello Matthias, I am missing emails. Is there anyway to correlate the id's from .fetchids with the email in the gmail inbox? I tried both pop and imap and in both instances, a few emails were not downloaded. I tried with --uidl and no --startnum. >> As an update, I just polled my pop3.live.com accounts and figured that >> --uidl works for me on that account. Note that using --keep without >> --uidl is a dangerous endeavor, meaning that if you use LAST or a >> locally recorded message number, this setup is prone to mail loss or >> skipping, should messages before the designated start number disappear >> or should a downloaded message fail to be forwarded. > > This seems to have happened and I did not get your email. Any thoughts, please? Thanks Joe |
|
From: Matthias A. <mat...@gm...> - 2013-12-11 00:43:19
|
I am sending a copy to fetchmail-users because this is of general interest, and also a recurring issue with Google Mail. Am 15.11.2013 16:02, schrieb Joe M: > Hello Matthias, > > I am missing emails. Is there anyway to correlate the id's from > .fetchids with the email in the gmail inbox? > > I tried both pop and imap and in both instances, a few emails were not > downloaded. > > I tried with --uidl and no --startnum. > >>> As an update, I just polled my pop3.live.com accounts and figured that >>> --uidl works for me on that account. Note that using --keep without >>> --uidl is a dangerous endeavor, meaning that if you use LAST or a >>> locally recorded message number, this setup is prone to mail loss or >>> skipping, should messages before the designated start number disappear >>> or should a downloaded message fail to be forwarded. >> >> This seems to have happened and I did not get your email. > > Any thoughts, please? Sorry for the late response, somehow missed your followup. Up front: gmail is quirky by design. Google did not model traditional mailboxes for POP3 or IMAP, but invented their own stuff around archiving and folders, and to add insult to injury, some of their help pages defame POP3 as unreliable when the fault lies with implementations, not the protocol. POP3 + UIDL can be made reliable - if you don't abuse it, that is. Still, I link to Google documentation (apparently the first two links now redirect to the same contents): <http://www.fetchmail.info/fetchmail-FAQ.html#I9> has a few links to Google documentation - check the "recent:" trick described there and see if that helps. <https://support.google.com/mail/answer/47948?hl=en> in particular explains that. The correlation between UIDs and messages is technically possible but yields only the POP3 message numbers for the .fetchids stored - and message numbers can change with every login (especially as messages are removed, or sometimes added, example below), and are therefore not really useful in your situation. If you still want to see them, all it takes is running fetchmail with -vv added on the command line. This logs the transcript of the POP3 exchange, which might look like this (this is from a Dovecot POP3 server, not from Gmail's POP3 interface): > fetchmail: POP3> USER joe > fetchmail: POP3< +OK > fetchmail: POP3> PASS * > fetchmail: POP3< +OK Logged in. > fetchmail: POP3> STAT > fetchmail: POP3< +OK 1314 77621905 > fetchmail: POP3> UIDL > fetchmail: POP3< +OK > fetchmail: POP3< 1 000000014eb1c89a > fetchmail: POP3< 2 000000034eb1c89a > fetchmail: POP3< 3 000000054eb1c89a > fetchmail: POP3< 4 000000064eb1c89a > fetchmail: POP3< 5 000000074eb1c89a ... That's all there is. You get a temporary message number (which is a count that may occasionally skip ahead over deleted messages) and the UID that you would also find in .fetchids after successful fetch + shipping. Deleting message 2 would then remove the 000000034eb1c89a, and on the next login you'd see the output below, so for 000000054eb1c89a the message number changed from 3 to 2. > fetchmail: POP3< 1 000000014eb1c89a > fetchmail: POP3< 2 000000054eb1c89a > fetchmail: POP3< 3 000000064eb1c89a > fetchmail: POP3< 4 000000074eb1c89a As said above, this is not very useful to hunt down missing messages. The best bet, given Google's documentation, would probably be to try reconfiguring your user name for Google's recent: mode, and run fetchmail with a --fetchall --keep --uidl once. That may cause duplicate message retrieval due to --fetchall, but better duplicate than none. |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X