Re: [Fetchmail-users] ATP dynamic delivery

[<sup...@gm...>]

It is very poorly documented - it also seems a daft amount of work to make something that should be hidden from the end user visible. I cannot see any tangible benefit other than,  "useless gimic marketing bollocks".

I will try the --keep option and observe what I get - it occurs to me that one option work simply be to "keep" for a limited time period and delete older messages.  Based on observations of the propagation delay, keeping for 1 hour would probably save 99% of attachments. However, I understand that some part of the triage of these attachments can involve a human in the loop for a small number of attachments so maybe simply keeping for 24 hours would restore full functionality without requiring maintenance.

I think that handling the dummy email (and possible duplicates) is easily done downstream (eg I can filter the dummy email out so I never see it).

Changing the server delivery policy to "replace" instead of "dynamic" appears to be the ideal solution but someone somewhere has gotten very excited by this so I'm not holding my breath.





On Saturday, 29 July 2023 07:21:11 BST Matthias Andree wrote:
> Am 29.07.23 um 00:31 schrieb sup...@gm...:
> > ATP dynamic delivery is a MS exchange concept whereby emails are delivered without attachments but preview links instead.
> >
> > The attachments are scanned off line and then the original email is sucked back and replaced by a new version of the email with the attachment - this can take quite a while.
> >
> > If fetchmail gets the email before this process occurs the suck back and replacement with the attachment never happens, the "preview" link goes dead and the attachment never arrives.
> >
> > It is definitely a server side issue but I can't change the server side behaviour (if you can change the server side settings then simple fix is to change the "dynamic" delivery policy with "replace" delivery policy which behaves like any normal system where the email is delivered after it is scanned).
> >
> > This is not the only problem with ATP but it seems to be the biggest source of loss of attachments.
> >
> > A possible tactic would be to exclude the fetching of emails containing these preview links and defer their download until the suck back and replacement process has happened.
> 
> I have found very little documentation on this, only that people
> complain that they need to close and re-open their email client window,
> and that apparently the messages are being replaced. I have not seen any
> information how I could identify "attachments being scanned" and "full
> e-mail now available" through the IMAP or POP3 interface?
> 
> The sound way to do that would be for the server to *DELETE* the preview
> message and *ADD* the full message at the end of the mailbox as though
> it were new.
> 
> If instead they were changing an e-mail we have already downloaded, in
> place, without changing the message's UID, we will never know.
> 
> So, first try you could do is running fetchmail with --keep (as in do
> not delete), unless you are already doing that, and see if you get such
> messages twice, once as preview, once as full message. If that works,
> fine, you know the workaround and we may be able to file down some rough
> edges and possibly add a generic-enough header matcher that skips a
> message when that header is present.
> 
> Note this would be prone to abuse if they don't strip that header from
> incoming messages with all the attachments in place (after scan) because
> I could then just put that header and prevent your message download...
> 
> If that does not help and they are replacing messages in place, I need
> to know the specification - would they put up a special header to mark
> that the message is only a preview that is reliable enough to "skip" a
> message? Is there reliable technical documentation? Note: I am not going
> to research this beyond my search engine probes this morning, someone
> will have to hand me the silver plate with all information. And after
> looking through a spec, I may conclude it's too unreliable to be worth
> investing more time.
> 
> 
> 
> 
> _______________________________________________
> Fetchmail-users mailing list
> Fet...@li...
> https://lists.sourceforge.net/lists/listinfo/fetchmail-users
>