Menu
▾
▴
fetchmail-devel
|
From: mojmir s. <moj...@2k...> - 2009-08-27 15:11:02
|
hello again,
i have few things on my mind:
1. portability of fetchmail
- what all platforms do you support/want to support?
(i've seen some linuxes, bsds, are there more?)
- same about compilers?
(gcc.. what about the bunch of others?)
- are there any restrictions about C language used in fetchmail?
(variadic macros for example etc)
- any crucial advice concerning fetchmail development?
2. mapi
i have found the problem with the overwrite of address, but
have no fix yet. the problem occurs when value of tag
PR_SENT_REPRESENTING_NAME does not contain FQDN (fully qualified
domain name... is that the name?), because it gets scrambled
by the reply_hack function from transact.c
this can be fixed by using PR_SENDER_EMAIL_ADDRESS, but it's
not perfect, since it contains weird stuff like
From: /O=TAKE-TWOINTERACTIVE/OU=2KGB/CN=RECIPIENTS/CN=AA...@2k...rp
so ideally the solution would be like:
if (PR_SENT_REPRESENTING_NAME is in form foo@FQDN)
use PR_SENT_REPRESENTING_NAME
else
use PR_SENDER_EMAIL_ADDRESS;
perhaps there's some flag in the mapi protocol but i can only
guess... so:
3. documentation for mapi...
...would be great. do you have any, please?
4. mapi todo
- the problem above
- AND the same problem applying to To: and Cc: fields
- threading of messages does not work
- valgrind reporting some memory stuff (reads beyond etc)
- fetching messages is quite slow
- when > 1000 messages, fetchmail says it cannot open MDA
with some IO error. it fails in sink.c:open_mda_sink after
the popen.
report(stderr, GT_("MDA open failed\n"));
perhaps some descriptor leak? do you have any ideas, please?
- flushing messages is also quite slow
- messages like:
- xxx:54 was not the expected length (1032 actual != 4863 expected)
kind regards,
mojmir
|
|
From: Matthias A. <mat...@gm...> - 2009-08-27 19:17:52
|
Hi Mojmir,
and thanks for your continued care of the MAPI function.
> hello again,
> i have few things on my mind:
>
> 1. portability of fetchmail
> - what all platforms do you support/want to support?
> (i've seen some linuxes, bsds, are there more?)
I usually care about Linux (Debian, Fedora, openSUSE), FreeBSD and
Solaris. Not breaking Cygwin and the other mainstream BSDs would be nice.
I don't care if you break BeOS or EMX or if Red Hat Linux 5.2 won't work.
I think current versions of Linux, BSD, Solaris/openSolaris (as
representative for the SysV systems), Cygwin. That should cover the
important ones. If obscure or obsolete OSes break with MAPI support, never
mind.
> - same about compilers?
> (gcc.. what about the bunch of others?)
GCC 3.4 and 4.3 are a must, 3.3 - 4.4 would be nice. In doubt, try
"-pedantic" :)
> - are there any restrictions about C language used in fetchmail?
> (variadic macros for example etc)
I'm undecided, I don't want to break older systems deliberately, so avoid
if you can for now.
If it's used only for MAPI, use whatever suits your purpose. Note that
macros defeat type checking, so use functions if you can.
> - any crucial advice concerning fetchmail development?
* Don't introduce security holes, avoid static buffers, and avoid
unbounded access (array subscripts/indexes, non-length-limiting string
functions) at all cost, particularly for statically sized buffers these
are a must.
* Where it makes sense to translate strings, please mark them
translatable, and - I probably don't need to tell you - if there are
various plural forms, please use the NGT_() macro, not just GT_(). Please
don't assemble translated strings with fragments of English phrases.
> 2. mapi
> i have found the problem with the overwrite of address, but
> have no fix yet. the problem occurs when value of tag
> PR_SENT_REPRESENTING_NAME does not contain FQDN (fully qualified
> domain name... is that the name?), because it gets scrambled
> by the reply_hack function from transact.c
>
> this can be fixed by using PR_SENDER_EMAIL_ADDRESS, but it's
> not perfect, since it contains weird stuff like
>
> From:
> /O=TAKE-TWOINTERACTIVE/OU=2KGB/CN=RECIPIENTS/CN=AA...@2k...rp
Looks like ITU-T X.500 series stuff, which isn't overly weird, it's just
that you're usually interested in the CN field (common name). Depending on
how openMAPI gets this information to you, take care of escaping and
continuation lines. LDIF for instance has rather unusual (in my personal
perspetion) line continuation format... just chop, insert LF and blank...
> 3. documentation for mapi...
> ...would be great. do you have any, please?
I don't - OpenChange perhaps? Otherwise ask Jelmer Vernooi if he wants to
help (he mentored Yang-Yan Li last year) or the openchange lists.
> 4. mapi todo
> - the problem above
> - AND the same problem applying to To: and Cc: fields
> - threading of messages does not work
Is the cause known yet? Incomplete headers, such as In-Reply-To: or
References:?
> - valgrind reporting some memory stuff (reads beyond etc)
If they're in OpenSSL, disregard. If they're in OpenChange, they'll
probably want to know...
> - fetching messages is quite slow
Too bad. I'd say postpone this. If it works slowly, than that's better
than "not at all" :)
> - when > 1000 messages, fetchmail says it cannot open MDA
> with some IO error. it fails in sink.c:open_mda_sink after
> the popen.
> report(stderr, GT_("MDA open failed\n"));
> perhaps some descriptor leak? do you have any ideas, please?
Try running with -vvv and see if you still get the "about to deliver
with:" message, and please show it to me.
Can I also see your mda option's argument? In doubt, report the usual
stuff I ask users to report,
http://www.fetchmail.info/fetchmail-FAQ.html#G3
WRT descriptor leaks: Try "lsof -c fetchmail" in a separate console while
fetchmail is fetching and short before it fails, i. e. after say 500 or
700 messages. If it's a descriptor leak, you'll get to know. Otherwise,
strace or truss could help. You could also grep "ps ax" or "ps -ef" output
for hints of process table exhaustion (grep for your MDA software).
> - flushing messages is also quite slow
> - messages like:
> - xxx:54 was not the expected length (1032 actual != 4863 expected)
Probably similar artifacts to
<http://www.fetchmail.info/fetchmail-FAQ.html#S2> - does it help to fiddle
with the Exchange registry?
HTH
Best regards
Matthias
--
Matthias Andree
|
|
From: mojmir s. <moj...@2k...> - 2009-09-02 10:22:53
|
good morning Matthias,
* mat...@gm... <mat...@gm...> [2009-08-28 06:15:43 +0000]:
> * Don't introduce security holes, avoid static buffers
oh i must read some secure programming stuff, i was never involved :)
by avoiding static buffers (local? global ones?) you mean allocate
every buffer on heap?
> > 2. mapi
> > /O=TAKE-TWOINTERACTIVE/OU=2KGB/CN=RECIPIENTS/CN?A....@2k...rp
>
> Looks like ITU-T X.500 series stuff, which isn't overly weird, it's just
> that you're usually interested in the CN field (common name). Depending on
> how openMAPI gets this information to you, take care of escaping and
> continuation lines. LDIF for instance has rather unusual (in my personal
> perspetion) line continuation format... just chop, insert LF and blank...
oka, will try
> > 4. mapi todo
> > - threading of messages does not work
>
> Is the cause known yet? Incomplete headers, such as In-Reply-To: or
> References:?
yes, only very few fields from the whole header is passed to mda. i have
to add the other fields, the problem is which ones, mapi has cca 1000 of
these tags :) i'll talk with openchange people about this.
> > - when > 1000 messages, fetchmail says it cannot open MDA
> > report(stderr, GT_("MDA open failed\n"));
>
> Try running with -vvv and see if you still get the "about to deliver
> with:" message, and please show it to me.
oka, next time, i had to run outlook and flush everything so every time
that happens i loose some of messages.
> Can I also see your mda option's argument?
mda "procmail -d x"
> In doubt, report the usual stuff I ask users to report,
> http://www.fetchmail.info/fetchmail-FAQ.html#G3
gentoo, i686-pc-linux-gnu
gcc version 4.3.4 (Gentoo 4.3.4 p1.0, pie-10.1.5)
mda = procmail v3.22 2001/09/10
using fetchmail -kvvv options. -F for separate flush
more info next time i'll investigate that behaviour.
> WRT descriptor leaks: Try "lsof -c fetchmail" in a separate console while
i did that in fact and according to lsof there seems not to be descriptor leak.
> strace or truss could help. You could also grep "ps ax" or "ps -ef" output
good idea - will do next time that happens.
weren't there some performance monitor for linux? three or four letters
that was... oh i cannot remember the name :)
> <http://www.fetchmail.info/fetchmail-FAQ.html#S2> - does it help to fiddle
> with the Exchange registry?
ee, i do not have access to exchange server as i am trying it on company's mail
server. well these messages seem non fatal, so i'd postpone this too.
i have to hurry back to work so bye and many thanks for your responses,
mojmir
|
|
From: Matthias A. <mat...@gm...> - 2009-09-02 19:49:07
|
Am 02.09.2009, 10:25 Uhr, schrieb mojmir svoboda
<moj...@2k...>:
> good morning Matthias,
>
> * mat...@gm... <mat...@gm...> [2009-08-28 06:15:43
> +0000]:
>
>> * Don't introduce security holes, avoid static buffers
>
> oh i must read some secure programming stuff, i was never involved :)
> by avoiding static buffers (local? global ones?) you mean allocate
> every buffer on heap?
Most importantly: don't trust data you got over the network, through
openchange or any other library. Don't assume data makes sense or has
sensible lenghts, or integers are positive or make sense. Sanity check
array indexes that they aren't out of bounds. Make sure string copies are
always length limited or that you the output is allocated properly. Do not
use functions that copy arbitrary length data. Note that strncpy has
awkward implications about the NUL byte termination, but you can use
strlcpy and strlcat instead.
>> > 4. mapi todo
>> > - threading of messages does not work
>>
>> Is the cause known yet? Incomplete headers, such as In-Reply-To: or
>> References:?
>
> yes, only very few fields from the whole header is passed to mda. i have
> to add the other fields, the problem is which ones, mapi has cca 1000 of
> these tags :) i'll talk with openchange people about this.
:-( That's tough luck. I had hoped that there were some function to fetch
internet headers fully (Outlook 2003 can do that, so I'd assume Exchange
can deliver that - but I may be wrong and Outlook just emulates it).
>
>> > - when > 1000 messages, fetchmail says it cannot open MDA
>> > report(stderr, GT_("MDA open failed\n"));
>>
>> Try running with -vvv and see if you still get the "about to deliver
>> with:" message, and please show it to me.
>
> oka, next time, i had to run outlook and flush everything so every time
> that happens i loose some of messages.
>
>> Can I also see your mda option's argument?
>
> mda "procmail -d x"
Hm. Should be safe - but make sure that "x" is sensible - procmail tries
to setuid to x, and if x doesn't exist, there's no way it will succeed --
the exact implications depend on whether fetchmail is in multidrop or
singledrop mode.
>> strace or truss could help. You could also grep "ps ax" or "ps -ef"
>> output
>
> good idea - will do next time that happens.
> weren't there some performance monitor for linux? three or four letters
> that was... oh i cannot remember the name :)
Hm. top? But that doesn't show leaks. There are several memory leak
checkers around, Google should come up with things like memprof, valgrind,
an efence spinoff with a new name that I forgot.
HTH
Best regards
--
Matthias Andree
|
|
From: Jelmer V. <je...@op...> - 2009-09-02 19:59:29
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Matthias Andree wrote: >>> strace or truss could help. You could also grep "ps ax" or "ps -ef" >>> output >> good idea - will do next time that happens. >> weren't there some performance monitor for linux? three or four letters >> that was... oh i cannot remember the name :) > > Hm. top? But that doesn't show leaks. There are several memory leak > checkers around, Google should come up with things like memprof, valgrind, > an efence spinoff with a new name that I forgot. > If you want to find memory leaks, please use the latest versions of the openchange libraries. A lot of memory leaks have been fixed recently. Cheers, Jelmer -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10rc1 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQIcBAEBAgAGBQJKnrAxAAoJEACAbyvXKaRXkxQP+wSbH6mlKk3S6v3+IyHPbJtt p1+gNgeD2fMLMIPmA8k/CtSa/JkFh30pmpiMPOhqbPMROUl612c2cMwaproc/lw/ WfId6kflWCZBmgyMSYssJeJyrgm19sKJC3uhw2teAxySqCyr50OIYrQ8/F8kVAbm 3IX2wBq6sbJBjLYgD2inWjvNCsKSxmbVc2Bte2CIWjs6Q6/iVAMhb+5Q5ac/I5bw LgdFCKEG4ffNaRgQ6jr3iEDfZtPyjzxgtmnL++d0oeeKlVZ21doUiLpYJWC65QHe lq5wRTdNFdv2s1KZPFzvZ73angjf2nEJHa6G245alwpZFBP6QyvgKkj9aap3QAei n9iGFKYtLAEoGcOUsOIztXSvqWjQ5z4N0qzfNLO1x6/xTO7ysApr0xi6/Fv/cJJy 9yUg7Gkvn+PDKx9BMyKY8/DRtPi1D59Wd7VjosyFu3KO3zmdmoFPcSaHipMOy810 mszw/Rw9e5yNwFovpxwOzLNpB6lQNfkUdObUmUkq5x7qTzmd2aeTAwcn3Oxwdfb7 1AzKDljPUOJO2Jlk88vDJiFe3hyIxTpbNM4kgumkSozA/i1l9dEp5HZvuAgjrZIp LDyQYHJlc0OVOdVFeBGo41aWuDilHgFswJ4ENv/EeO5r6uBhhmAbHr4fFrJXnRf+ DzTx3iQ2G3UO2KwcRvhb =xx+P -----END PGP SIGNATURE----- |
|
From: Matthias A. <mat...@gm...> - 2009-09-02 19:58:34
|
Am 02.09.2009, 19:50 Uhr, schrieb Jelmer Vernooij <je...@op...>: > If you want to find memory leaks, please use the latest versions of > the openchange libraries. A lot of memory leaks have been fixed recently. That's good news. Thanks to all those involved in the plugging. -- Matthias Andree |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X