noffle-devel

[Noffle-devel] [ noffle-Patches-479387 ] Minor corrections to Makefile.am

[<no...@so...>]

Patches item #479387, was opened at 2001-11-07 17:04
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=479387&group_id=1044

Category: None
Group: None
>Status: Closed
>Resolution: Accepted
>Priority: 3
Submitted By: Mirko Liss (mirkol)
Assigned to: Mirko Liss (mirkol)
Summary: Minor corrections to Makefile.am

Initial Comment:
for noffle 1.1.1-unstable-develop

1. copy noffle.conf.example to 
   $(CONFIGFILE).example
2. include some support for 
   filename transformations upon
   noffle main executable

Tested: Once, Linux, distribution suse 7.0


----------------------------------------------------------------------

Comment By: Jim Hague (bears)
Date: 2001-11-08 01:42

Message:
Logged In: YES 
user_id=184

Thanks again Mirko.

Looks good to me. I had to read up a bit on automake, but 
got there in the end.

Rather than add this myself (I'm a bit busy today and 
tomorrow), and since you obviously know what you're about, 
I've added you to the list of Noffle developers and 
assigned the patch to you. You should now have checkin 
rights to CVS(*). Can you apply the patch yourself? Don't 
forget to update ChangeLog too.

(*) So you won't need to go through the patch business in 
future, just check your changes straight in. Welcome 
aboard.


----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=479387&group_id=1044

[Noffle-devel] new filter option: filter post-status=yes|no|mod

[Mirko L. <mir...@we...>]

Hello,

I added a new filter option that checks the post-status.
Only the status of the current newsgroup will be checked.

Useful for statements like:

    filter post-status=moderated action=default
    filter from=".*@nym\.alias\.net" action=discard 
    filter size>50k action=over

(This skips all following filter statements for moderated
groups.)

I considered handling crossposts, but that creates the
same number of problems it solves. So I decided to keep it
simple.

Uh, by the way: Leftover temporary files in the overview/
directory will be deleted if you run expireContents().


I've looked at the USEFOR draft and I can't say I understand
how to implement most of these things, I'm sorry.
I'd rather add small features and fix some bugs until I've
gotten a better understanding of the source code.

Please don't expect me to cvs-commit the required patches
to src/post.c.


friendly regards,

Mirko

Re: [Noffle-devel] noffle --fetch [server pattern]

[Mirko L. <mir...@we...>]

Jim Hague wrote:
> On 12-Nov-2001 Mirko Liss wrote:
> > I just added the options:
> >     noffle --fetch [server pattern]
> >     noffle --query groups [server pattern]
> >     noffle --query desc [server pattern]
> That's a good idea. 

It's in the CVS now. Changes in src/noffle.c and docs/noffle.1.

> > On the other hand, when querying servers this way, one can
> > easily mess up the list of newsgroups. Using the noffle.conf
> > settings 'getgroups' and 'omitgroups' to get a largely
> > disjunct set of newsgroups can probihit this misuse, though.
> 
> I don't see that it would. From other messages it seems you are worrying about
> Noffle switching its source for a group from one server to another, and
> consequently trying to refetch messages needlessly.

Yes. Or not trying to fetch articles because it falsely believes to be
up-to-date after comparing the indices.

> First off, the refetch won't be that bad. Noffle will get the message IDs of
> the articles, discover it already has most or all of them, and only actually
> fetch articles it is missing.

My upstream server stores messages for 6 months. I set 'default-expire' to
14 days in noffle.conf. Noffle will refetch many articles from low and
medium-traffic newsgroups, only to expire them shortly afterwards, won't it?
I haven't tested this, though. It's just mere guesswork after leafing through
the code. And I'm often wrong at guessing.

> What I have wondered about is whether we should move to imitating Leafnode in
> the question of which server to use. Noffle currently associates a group with a
> single server. Leafnode, AIUI, attempts to retrieve articles for interesting
> groups from all servers it knows about. This is both good (a great way of
> getting a good newsfeed from several patchy upstreams) and bad (more time spent
> talking to each server - or does Leafnode keep separate group lists for each
> server?).

Getting the same newsgroup from different servers was something I haven't
thought about much. I'd never expected noffle (or leafnode) to get this done
the Right Way(tm). Isn't this a job for relayers higher up in the NetNews
food chain?

Leafnode tries to do this, but it throws away articles at any problem because
it doesn't want to force a newsadmin to babysit the database. You can
configure leafnode to connect to lots of upstream servers and merge their
newsfeed. But what do you gain by this? If at least one of the upstream servers
is well-administrated, you don't increase reliability significantly. But
you receive articles some hours earlier. Is that worth the extra bandwidth?

> I don't think it worth trying to change at this stage (i.e. before 1.2) -

I don't think it's worth a change today or later.

I believe there will be lots of computers with a slow network access in the
future. Laptops using cell-phones or wireless LANs won't have much bandwidth
to waste. Some mobile-access ISPs will definitely use pay-per-volume and not
pay-per-minute. An efficient newsserver that's optimized for low bandwidth
and that can recover from connection breakdown won't be obsolete.

Leafnode still suffers from bad network connections. Pull the plug and
fetchnews will hang indefinitely. It doesn't support time-outs and
I think even the locking has been patched recently.


Friendly regards,

Mirko

Re: [Noffle-devel] noffle --fetch [server pattern]

[Jim H. <jim...@ac...>]

On 12-Nov-2001 Mirko Liss wrote:
> I just added the options:
>     noffle --fetch [server pattern]
>     noffle --query groups [server pattern]
>     noffle --query desc [server pattern]
> 
> Servers in noffle.conf not matching the pattern 'server pattern'
> will be ignored when fetching articles or querying groups.
> Without a pattern, no servers will be ignored.

That's a good idea. 

> I think this is useful for people using upstream newsservers
> that aren't available all the time, i.e. non-public or broken
> servers.

Absolutely.

> On the other hand, when querying servers this way, one can
> easily mess up the list of newsgroups. Using the noffle.conf
> settings 'getgroups' and 'omitgroups' to get a largely
> disjunct set of newsgroups can probihit this misuse, though.

I don't see that it would. From other messages it seems you are worrying about
Noffle switching its source for a group from one server to another, and
consequently trying to refetch messages needlessly.

First off, the refetch won't be that bad. Noffle will get the message IDs of
the articles, discover it already has most or all of them, and only actually
fetch articles it is missing.

Secondly, switching servers will only happen if Noffle discovers a group is
available from a 'better' server than it has been using up to now. 'better' in
this case means 'specified earlier in noffle.conf'. So as long as you don't
alter the config info, nothing should change.

What I have wondered about is whether we should move to imitating Leafnode in
the question of which server to use. Noffle currently associates a group with a
single server. Leafnode, AIUI, attempts to retrieve articles for interesting
groups from all servers it knows about. This is both good (a great way of
getting a good newsfeed from several patchy upstreams) and bad (more time spent
talking to each server - or does Leafnode keep separate group lists for each
server?).

I don't think it worth trying to change at this stage (i.e. before 1.2) -
addressing the checking on posts and line length issues, OTOH, definitely
should be done.

-- 
Jim Hague - jim...@in... (Work), ji...@be... (Play)
Never trust a computer you can't lift.

Re: [Noffle-devel] Handling outgoing articles (unstable-develop)

[Jim H. <jim...@ac...>]

On 12-Nov-2001 Mirko Liss wrote:
> Jim Hague wrote:
>> On 11-Nov-2001 Mirko Liss wrote:
>> > g A single header line is limited to 2048 chars and Utl_getLn() throws
>> >   an error message if this limit is exceeded. But a single header
>> >   (including all continuations) has the same limit and throws a SIGSEGV.
>> >   This limit is not generous at all, btw. Think of References of a
>> >   long-winding and ancient thread.
>> 
>> Having a single line limited to 2048 chars is (I think) not unreasonable,
>> but
>> we should do better when combining into a single header line.
> 
> Um, I asked questions on the usenet and digged through some rfcs.
> I haven't found anything that obsoletes rfc1036, just maybe
> son-of-rfc1076. But rfc2822 obsoletes rfc0822 (email message format
> conventions). It doesn't limit the size of message headers, but limits
> the size of a single line to 1000 bytes: 998 bytes of text + CRLF.
> Rfc2822 recommends lines of max. 80 bytes: 78 bytes + CRLF

There is a current IETF draft replacing RFC1036. Check out 
http://www.ietf.org/ids.by.wg/usefor.html. This also specifies the 998 octet
(plus CRLF) limit, but does not limit the overall size of article headers.
BTW, http://www.ietf.org/ids.by.wg/nntpext.html is the source for Pravda on
NNTP itself.

> Leafnode 1.9 places no limits on outgoing headers (if you believe its
> advocates), but Leafnode 2.0b rejects postings with headers longer
> than 1000 bytes.
> 
> I suppose we should place that limit upon postings and return a "441"
> status to the newsreader.

The Usefor draft is less clear, simply stating that you must support at least
998 octets (plus CRLF). And there are stricter requirements placed on
'relaying' agents; you have to cope with any line length. By my reading,
however, given Noffle's intended usage it isn't a relaying agent. I think for
postings you are right; we should reject new posts with lines >1000 octets.

> But how do we treat incoming messages?

Incoming messages currently have lines > MAXCHAR octets split at MAXCHAR (i.e.
a CRLF is injected into the line at that point). Looking at the code, I don't
think it would be difficult to change it to deal with arbitary length lines
(for sensible values of arbitary :-)); currently we assume we're collecting in
lines and strip and re-add CRLF ourselves. I think it wouldn't be too hard to
alter the code to permit arbitary-length lines being collected and stored.
Dealing with individual header lines (not including continutations) exceeding
MAXCHAR would be rather more difficult, and by my reading unnecessary.

> Currently, Utl_getHeader() doesn't concat a folded header if it would
> exceed 2047 bytes, but chops off the last lines.
> 
> We can make Utl_getHeader() return an error flag and label the
> incoming message with sth like "X-NOFFLE-X-STATUS: HEADER-CORRUPTED".
> 
> Or we can use dynamic strings for headers of incoming messages.

I think (I haven't looked) this is only an issue when we need to go digging
through headers, and in that case using dynamic strings should be possible.

> I don't know how this affects gdbm, though.

Shouldn't affect it at all. It is just storing an arbitary chunk of data.

I've update the TODO file with some items that Mirko has raised and points
following from them. 

-- 
Jim Hague - jim...@in... (Work), ji...@be... (Play)
Never trust a computer you can't lift.

Re: [Noffle-devel] noffle --fetch [server pattern]

[Mirko L. <mir...@we...>]

Mardy wrote:
> On Mon, Nov 12, 2001 at 03:02:23AM +0100, Mirko Liss wrote:
> > On the other hand, when querying servers this way, one can
> > easily mess up the list of newsgroups. Using the noffle.conf
> > settings 'getgroups' and 'omitgroups' to get a largely
> > disjunct set of newsgroups can probihit this misuse, though.
> 
> I didn't understand this point; how could the list get messed up?

When you fetch articles from an upstream server, there are
two ways to identify a message:
    a. by its Message-Id
    b. by its number in the current newsgroup.
The number is not the same on different servers.
Noffle uses the number to decide which articles to fetch and
the fetched articles are stored in the database by Message-Id.

Lets say this is your noffle.conf file:
    server news.free-and-fast.org
    server news.my-ancient-college-server-is-broken.edu
    max-fetch 200

And you placed only one newsgroup on the fetchlist:
    alt.fan.golden-newsfeed

Let's telnet to the first one:
$ telnet news.free-and-fast.org nntp
    Trying 327.0.0.1...
    Connected to news.free-and-fast.org.
    Escape character is '^]'.
    200 NNTP server NOFFLE 1.6.1803398875-our-guru-patched-it
group alt.fan.golden-newsfeed
    211 37 567 603 alt.fan.reliable-newsfeed selected

"211" is just a status flag
"37" is the number of stored articles
"567" is the number of the oldest article
"607" is the number of the article that arrived most recently.

Let's compare this with the secondary server:

    Trying 427.0.0.1...
    Connected to news.my-ancient-college-server-is-broken.edu.
    Escape character is '^]'.
    200 NNTP server NOFFLE 95^H^H98^H^HXP-gee-i-hope-this-works
group alt.fan.golden-newsfeed
    211 2034 22803 24836 alt.fan.reliable-newsfeed selected

What happens if you get your news from the secondary server
without querying that server for your newsgroup?

Noffle requests the last 200 articles (see max-fetch) and
stores them in the database. Oops.

And yes, this can go wrong the other way, as well, with noffle
not requesting articles for years.

The patch I suggested doesn't solve this problem at all.

> I once complained it was a bit annoying that when you add a new server
> in noffle.conf you had to refetch the list of all the groups (even of
> the ones in the servers present before); I was suggested to remove
> temporarily the old servers from noffle.conf, query the groups and then
> re-write the servers back in the config file. This patch would help
> greatly, wouldn't it?

In this situation, it wouldn't help that much, I'm sorry. You have to
edit noffle.conf anyway. If speed of the query matters to you,
you should add a "getgroups <newsgroupspattern>" to your new server entry
in noffle.conf. This reduces the number of queried newsgroups.
Any "omitgroups <nonewsgroupspattern>" doesn't speed up
the query of the new server, but may help avoid problems.

And there _will_ be problems like the aforementioned one, if noffle
queries a group from server B but fetches articles posted to this
newgroup from server A.

If you want to dig into the sources, please look at:
    src/client.c:Client_getGrps()


The patch I suggested doesn't help you avoid querying the wrong
server for a newsgroup hierarchy. The only way to avoid this
is to use "getgroups" and "omitgroups" in noffle.conf to set
newsgroups patterns for each server.

If you always query groups and fetch articles from
all the servers, it is safe to set noffle.conf to:

    server news.free-and-fast.org
    server news.my-ancient-college-server-is-unreliable.edu
    server news.binaries-if-you-pay-us.biz FreddieFoobar OhSoSecret 

With the new suggested patch (and with the hack you
wrote about) you'd better write:

    server news.free-and-fast.org
    omitgroups my-college.*
    omitgroups alt.binaries.useless.*
    server news.my-ancient-college-server-is-unreliable.edu          
    getgroups my-college.*
    server news.binaries-if-you-pay-us.biz FreddieFoobar OhSoSecret
    getgroups alt.binaries.useless.*


I am not sure if my patch should really be added to the
cvs. It is a bit dangerous because of these issues and
because some users might perhaps think the new optional
pattern after noffle -f or noffle -q .. didn't select
previously configured servers but newsgroups.

I've got access to the cvs tree for just a few days.
I believe I shouldn't throw too much garbage into the
sources, so I'll wait for an ok from one of the guys
who worked hard to pave the way.

For further questions and to get even more detailed
explanations mixed with mindless chatter, please
reply to:
    mir...@we...


Cordiali saluti,

Mirko Liss

Re: [Noffle-devel] noffle --fetch [server pattern]

[Mardy <ma...@us...>]

On Mon, Nov 12, 2001 at 03:02:23AM +0100, Mirko Liss wrote:
> I just added the options:
>     noffle --fetch [server pattern]
>     noffle --query groups [server pattern]
>     noffle --query desc [server pattern]
> 
> Servers in noffle.conf not matching the pattern 'server pattern'
> will be ignored when fetching articles or querying groups.
> Without a pattern, no servers will be ignored.

Great!

> On the other hand, when querying servers this way, one can
> easily mess up the list of newsgroups. Using the noffle.conf
> settings 'getgroups' and 'omitgroups' to get a largely
> disjunct set of newsgroups can probihit this misuse, though.

I didn't understand this point; how could the list get messed up?

I once complained it was a bit annoying that when you add a new server
in noffle.conf you had to refetch the list of all the groups (even of
the ones in the servers present before); I was suggested to remove
temporarily the old servers from noffle.conf, query the groups and then
re-write the servers back in the config file. This patch would help
greatly, wouldn't it?

-- 
Saluti,
    Mardy
http://www.interlingua.com

Re: [Noffle-devel] Handling outgoing articles (unstable-develop)

[Mirko L. <mir...@we...>]

Jim Hague wrote:
> On 11-Nov-2001 Mirko Liss wrote:
> > g A single header line is limited to 2048 chars and Utl_getLn() throws
> >   an error message if this limit is exceeded. But a single header
> >   (including all continuations) has the same limit and throws a SIGSEGV.
> >   This limit is not generous at all, btw. Think of References of a
> >   long-winding and ancient thread.
> 
> Having a single line limited to 2048 chars is (I think) not unreasonable, but
> we should do better when combining into a single header line.

Um, I asked questions on the usenet and digged through some rfcs.
I haven't found anything that obsoletes rfc1036, just maybe
son-of-rfc1076. But rfc2822 obsoletes rfc0822 (email message format
conventions). It doesn't limit the size of message headers, but limits
the size of a single line to 1000 bytes: 998 bytes of text + CRLF.
Rfc2822 recommends lines of max. 80 bytes: 78 bytes + CRLF

Leafnode 1.9 places no limits on outgoing headers (if you believe its
advocates), but Leafnode 2.0b rejects postings with headers longer
than 1000 bytes.

I suppose we should place that limit upon postings and return a "441"
status to the newsreader.


But how do we treat incoming messages?

Currently, Utl_getHeader() doesn't concat a folded header if it would
exceed 2047 bytes, but chops off the last lines.

We can make Utl_getHeader() return an error flag and label the
incoming message with sth like "X-NOFFLE-X-STATUS: HEADER-CORRUPTED".

Or we can use dynamic strings for headers of incoming messages.
I don't know how this affects gdbm, though.

What do you think about it? Rfc2822 is just half a year old, and I
suppose we have to expect to fetch messages with long headers for
quite some time.


friendly regards,

Mirko

[Noffle-devel] noffle --fetch [server pattern]

[Mirko L. <mir...@we...>]

Hello,

I just added the options:
    noffle --fetch [server pattern]
    noffle --query groups [server pattern]
    noffle --query desc [server pattern]

Servers in noffle.conf not matching the pattern 'server pattern'
will be ignored when fetching articles or querying groups.
Without a pattern, no servers will be ignored.


I think this is useful for people using upstream newsservers
that aren't available all the time, i.e. non-public or broken
servers.

On the other hand, when querying servers this way, one can
easily mess up the list of newsgroups. Using the noffle.conf
settings 'getgroups' and 'omitgroups' to get a largely
disjunct set of newsgroups can probihit this misuse, though.

It's just a dozen lines of code and seems to work without
errors. What do you think about it?

I'll commit the changes if you give me a thumbs up.
Patches via email upon request.


Friendly regards,

Mirko

Re: [Noffle-devel] Handling outgoing articles (unstable-develop)

[Markus E. <me...@ma...>]

On Sun, Nov 11, 2001 at 04:20:00PM +0100, Mirko Liss wrote:

> > The reason for this feature was, that the biggest German Internet provider
> > T-Online does replace the From-header of postings by the email address of the
> > account holder, if your email address is elsewhere, this is an annoying
> > thing.
> 
> Are you sure that this annoyance can't be switched off?
> I gdbmgrep-ed through my newsspool and found:
> 
> | Message-ID: <9q9jap$29a$00$2...@ne...>
> | From: Matthias Kordell <m_k...@gm...>
> | Date: Sat, 13 Oct 2001 16:32:56 +0200

Maybe T-Online changed this now (I know that they still replace the
From-header of sent email on their default mail servers),
but it was a concern at the time when I implemented it.

Of course, if nobody needs this workaround anymore, it should be removed
from the sources.

- Markus
 

Re: [Noffle-devel] Handling outgoing articles (unstable-develop)

[Mirko L. <mir...@we...>]

Markus Enzenberger wrote:
> On Sun, Nov 11, 2001 at 12:01:46PM -0000, Jim Hague wrote:
> > > c It adds a redundant 'Reply-To:'-header if none of present. That's not
> > >   a Bad Thing I guess, but rather undesirable.
> > Agreed.
> The reason for this feature was, that the biggest German Internet provider
> T-Online does replace the From-header of postings by the email address of the
> account holder, if your email address is elsewhere, this is an annoying
> thing.

Are you sure that this annoyance can't be switched off?
I gdbmgrep-ed through my newsspool and found:

| Message-ID: <9q9jap$29a$00$2...@ne...>
| From: Matthias Kordell <m_k...@gm...>
| Date: Sat, 13 Oct 2001 16:32:56 +0200

> It should be a config option however and be switched off by default.

What about adding a list of known issues with popular upstream servers
and suggested workarounds to the docs? With entries like:


Server: news.cis.dfn.de:119
last updated: 11 Nov 2001 Mirko Liss 
Access: public; password required; posting allowed
Known issues: Kills connection when trying to post approved articles.
    Replaces 'Path:' header. Adds several 'X-' headers.
    Slow, esp. when requesting group list.
Suggested workarounds: None, just don't post approved articles.


May I suggest requesting users' experiences?


Friendly regards,

Mirko

Re: [Noffle-devel] Handling outgoing articles (unstable-develop)

[Mirko L. <mir...@we...>]

Jim Hague wrote:
> (Some off the cuff thoughts. I need to more time to check I have the current
> RFCs to check some of these points).

I didn't intend to inflict any pain on you. I'll try not to use manacles
outside the master bedroom again. ;-)


I've fixed the SIGSEGV in src/util.c last night but didn't change the header
size limit.

If we need to discuss RFCs then I suggest moving to a public newsgroup to
receive the blessings of higher beings. I'm not a professional programmer so
these technical details are quite difficult for me to understand. I don't
know what to think about the suggestions in son-of-rfc1036. It's an
unofficial document, after all.


Friendly regards,

Mirko

Re: [Noffle-devel] Handling outgoing articles (unstable-develop)

[Markus E. <me...@ma...>]

On Sun, Nov 11, 2001 at 12:01:46PM -0000, Jim Hague wrote:

> > c It adds a redundant 'Reply-To:'-header if none of present. That's not
> >   a Bad Thing I guess, but rather undesirable.
> 
> Agreed.

The reason for this feature was, that the biggest German Internet provider
T-Online does replace the From-header of postings by the email address of the
account holder, if your email address is elsewhere, this is an annoying
thing.

It should be a config option however and be switched off by default.

- Markus

Re: [Noffle-devel] Handling outgoing articles (unstable-develop)

[Jim H. <jim...@ac...>]

(Some off the cuff thoughts. I need to more time to check I have the current
RFCs to check some of these points).

On 11-Nov-2001 Mirko Liss wrote:
> when leafing through src/post.c and protocol.c, I saw that noffle does
> some weird things with outgoing articles:
> 
> a I haven't seen a place in src/post.c where all components of the
>   'static struct Article article' are re-initialized. Maybe I'm too
>   careful, but I'm afraid this could lead to problems. Think of two
>   outgoing articles with the second one lacking a 'References:' line.
>   The static variable article.over.ref would still be set to the old
>   value. The tidying-up in src/post.c:Post_close() doesn't reset any
>   article.over components. What about moving the content of Post_close()
>   to a new static function postInit() and calling it from Post_open(),
>   btw?

It would definitely be better to move the initialisation to a single place.
However, the current situation isn't in fact quite as bad as you think - the
static article overview info does get reset, but it is carefully concealed
(abeit unintentionally) at the start of getArticleText.

> b Noffle doesn't allow posting to external groups with post-status 'n',
>   even if an 'Approved:' header is present. The newsgroup alt.dev.null
>   has this status and its purpose is the testing of self-approved
>   articles. At least if I can trust the FAQ of alt.tech-support.recovery
>   from Tim Franklin.

Currently it won't allow posting to any post-status 'n' group. And alt.dev.null
is (on my upstream feed) marked 'm', so Noffle should post articles posted
there to the upstream server, regardless of presence or absence of Approved:
header. Approved: only matters to Noffle for local groups. 

> c It adds a redundant 'Reply-To:'-header if none of present. That's not
>   a Bad Thing I guess, but rather undesirable.

Agreed.

> d It insists on adding a 'Path:' if none is present. Can we please
>   switch this off unless 'post-locally' is set to 'yes' in noffle-conf?

Agreed. As I read it (though I need to spend some more time on this), Path:
shouldn't be added if we're acting like a news client, but should be added if
we're being a news trasnport.

I originally added Path: when implementing post-locally and local groups, when
it occurred to me that I was getting articles in the newsbase without a Path:.

> e Empty headers won't be deleted. I was able to post articles with a
>   blank 'From:   ' header, for example.
>   What about testing if '@' and '.' is present?
> f There's no check for duplicate headers. It is no problem to send an
>   article with eg. two 'From:' lines. See son-of-rfc1036, chapter 5. 

Further error checking of articles would be a Good Thing.

> g A single header line is limited to 2048 chars and Utl_getLn() throws
>   an error message if this limit is exceeded. But a single header
>   (including all continuations) has the same limit and throws a SIGSEGV.
>   This limit is not generous at all, btw. Think of References of a
>   long-winding and ancient thread.

Having a single line limited to 2048 chars is (I think) not unreasonable, but
we should do better when combining into a single header line.

> I'm sorry for all this hair-splitting and I hope I have just wasted
> your time by discussing problems that have already been resolved.

Not at all. Fresh input always welcome.

-- 
Jim Hague - jim...@in... (Work), ji...@be... (Play)
Never trust a computer you can't lift or you don't control.

[Noffle-devel] Handling outgoing articles (unstable-develop)

[Mirko L. <mir...@we...>]

Hello,

when leafing through src/post.c and protocol.c, I saw that noffle does
some weird things with outgoing articles:

a I haven't seen a place in src/post.c where all components of the
  'static struct Article article' are re-initialized. Maybe I'm too
  careful, but I'm afraid this could lead to problems. Think of two
  outgoing articles with the second one lacking a 'References:' line.
  The static variable article.over.ref would still be set to the old
  value. The tidying-up in src/post.c:Post_close() doesn't reset any
  article.over components. What about moving the content of Post_close()
  to a new static function postInit() and calling it from Post_open(),
  btw?
b Noffle doesn't allow posting to external groups with post-status 'n',
  even if an 'Approved:' header is present. The newsgroup alt.dev.null
  has this status and its purpose is the testing of self-approved
  articles. At least if I can trust the FAQ of alt.tech-support.recovery
  from Tim Franklin.
c It adds a redundant 'Reply-To:'-header if none of present. That's not
  a Bad Thing I guess, but rather undesirable.
d It insists on adding a 'Path:' if none is present. Can we please
  switch this off unless 'post-locally' is set to 'yes' in noffle-conf?
  I don't understand the purpose of this. Please refer to son-of-rfc1036,
  chapter 5.6. My upstream server news.cis.dfn.de changes my 'Path:' to
  'X-Path:' and adds its own. If it didn't do that, it wouldn't rightfully
  send me my own postings back.
e Empty headers won't be deleted. I was able to post articles with a
  blank 'From:   ' header, for example.
  What about testing if '@' and '.' is present?
f There's no check for duplicate headers. It is no problem to send an
  article with eg. two 'From:' lines. See son-of-rfc1036, chapter 5. 
g A single header line is limited to 2048 chars and Utl_getLn() throws
  an error message if this limit is exceeded. But a single header
  (including all continuations) has the same limit and throws a SIGSEGV.
  This limit is not generous at all, btw. Think of References of a
  long-winding and ancient thread.

Please refer to son-of-rfc1036, chapter 4.2, esp. 4.2.2 for further 
details on undesirable headers. I consider noffle is rather a
'posting agent' than a 'relayer', esp. in inews mode.
I suppose some additional checking should be performed.

What do you think about putting some additional checks of outgoing
article headers onto the TODO list?

I'm sorry for all this hair-splitting and I hope I have just wasted
your time by discussing problems that have already been resolved.


Friendly regards,

Mirko Liß

[Noffle-devel] [ noffle-Patches-479387 ] Minor corrections to Makefile.am

[<no...@so...>]

Patches item #479387, was opened at 2001-11-07 17:04
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=479387&group_id=1044

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Mirko Liss (mirkol)
>Assigned to: Mirko Liss (mirkol)
Summary: Minor corrections to Makefile.am

Initial Comment:
for noffle 1.1.1-unstable-develop

1. copy noffle.conf.example to 
   $(CONFIGFILE).example
2. include some support for 
   filename transformations upon
   noffle main executable

Tested: Once, Linux, distribution suse 7.0


----------------------------------------------------------------------

>Comment By: Jim Hague (bears)
Date: 2001-11-08 01:42

Message:
Logged In: YES 
user_id=184

Thanks again Mirko.

Looks good to me. I had to read up a bit on automake, but 
got there in the end.

Rather than add this myself (I'm a bit busy today and 
tomorrow), and since you obviously know what you're about, 
I've added you to the list of Noffle developers and 
assigned the patch to you. You should now have checkin 
rights to CVS(*). Can you apply the patch yourself? Don't 
forget to update ChangeLog too.

(*) So you won't need to go through the patch business in 
future, just check your changes straight in. Welcome 
aboard.


----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=479387&group_id=1044

[Noffle-devel] [ noffle-Patches-476758 ] configure --with-spooldir --with-etcdir

[<no...@so...>]

Patches item #476758, was opened at 2001-10-31 05:28
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=476758&group_id=1044

Category: None
Group: None
>Status: Closed
Resolution: Wont Fix
Priority: 5
Submitted By: Mirko Liss (mirkol)
Assigned to: Jim Hague (bears)
Summary: configure --with-spooldir --with-etcdir

Initial Comment:
Added options --with-etcdir 
and --with-spooldir to configure

Why?

I want to use two different versions
of noffle on the same system.

Tested?

Yes, once. On Linux,
distribution SuSE 7.0.


----------------------------------------------------------------------

Comment By: Jim Hague (bears)
Date: 2001-11-06 13:28

Message:
Logged In: YES 
user_id=184

Good idea again, Mirko. And I see you've got patch 
uploading working properly.

Unfortunately the patch is patching the wrong thing. 
'configure' is generated by the GNU autoconf tool from 
'configure.in'; you change alters 'configure' and the 
change will be lost when autoconf is next run.

I've had a go at doing --with-spooldir and 
--with-configfile (which I thought might be a little bit 
more flexible) and the result is in CVS. I'm no autoconf 
expert - please check it out and see what you think. I'd 
welcome improvements.


----------------------------------------------------------------------

Comment By: Jim Hague (bears)
Date: 2001-11-06 06:02

Message:
Logged In: YES 
user_id=184




----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=476758&group_id=1044

[Noffle-devel] [ noffle-Patches-479387 ] Minor corrections to Makefile.am

[<no...@so...>]

Patches item #479387, was opened at 2001-11-07 17:04
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=479387&group_id=1044

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Mirko Liss (mirkol)
>Assigned to: Jim Hague (bears)
Summary: Minor corrections to Makefile.am

Initial Comment:
for noffle 1.1.1-unstable-develop

1. copy noffle.conf.example to 
   $(CONFIGFILE).example
2. include some support for 
   filename transformations upon
   noffle main executable

Tested: Once, Linux, distribution suse 7.0


----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=479387&group_id=1044

[Noffle-devel] [ noffle-Patches-479387 ] Minor corrections to Makefile.am

[<no...@so...>]

Patches item #479387, was opened at 2001-11-07 17:04
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=479387&group_id=1044

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Mirko Liss (mirkol)
Assigned to: Nobody/Anonymous (nobody)
Summary: Minor corrections to Makefile.am

Initial Comment:
for noffle 1.1.1-unstable-develop

1. copy noffle.conf.example to 
   $(CONFIGFILE).example
2. include some support for 
   filename transformations upon
   noffle main executable

Tested: Once, Linux, distribution suse 7.0


----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=479387&group_id=1044

[Noffle-devel] [ noffle-Patches-476758 ] configure --with-spooldir --with-etcdir

[<no...@so...>]

Patches item #476758, was opened at 2001-10-31 05:28
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=476758&group_id=1044

Category: None
Group: None
Status: Open
>Resolution: Wont Fix
Priority: 5
Submitted By: Mirko Liss (mirkol)
Assigned to: Jim Hague (bears)
Summary: configure --with-spooldir --with-etcdir

Initial Comment:
Added options --with-etcdir 
and --with-spooldir to configure

Why?

I want to use two different versions
of noffle on the same system.

Tested?

Yes, once. On Linux,
distribution SuSE 7.0.


----------------------------------------------------------------------

>Comment By: Jim Hague (bears)
Date: 2001-11-06 13:28

Message:
Logged In: YES 
user_id=184

Good idea again, Mirko. And I see you've got patch 
uploading working properly.

Unfortunately the patch is patching the wrong thing. 
'configure' is generated by the GNU autoconf tool from 
'configure.in'; you change alters 'configure' and the 
change will be lost when autoconf is next run.

I've had a go at doing --with-spooldir and 
--with-configfile (which I thought might be a little bit 
more flexible) and the result is in CVS. I'm no autoconf 
expert - please check it out and see what you think. I'd 
welcome improvements.


----------------------------------------------------------------------

Comment By: Jim Hague (bears)
Date: 2001-11-06 06:02

Message:
Logged In: YES 
user_id=184




----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=476758&group_id=1044

[Noffle-devel] [ noffle-Patches-476758 ] configure --with-spooldir --with-etcdir

[<no...@so...>]

Patches item #476758, was opened at 2001-10-31 05:28
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=476758&group_id=1044

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Mirko Liss (mirkol)
>Assigned to: Jim Hague (bears)
Summary: configure --with-spooldir --with-etcdir

Initial Comment:
Added options --with-etcdir 
and --with-spooldir to configure

Why?

I want to use two different versions
of noffle on the same system.

Tested?

Yes, once. On Linux,
distribution SuSE 7.0.


----------------------------------------------------------------------

>Comment By: Jim Hague (bears)
Date: 2001-11-06 06:02

Message:
Logged In: YES 
user_id=184




----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=301044&aid=476758&group_id=1044

[Noffle-devel] Noffle 1.1 released

[Jim H. <jim...@ac...>]

I've done a final merge from the 1.0 branch into main, and taken the liberty of
packaging up the current CVS as Noffle 1.1. This is intended to be the first of
a (hopefully) not too long development branch leading to a stable 1.2. Right
now there is a source tarball on SourceForge, and I'm working on RPMs for i386
and Sparc. When they are done I'll let FreshMeat know.

I have also taken the liberty of reclassifying Noffle in Trove as 'Stable'. 1.0
and 1.0.1 having been out for a while, it seemed past 'beta' by now.

My intention is that 1.2 will be Noffle fundamentally much as it is now. Bug
fixes and extension to existing functionality (e.g. more filter rules - Mirko
Liss has a nice idea for a group posting status filter rule). BTW, I haven't
time to be terribly thorough about release testing at the moment; if changes go
into CVS I will try to get new 1.1.x releases out promptly at the risk of
making an idiot of myself every so often. Please also bear with me over periods
of inactivity - unless someone can upgrade the number of hours in the day
substantially, Real Life will win sometimes.

Once that is nicely along I have a notion of starting work on Noffle 2.0.
Design hand waving starts below. The big change I have in mind here is to add a
different newsbase to the current gdbm one. In particular, I want something
along the lines of the leafnode newsbase.

The reason behind this is simply that I have over time had a couple of people
comment to me that they'd like to use Noffle, but they rely on various scripts
to collect information of interest to them from the traditional
/var/spool/news newsbase. Leafnode works for them, Noffle doesn't. I would also
be hoping as part of this exercise to remove the single choke point of a gdbm
database that can't be opened by multiple processes at once.

In the past I've worried at Markus that this would not be easy, and he's told
me not to be such a twit. With regards to Leafnode, I now see that the
components in its newsbase do map nicely to Noffle's.

At this stage I'm uncertain as to whether Leafnode's newsbase should be
supported exactly (I should say here I mean article & message ID databases,
group list, overviews, NOT the out.going and other directories that Leafnode
uses for internal purposes), or whether to try to improve on Leafnode a bit.
The thing about Leafnode I don't much like is the group list. This is a single
text file with groups held in alphabetical order. Handling alterations to it is
a right pain - Leafnode goes to some trouble to sort and merge efficiently -
and (more importantly) locking a single group while accessing it would be
tricky. We could lock the whole thing, but that nicely re-introduces a single
choke point. Everything else we need to lock can be locked on at a group level,
so two processes accessing comp.risks and comp.compilers can proceed in full
parallel.

However, I don't have a nice alternative for the group list right now. Ideally
it would be cheap to implement, ubiquitous (why should Noffle just be Linux),
and support record-level or near locking. I'm wondering about a file system
based scheme similar to the message IDs(*), e.g. a sorted text file in
grouplist/alt/foo, but I'm not convinced that it would be a good idea. There's
over 700 top level domains, for a start.

(*) If you've never looked at Leafnode, roughly its message ID database is a
file hierarchy of hard links to article files stored in a conventional
newsbase. E.g. article <xy...@zo...>, number 25 in alt.foo.bar has its
article file in group/alt/foo/bar/25, and a hard link in
message-id/x/y/z/xy...@zo....

-- 
Jim Hague - jim...@in... (Work), ji...@be... (Play)
Never trust a computer you can't lift or you don't control.

Re: [Noffle-devel] Re: Noffle main CVS line

[Markus E. <me...@ma...>]

On Tue, Oct 30, 2001 at 09:00:04AM -0000, Jim Hague wrote:
> On 28-Oct-2001 Markus Enzenberger wrote:
> > Is it ok, if you take over the maintainance for the main CVS line, Jim?
> 
> Yes, of course. Although I have a cable modem these days, I do still use Noffle
> for all my news reading, because I need to have local newsgroups, and I'm
> addicted to an old newsreader (trn) which can't really cope with reading news
> from multiple sources. So Noffle consolidates my local groups and external
> groups from a couple of sources into one place.

Great, I am happy with that, Jim, and think that the future Noffle is
in very good hands at your side.

> As you'll have noticed, my Noffle activities are, er, sporadic, as Real Life
> and other projects intervene. However, I am currently plotting a switch to a

No problem, you did an excellent job so far, and it is a pleasure working
with you. I am sure you will get a lot of help from users like I did.

> Leafnode-like newsbase. I'll explain more in another post.

I am looking forward to read it.

See you

- Markus

[Noffle-devel] Re: Noffle main CVS line

[Jim H. <jim...@ac...>]

On 28-Oct-2001 Markus Enzenberger wrote:
> there are 2 patches and a bug-fix pending for the main CVS line
> on Noffle's Sourceforge page.
> 
> Jim, may I ask you to check and apply them?
> They partially affect the filter code you wrote.

Yes, will do.

> All patches are from Mirko Liss. Maybe you want to contact him,
> if any questions arise, or give developer access to him.

He does seem to have been giving things a close look. I'll get in touch.

> The bad news is, that I won't be able to do much maintaining stuff
> for the main Noffle line in the future.
> One of the reasons is that I do not use Noffle personally a lot now.
> Since I got a DSL connection at home combined with a flatrate,
> there is no much need for me for a news server optimized for low-bandwith
> dial-up connections anymore.
> 
> Is it ok, if you take over the maintainance for the main CVS line, Jim?

Yes, of course. Although I have a cable modem these days, I do still use Noffle
for all my news reading, because I need to have local newsgroups, and I'm
addicted to an old newsreader (trn) which can't really cope with reading news
from multiple sources. So Noffle consolidates my local groups and external
groups from a couple of sources into one place.

As you'll have noticed, my Noffle activities are, er, sporadic, as Real Life
and other projects intervene. However, I am currently plotting a switch to a
Leafnode-like newsbase. I'll explain more in another post.

Thank you for Noffle. It is a real pleasure to work on good quality base code.

I shall try and get a 1.1 (leading to 1.2) release out soon.

-- 
Jim Hague - jim...@in... (Work), ji...@be... (Play)
Never trust a computer you can't lift or you don't control.

[Noffle-devel] Noffle main CVS line

[Markus E. <me...@ma...>]

Hello,

there are 2 patches and a bug-fix pending for the main CVS line
on Noffle's Sourceforge page.

Jim, may I ask you to check and apply them?
They partially affect the filter code you wrote.
All patches are from Mirko Liss. Maybe you want to contact him,
if any questions arise, or give developer access to him.

The bad news is, that I won't be able to do much maintaining stuff
for the main Noffle line in the future.
One of the reasons is that I do not use Noffle personally a lot now.
Since I got a DSL connection at home combined with a flatrate,
there is no much need for me for a news server optimized for low-bandwith
dial-up connections anymore.

Is it ok, if you take over the maintainance for the main CVS line, Jim?

I do promise to keep up the maintaince of the "release-1-0" branch,
there isn't happening a lot there anyway, it seems pretty stable now.
Paul Slootman recently did an update of Noffle's Debian package
to version 1.0.1.

Thanks for all the help

- Markus