Leafnode Git
Brought to you by:
m-a
Menu
▾
▴
Tree [c1e4d2] master / History
Matthias Andree
Read Me
===== IMPORTANT
Make sure that you have the most current version of leafnode installed.
Check http://sourceforge.net/projects/leafnode/ for updates.
You are encouraged to subscribe to the leafnode-announce mailing list,
available from the address above by clicking on "Lists".
===== TABLE OF CONTENTS
- OVERVIEW
- FILES AND DIRECTORIES
- THE FILTER FILE
- INSTALLATION
- UPDATE FROM VERSIONS < 1.6
- UPDATE FROM VERSIONS < 1.9.3
- UPDATE FROM VERSIONS < 1.9.20 -- IMPORTANT! INCOMPATIBLE CHANGES HERE!
- UPDATE FROM VERSIONS < 1.9.23 -- IMPORTANT! INCOMPATIBLE CHANGES HERE!
- UPDATE FROM VERSIONS < 1.11.0 -- IMPORTANT! INCOMPATIBLE CHANGES HERE!
- TROUBLESHOOTING (formerly called DEBUGGING)
- PROBLEMS
- HACKING
===== OVERVIEW
Leafnode is a USENET software package designed for small sites, with a
few tens of readers and only a slow link to the net.
It consists of several programs, three essential ones and several add-ons.
- Leafnode is the NNTP server. It talks to the normal news clients,
and stores readership data.
- Fetchnews is the NNTP news-gatherer. It looks at the readership data
Leafnode stores, and selects what groups to pull news from.
- Texpire is responsible for deleting old and uninteresting news. It
deletes all discussion threads that are old and not recently read.
These are the add-ons that ship with leafnode:
- Applyfilter lets you delete articles from your local news spool
which fit a certain pattern.
- Checkgroups inserts the titles of newsgroups into the newsgroup
database.
- Newsq shows which news are waiting to be transferred to your upstream
server.
There are third-party add-ons for leafnode, listed in the "ADD-ONS" file.
Only groups that someone has been reading in the past week are fetched
from the upstream NNTP server. When someone stops reading a group,
fetchnews will stop reading that group a week later (this is the default
which can be configured), and when someone starts reading a group, fetchnews
will grab all the articles it can in that group the next time it runs.
Here are Leafnode's distinguishing features:
- Uses very little disk space and bandwidth compared to other servers.
Obviously truer for 3-user sites than for 30-user, and probably wholly
untrue for 300-user sites.
- Easy configuration and maintenance. Leafnode tries very hard to
recover automatically from error situations.
- The newsspool can be easily manipulated by small scripts. Some tiny
examples are in the subdirectory tools/ .
As most programs, Leafnode has also weaknesses:
- Scales very badly.
- Logging and screen output are inconsistent at times.
The current version of leafnode is available from:
http://sourceforge.net/projects/leafnode/
http://www.leafnode.org/
There is also a leafnode mailing list. Send mail to
leafnode-list-subscribe@leafnode.org
to subscribe, or visit
https://www.leafnode.org/mailman/listinfo/leafnode-list/
The list subscription triggers a challenge email that you need to answer
before the subscription becomes effective, so that nobody can subscribe you
without your prior consent.
The list archives are at:
https://www.leafnode.org/pipermail/leafnode-list/
===== FILES AND DIRECTORIES
Leafnode puts its files in three separate directories: The spool
directory, the library directory, and the binaries directory. All
directories can be changed at compile time.
In the spool directory you find the stored news, the active file and
some other short-lived configuration file. It defaults to /var/spool/news
and can be changed at compilation time. There are some special
directories here; see the leafnode(8) man page.
The configuration directory contains long-lived configuration files. It
defaults to /etc/leafnode.
The binaries directory, /usr/local/sbin by default, contains the
executable programs applyfilter, texpire, fetchnews and leafnode.
The user directory, /usr/local/bin by default, contains the newsq
and leafnode-version programs.
===== THE FILTER FILE
Since version 1.8, leafnode supports filtering of incoming headers for
arbitrary regular expressions. This feature is by default deactivated.
To activate it, you must write a file which contains the regular
expressions to be filtered (the "filter file") and in addition point
leafnode to this file by entering a line of the format
filterfile = /path/to/your/filterfile
into your $(LIBDIR)/config.
The format of the filterfile is as follows:
1) One regular expression per line
2) Comments start with # at the beginning of a line
3) Empty lines are allowed
Here is a short example which will kill everything posted to alt.flame:
^Newsgroups:.*[, ]alt.flame$
^Newsgroups:.*[, ]alt.flame,
You cannot use
^Newsgroups:.*alt.flame
because this will also kill postings to alt.flame.nice or comp.alt.flame.
Since version 1.9.3 and up to 1.11.X, leafnode used Perl-compatible regular
expressions which are implemented in the PCRE library written by Philip
Hazel, University of Cambridge, UK. Perl regular expressions should be
explained in any reasonable book about Perl. There is also a whole book
about regular expressions published by O'Reilly. A rather technical
explanation is given in the pcre2(3) manual page.
Since version 1.12.0, leafnode uses the PCRE2 library by the same author,
see pcre2(3) or https://www.pcre.org/current/doc/html/index.html
or its home page https://github.com/PhilipHazel/pcre2
===== INSTALLATION
Read the file INSTALL.
===== UPDATE FROM VERSIONS < 1.6
There are some major changes from leafnode versions previous to 1.6.
These include leafnode-1.6alpha.
1) The format of the groupinfo file has changed.
2) Some files (including the groupinfo file) have moved from
/usr/lib/leafnode/ to /var/spool/news/leaf.node/
3) The article information for the main upstream server has
been moved from groupinfo into a separate file.
To update correctly, do a "make update" as root after you have
successfully completed "make install". See the file INSTALL for more
details.
===== UPDATE FROM VERSIONS < 1.9.3
There is one more change from leafnode versions prior to 1.9.3. These
include leafnode-1.9.3beta versions up to leafnode-1.9.3b5.
The change is that the groupinfo file is now sorted in a case-insensitive
manner. To update correctly, do a "make update" as root after you have
successfully completed "make install". See the file INSTALL for more
details.
===== UPDATE FROM VERSIONS < 1.9.20
Note: some things have changed in 1.9.20.
1. Fetchnews now tries to send XOVER to figure which articles to
download, older versions only used XHDR. Leafnode will automatically fall
back to using XHDR if XOVER fails.
2. Most leafnode programs enforce a fully-qualified unique domain name
and exit if gethostbyname("HOSTNAME") does not yield one. (HOSTNAME
is the hostname obtained from gethostname.) The important thing here
is that the hostname must be globally unique: only one computer (and
none else) in the world - yours - may use that name. You will lose
your own posts if this is not the case. Therefore, domain names like
"localhost.localdomain" do not qualify and are rejected.
This hostname need not be registered in DNS if it's explicitly
specified in the configuration file.
See the separate README-FQDN* documentation for details, it is
available (with identical contents) in plain text and HTML formats.
3. The behaviour of the out.going posting queue has changed. If a post
to an upstream server fails, the article is now moved straight into
failed.postings. Formerly, it would be tried on other servers.
4. In adverse conditions, when fetchnews cannot obtain the lock file, it
may leave files like /var/spool/news/leaf.node/lock.file.XXFjkle8w3
behind. You can remove these files at any time UNLESS fetchnews or
texpire is running at the same time.
===== UPDATE FROM VERSIONS < 1.9.23
--- REMOTE ACCESS TO LEAFNODE LISTENING ON NNTP PORT
If you want to give remote users (that are not in your LANs) access to your
server, you previously only needed to allow them in access in hosts.allow.
Since leafnode 1.9.23, an additional parameter is required to be set to allow
users outside of your LAN access to your leafnode. The config.example file
has documentation on how to set this parameter. This is meant as additional
security measure when some novice user hoses his access control
configuration, there have been recurring reports of "I did not ever touch
that newsgroup, but leafnode is pulling news for that group nonetheless" --
these may have been caused by unauthorized access that was accidentally
granted. Note: This code is somewhat stupid for IPv6, and considers link- or
site local addresses "local". The IPv4 code checks the leafnode computer's
addresses and netmasks instead.
DISTRIBUTORS: DO _NOT_ SET THAT PARAMETER OR EVEN CHANGE THE COMMENT IN
YOUR EXAMPLE CONFIG. YOU WILL BE PUBLICALLY ACCUSED ON BUGTRAQ ET AL.
WE CAN SAFELY AND SOUNDLY ASSUME THAT PEOPLE WHO WANT TO RUN WORLD-WIDE
ACCESSIBLE SERVERS CAN READ DOCUMENTATION.
===== UPDATE FROM VERSIONS < 1.11.0
1. Configuration change:
The parser for the configuration and filter files now understands quoted
strings on the right hand side of the configuration or filter lines, with
simple backslash (\) escapes, to allow for hash marks (#), quote marks (")
and leading or trailing blanks in passwords. If such characters are used in
filters or configuration options, you will need to change the lines, see the
leafnode(8) manual page for details.
2. IPv6 support in fetchnews:
fetchnews supports IPv6 since v1.11.0. This still needs the ./configure
option --with-ipv6 and, of course, an operating system with IPv6 and IPv6
network connectivity - the latter is not widespread yet. No special
configuration options beyond the compile-time option is required, fetchnews
will transparently use IPv6 addresses if the name lookup for a server yields
IPv6 addresses.
===== TROUBLESHOOTING (formerly called DEBUGGING)
To get more verbose logs out of leafnode, do this:
1. log in as root
2. in your /etc/leafnode/config, add a line "debugmode=1" to the top of
the file.
3. in your /etc/syslog.conf, add this line:
news.debug /var/log/news.all
There is a tab character between the columns here.
Note that a line with news.=debug is fundamentally different.
"news.debug" means "debug level or more severe" whereas
"news.=debug" means "only debug level".
DO NOT USE news.=debug, BUT CHANGE IT TO news.debug INSTEAD
if you want to see what's going on.
4. touch /var/log/news.all
5. kill -HUP `cat /var/run/syslogd.pid`
On some syslogds (Linux), you can add a - before the file name of step
#2 to have syslog write asynchronously for speed, like this:
news.* -/var/log/news.all
Again, there is a tab character between the columns here.
Some syslogds may complain about the asterisk in news.*. If yours does,
write news.debug instead of news.*.
===== PROBLEMS
If you don't find information about your problem below, please see the FAQ.
----- "fatal: libpcre.so.0: open failed: No such file or directory"
Add the path to your libpcre to LD_LIBRARY_PATH, for bash/ash/ksh/sh:
LD_LIBRARY_PATH=/usr/local/lib ; export LD_LIBRARY_PATH
(For csh/tcsh, go figure yourself. These shells are unsuitable for
anything, find the "csh programming considered harmful" article for
details. csh and tcsh are not supported by leafnode.)
You can make this change permanent with the crle(1) tool on some
systems.
----- make distcheck fails
Currently, make distcheck requires that a valid "news" user exists in
the system.
----- compiler warnings
Solaris 2.5 emits a lot of bogus warnings at compile time, complaining
about missing prototypes, for mkstemp, gethostname and snprintf; about
the scope of struct sockaddr_in. These warnings can safely be ignored.
----- portability -- other build-time problems
leafnode cannot be tested on all possible systems. Report compile or
run-time portability problems to the leafnode mailing list. However, of
the free operating systems (FreeBSD, NetBSD, OpenBSD, Linux), only
versions supported by the vendor are supported by leafnode. Hurd,
Windows and Solaris 1.x (SunOS 4.x) are unsupported, meaning that no
workarounds for these systems will be employed. CygWin and AmigaOS are
currently unsupported, but may be supported in the future. Feedback is
appreciated.
Bug and portability fixes for unsupported operating systems that don't
impair the performance on the supported systems will however be accepted
if sent.
----- other problems
You also may want to consult the Mini-Leafsite-HOWTO by Florian Kuehnert
<sutok@gmx.de> which was unfortunately written for leafnode-1.4 and is
therefore somewhat out of date. It is available from the usual places
(sunsite and mirrors).
If you want to ask on Usenet about leafnode, look in the newsgroups
news.software.nntp for English language and de.comm.software.newsserver
for German-language. Do not crosspost!
===== HACKING
Leafnode 1.X is in maintenance mode. No new features will be added. Only
major bugs will be fixed. Please file any feature requests against
leafnode 2.0beta.
Hackers, remember to ./configure --enable-maintainer-mode.
Cornelius Krasel <krasel@wpxx02.toxi.uni-wuerzburg.de>
Matthias Andree <matthias.andree@gmx.de>
vim:tw=76:
