UnrealIRCd 4.0.1 released

[Bram M. <sy...@vu...>]

It's time for an update to the UnrealIRCd 4 series. In UnrealIRCd 4.0.1 we fix 
two crash issues & more, see below.

Thanks to everyone who provided feedback and suggestions!

*Changes between version 4.0.0 and 4.0.1
*Enhancements

  * The blacklist module <https://www.unrealircd.org/docs/Blacklist_block> now
    supports %ip (=banned IP) in blacklist::reason.
  * *NIX: You can use cron again, see https://www.unrealircd.org/docs/Cron_job
  * /MODULE now lists only 3rd party modules by default so you don't get flooded.
  * *NIX: Added './unrealircd reloadtls' to reload TLS certificate and keys.

Major issues fixed

  * Possible crash on-link if a user was in the process of connecting during
    linking
  * Crash if you removed a listen { } block with active clients on that port
  * MODEs set by a server (not by a user) were not always propagated correctly
    across the network. In practice this only affected /SAMODE and possibly
    some services that don't send MODEs from ChanServ/BotServ.

Minor issues fixed

  * When doing /LIST under mIRC it would hide empty +P channels.
  * Servers wouldn't link if link::outgoing::hostname was a CNAME.
  * SSL Certificate fingerprint not communicated properly to servers/services.
  * *NIX: ./unrealircd [stop|rehash] failed if not installed to ~/unrealircd.
  * Windows: IRCd could crash after showing the config error screen on startup.
  * Possibly some interoperability issues with services.

*What's new in UnrealIRCd 4
*A short overview of the most important changes:*
*

  * <https://www.unrealircd.org/docs/Modules>You decide what to load
    <https://www.unrealircd.org/docs/Modules>. We have moved as much
    functionality as possible to 150+ individually loadable modules (commands
    <https://www.unrealircd.org/docs/User_%26_Oper_commands>, user modes
    <https://www.unrealircd.org/docs/User_modes>, channel modes
    <https://www.unrealircd.org/docs/Channel_modes>, extbans
    <https://www.unrealircd.org/docs/Extended_bans>, snomasks, ..). You decide
    which features your UnrealIRCd should have.
  * Fine-grained IRCOp privileges
    <https://www.unrealircd.org/docs/Operclass_block>. The way IRCOp
    privileges are granted has been redone entirely. This allows you to
    configure oper privileges on a very detailed level. You don't want
    OperOverride? You don't want opers to see secret channels? Or you want an
    oper with a very minimal set of privileges? This is all possible.
  * Wiki <https://www.unrealircd.org/docs/UnrealIRCd_4_documentation>. All
    documentation has been moved to a wiki <https://www.unrealircd.org/docs/>.
    It's even better than before and more accessible to people who are new to
    IRCd's. The wiki also allows easy translation
    <https://www.unrealircd.org/docs/Translating_UnrealIRCd_wiki_pages> by
    community members.
  * New directory structure
    <https://www.unrealircd.org/docs/UnrealIRCd_files_and_directories>. On
    *NIX the IRCd is now always installed to a different directory than where
    you compile from (~/unrealircd by default). No more mess. On both *NIX and
    Windows configuration files go in conf/, modules go in modules/, etc..
    Configuration files can be identical on Windows and *NIX. This new
    directory structure also allows easier packaging.
  * New I/O system using kqueue & epoll. The IRCd can now handle thousands of
    users more easily.
  * Improved SSL/TLS support. SSL has always been a major feature in
    UnrealIRCd but has been enhanced. UnrealIRCd is now always built with SSL
    support (both on *NIX and Windows). SSL client certificate fingerprints
    are visible in /WHOIS, a new certfp extban
    <https://www.unrealircd.org/docs/Extended_bans>
    (~S:certificatefingerprint), better defaults including 4096 bit keys and
    Perfect forward secrecy <https://en.wikipedia.org/wiki/Forward_secrecy>, etc.
  * DNS Blacklist support <https://www.unrealircd.org/docs/Blacklist_block>
    (DNSBL/RBL). Great for combating drones and other abusers.
  * Better and more helpful error messages. Especially regarding the
    configuration file.
  * More modern server-to-server protocol.
    <https://www.unrealircd.org/docs/Server_protocol:Changes> Such as using
    UID/SID's. Resulting in less desynch. issues.
  * Lowering the bar for Spamfilter
    <https://www.unrealircd.org/docs/Spamfilter#Block_simple_spam>. You can
    now choose between 'regex' and 'simple' matching. Simple matching allows
    using the usual '?' and '*' wildcards that everyone knows about. The regex
    engine has been moved from TRE to PCRE (=about twice as fast).
  * Configuration is more logical
    <https://www.unrealircd.org/docs/Upgrading_from_3.2.x>. Around 30% of the
    configuration blocks have been restructured. Don't worry, we include an
    UnrealIRCd 3.2.x to 4.x configuration file converter.
  * Easier 3rd party module management. On *NIX you now just put your 3rd
    party modules in /src/modules/third/ and then each time you run 'make'
    they will be compiled if needed.
  * Easier upgrading. On *NIX, when upgrading to a new version, ./Config will
    ask you to import settings from a previous installation, remembering your
    installation directory and other settings. It will also copy the 3rd party
    modules from the old to the new installation and re-compile them.
  * More secure. Even better secure defaults, more warnings about insecure
    behavior, ..
  * *IPv6 now also on Windows* <https://www.unrealircd.org/docs/Ipv6>.

For developers:

  * Easier source navigation. Because we moved almost everything to modules,
    it's now much easier to see all the code for a particular feature.
  * Cleaner code. There have been a lot of source code cleanups. Code has been
    restructured or rewritten. Old irrelevant code has been deleted.
  * Development documentation can be found on the wiki
    <https://www.unrealircd.org/docs/Main_Page>. We explain how to write a
    module in C and list all the details on the various Module API's such as
    how to write commands, channel modes, plug-in by using Hooks, etc...

*Upgrading from 3.2.x**to UnrealIRCd 4*
If you are upgrading from 3.2.x to 4.x then there are three important things 
to know:
*1) New file locations*
In UnrealIRCd 4 the location of the configuration files and other files have 
been changed. On *NIX the directory where you compile the IRCd from 
(previously 'Unreal3.2.X', now 'unrealircd-4.0.0') is no longer the same as 
the directory where the IRCd will be running from.
By default the IRCd is installed to //home/yourusername/unrealircd/ on *NIX. 
On Windows UnrealIRCd will install to /C:\Program Files (x86\UnrealIRCd 4/.

The new directory structure is as follows (both on Windows and *NIX):
conf/           contains all configuration files
logs/           for log files
modules/   all modules (.so files on *NIX, .dll files on Windows)

*2) Configuration file changes
*There have also been changes in various configuration blocks and settings. 
Don't worry, UnrealIRCd can convert your existing 3.2.x configuration files to 
UnrealIRCd 4 format. There's no need to start from scratch.
Please read https://www.unrealircd.org/docs/Upgrading_from_3.2.x for more 
information on the config file conversion.

*3) Third party modules*
If you are using 3rd party modules (modules not developed by the UnrealIRCd 
team) then they will need an update to run on UnrealIRCd 4.
Contact your developer for a new version or ask on our Modules forum 
<https://forums.unrealircd.org/viewforum.php?f=52> where someone may be kind 
enough to convert the module for you if you ask nicely.
Due to the many core changes in UnrealIRCd 4 it was simply impossible to make 
3.2.x modules work out-of-the-box on 4.x as well.

*Running a mixed 3.2.x / 4.x network*
You can run a mixed 3.2.x <-> 4.x network if you a follow a few simple rules 
<https://www.unrealircd.org/docs/Running_a_mixed_UnrealIRCd_3.2_and_UnrealIRCd_4_network>.

*End of the 3.2.x series*
With the release of UnrealIRCd 4.0.0 we have deprecating the previous series.
All support for the 3.2.x series will stop after December 31, 2016.
See https://www.unrealircd.org/docs/UnrealIRCd_3.2.x_deprecated

*Full summary of changes*
We did our best to "summarize" the 1100+ changesets in about 120 bullet points 
but it's still a long read.
The changes are split in the sections: NEW, CHANGED, REMOVED and MODULE CODERS 
/ DEVELOPERS.
==[ NEW ]==
* We moved a lot of functionality, including most channel modes, user
   modes and all extended bans into 138 separate modules.
   This makes it...
   A) possible to fully customize what exact functionality you want to load.
      You could even strip down UnrealIRCd to get something close to the
      basic RFC1459 features from the 1990s. (No idea why you would want
      that, but it's possible)
   B) easier for coders to see all source code related to a specific feature
   C) possible to fix bugs and just reload rather than restart the IRCd.

   Have a look at modules.default.conf which contains the "default" set of
   modules that you can load if you just want to load all functionality.
   If you want to customize the list of modules to load then simply make
   a copy of that file, give it a different name, and include that one
   instead. Since the file is fully documented, you can just comment out
   or delete the loadmodule lines of things you don't want to load.
* Oper permissions have changed completely: [A4+]
   * All previous oper levels/ranks no longer exist (Netadmin, Admin, ..)
   * oper::flags has been removed. Instead you must specify an operclass
     in oper::operclass (for example, 'operclass netadmin').
   * In operclass block(s) you define the privileges. You can now control
     exactly what an IRCOp can and cannot do.
     Have a look at operclass.default.conf which ships with UnrealIRCd,
     it contains a number of default operclass blocks suitable for the
     most common situations. See also the operclass block documentation:
https://www.unrealircd.org/docs/Operclass_block
   * If you ask UnrealIRCd to convert your 3.2.x configuration file then
     it will try to select a suitable operclass for the oper. This will
     not always 100% match your current oper block rights, though.
   * Channel Mode +A (Admin Only) has been removed. You can use the new
     extended ban ~O:<operclass>. This allows you to, for example, create
     an operclass 'netadmin' only channel: /MODE #chan +iI ~O:netadmin*
   * set::hosts has been removed, use oper::vhost instead.
   * Since oper levels have been removed you no longer see things like
     "OperX is a Network Administrator" in /WHOIS by default.
     If you want that, then you can set oper::swhois to
     "is a Network Administrator" (or any other text).
* Entirely rewritten I/O and event loop. This allows the IRCd to scale
   more easily to tens of thousands of clients by using kernel-evented I/O
   mechanisms such as epoll and kqueue.
* Memory pooling has been added to improve memory allocation efficiency
   and performance.
* On-connect DNSBL/RBL checking via the new blacklist block. [B1]
* The Windows version now has IPv6 support too. [B3]
* On all OS's we compile with IPv6 support enabled. You can still
   disable IPv6 at runtime by setting set::options::disable-ipv6. [B3]
* The local nickname length can be modified without recompiling the IRCd
* Channel Mode +d: This will hide joins/parts for users who don't say
   anything in a channel. Whenever a user speaks for the first time they
   will appear to join. Chanops will still see everyone joining normally
   as if there was no +d set.
* If you connect with SSL/TLS with a client certificate then your SSL
   Fingerprint (SHA256 hash) can be seen by yourself and others through
   /WHOIS. The fingerprint is also shared with all servers on the network.
* ExtBan ~S:<certificate fingerprint> for ban exceptions / invex. This
   can be used like +iI ~S:000000000etc.
* bcrypt has been added as a password hashing algorithm and is now the
   preferred algorithm [A3]
* './unreal mkpasswd' will now prompt you for the password to hash [A3]
* Protection against SSL renegotiation attacks [A3]
* When you link two servers the current timestamp is exchanged. If the
   time differs more than 60 seconds then servers won't link and it will
   show a message that you should fix your clock(s). This requires
   version alpha3 (or later) on both ends of the link [A3]
* Configuration file converter that will upgrade your 3.2.x conf to 4.x.
   On *NIX run './unreal upgrade-conf'. On Windows simply try to boot and
   after the config errors screen UnrealIRCd offers the conversion. [A3]
* The IRCd can now better handle unknown channel modes which expect a
   parameter. This can be useful in a scenario where you are slowly
   upgrading all your servers.
* If you want to unset a vhost but keep cloaked then use /MODE yournick -t
* A "crash reporter" was added. When UnrealIRCd is started it will check
   if a previous UnrealIRCd instance crashed and (after booting a new
   instance) it will spit out a report and ask if you want to submit it
   to the UnrealIRCd developers. Doing so will help us a lot as many bugs
   are often not reported. Note that UnrealIRCd will always ask before
   sending any information and never do so automatically. [B3]
* SSL: Support for ECDHE has been added to provide "forward secrecy". [B4]

==[ CHANGED ]==
* Numerics have been removed. Instead we now use SIDs (Server ID's) and
   UIDs (User ID's). SIDs work very similar to server numerics and UIDs
   help us to fix a number of lag-related race conditions / bugs.
* The module commands.so / commands.dll has been removed. All commands
   (those that are modular) are now in their own module.
* Self-signed certificates are now generated using 4096 bits, a SHA256
   hash and validity of 10 years. [A2]
* Building with SSL (OpenSSL) is now mandatory [A2]
* The link { } block has been restructured, see
https://www.unrealircd.org/docs/Upgrading_from_3.2.x#Link_block [A3]
* Better yet, check out our secure server linking tutorial:
https://www.unrealircd.org/docs/Tutorial:_Linking_servers
* If you have no set::throttle block you now get a default of 3:60 [A3]
* password entries in the conf no longer require specifying an auth-type
   like password "..." { md5; };. UnrealIRCd will now auto-detect. [A3]
* You will now see a warning when you link to a non-SSL server. [A3]
* Previously we used POSIX Regular expressions in spamfilters and at
   some other places. We have now moved to PCRE Regular expressions.
   They look very similar, but PCRE is a lot faster.
   For backwards-compatibility we still compile with both regex engines. [A3]
* Spamfilter command syntax has been changed, it now has an extra option
   to indicate the matching method:
   /SPAMFILTER [add|del|remove|+|-] [method] [type] ....
   Where 'method' can be one of:
   * -regex: this is the new fast PCRE2 regex engine
   * -simple: supports just strings and ? and * wildcards (super fast)
   * -posix: the old regex engine for compatibility with 3.2.x.  [A3]
* If you have both 3.2.x and 4.x servers on your network then the
   4.x server will only send spamfilters of type 'posix' to the 3.2.x
   servers because 3.2.x servers don't support the other two types.
   So in a mixed network you probably want to keep using 'posix' for
   a while until all your servers are running UnrealIRCd 4. [A3]
* set::oper-only-stats now defaults to "*"
* oper::from::userhost and vhost::from::userhost are now called
   oper::mask and vhost::mask. The usermask@ part is now optional and
   it supports two syntaxes. For one entry you can use: mask 1.2.3.*;
   For multiple entries the syntax is: mask { 192.168.*; 10.*; };
* Because having both allow::ip and allow::hostname in the same allow
   block was highly confusing (it was an OR-match) you must now choose
   between either allow::ip OR allow::hostname. [A3]
* cgiirc block is renamed to webirc and the syntax has changed [A4]
* set::pingpong-warning is removed, warning always off now [A4]
* More helpful configuration file parse error messages [A4]
* You can use '/OPER username' without password if you use SSL
   certificate (fingerprint) authentication. The same is true for
   '/VHOST username'. [A4]
* You must now always use 'make install' on *NIX [A4]
* Changed (default) directory structure entirely, see the section
   titled 'CONFIGURATION CHANGES' about 100 lines up. [A4]
* badword quit { } is removed, we use badword channel for it. [A4]
* badwords.*.conf is now just one badwords.conf
* To load all default modules you now include modules.default.conf.
   This file was called modules.conf in earlier alpha's.
   The file has been split up in sections and a lot of comments have
   been added to aid the user in deciding whether to load or not to
   load each module. [A4]
* Snomask +s is now (always) IRCOp-only. [A4]
* Previously there was little logic behind what modes halfops could
   set. Now the idea is as follows: halfops should be able to help out
   in case of a flood but not be able to change any 'policy decission
   modes' such as +G, +S, +c, +s. Due to this change halfops can now
   set modes +beiklmntIMKNCR (was: +beikmntI). [A4]
* If no link::hub or link::leaf is specified then assume hub "*". [B1]
* SWHOIS (Special whois title) has been extended in a number of ways:
   * We now "track" who or what set an swhois. This allows us to
     remove the swhois received via oper/vhost on de-oper/de-vhost.
   * You can now have multiple swhois lines
   * Multiple oper::swhois and vhost::swhois items are supported. [B1]
* When trying to link two servers without link::outgoing::options::ssl
   (which is not recommended) we try to use STARTTLS in order to
   'upgrade' the connection to use SSL/TLS anyway. This can be disabled
   via link::outgoing::options::insecure. [B2]
* SSLv3 has now been disabled for security. This also means you can only
   link UnrealIRCd 4 with 3.2.10.3 and later because earlier versions
   used SSLv3 instead of TLS due to an OpenSSL API mistake. [B4]

==[ REMOVED / DROPPED ]==
* Numeric server IDs, see above. [A1]
* PROTOCTL TOKEN and SJB64 are no longer implemented. [A1]
* Ziplinks have been removed. [A1]
* WebTV support. [A3]
* Channel Mode +j was removed and replaced by the configuration setting
   set::anti-flood::join-flood (default: 3 per 90 seconds). [B1]
* /CHATOPS: use /GLOBOPS instead which does the same
   /ADCHAT & /NACHAT: gone as we don't have such oper levels anymore
   Your opers should actually be in an #opers channel. If you also want
   special classes of oper channels like #admins then use +iI ~O:*admin*
* User modes:
   * +N (Network Administrator): see 'Oper permissions' under NEW as for why
   * +a (Services Administrator): same
   * +A (Server Administrator: same
   * +C (Co Administrator): same
   * +O (Local IRC Operator): same
   * +h (HelpOp): all this did was add a line "is available for help" in
     WHOIS. You can use a vhost block with vhost::swhois as a replacement
     or for opers just add an oper::swhois item.
   * +g (failops): we already have snomasks and the +o usermode for this
   * +v (receive infected DCC SEND rejection notices): moved to snomask +D

==[ MODULE CODERS / DEVELOPERS ]==
* A lot of technical documentation for module coders has been added
   at https://www.unrealircd.org/docs/ describing things like how to
   write a module from scratch, the User & Channel Mode System, Commands,
   Command Overrides, Hooks, attaching custom-data to users/channels,
   and more. [A2+]
* For commands: do not read from parv[0] anymore, doing so will lead
   to a crash. Use sptr->name instead. This change is necessary as
   the "name" in parv[0] could possibly point to a UID/SID rather than
   a nick name. Thus, if you would send parv[0] to a non-UID or non-SID
   capable server this would lead to serious issues (not found errors).
* Added MOD_OPT_PERM_RELOADABLE which permits reloading (eg: upgrades)
   but disallows unloading of a module [A3]
* There have been *a lot* of source code cleanups (ALL)
* We now use the information from PROTOCTL CHANMODES= for parameter
   skipping if the channel mode is unknown. Also, when channel modes
   are loaded or unloaded we re-broadcast PROTOCTL CHANMODES=. [B1]
* The server protocol docs have been removed. The protocol is now
   documented at https://www.unrealircd.org/docs/Server_protocol
   See also https://www.unrealircd.org/docs/Server_protocol:Changes
   for a list of changes between the 3.2 and 4.0 server protocol.
* GCC typechecking has been added to make sure your HookAdd... calls
   are adding hook functions with the correct parameter (types).

*Download*
As always, you can download UnrealIRCd from https://www.unrealircd.org/
All releases are signed with our PGP key (short key id 0x108FF4A9 and long id 
0xA7A21B0A108FF4A9)

Please report all bugs and feature suggestions at https://bugs.unrealircd.org/

-- 
Bram Matthys
Software developer/IT con...@vu...
Website:www.vulnscan.org
PGP key:www.vulnscan.org/pubkey.asc
PGP fp: EBCA 8977 FCA6 0AB0 6EDB  04A7 6E67 6D45 7FE1 99A6