Deployment

Return to Home Page

The vision for Syslogd2 deployment is as a network-infrastructure component, effectively no different from the building wiring and component interconnections of the network itself (with "specialized" administration requirements like any firewall, vpn or laod-balancer). Syslogd2 was built and released as open-source instead of as "pay-ware" specifically to complement and emulate the successes and value-points of the Linux kernel and Linux operating system by providing high-quality (downloadable) software that keeps costs down on minimal-cost Intel hardware. Once installed and configured, Syslogd2 can (mostly) be maintained by network staff with minimal system-administration requirements.

Syslogd2 is (essentially) a multi-homed Linux syslog daemon designed specifically for the expanded roles of network-management (and host-management) data-collection, traffic-filtering, traffic reduction and the safe-guarding / retransmission of data during times of network events and outages. Major secondary design criteria involve providing configuration options that balance the security roles of Linux administrator with the ability of network staff (those with the most investment in network-management) to control the day-to-day operation of the software components that influence their environment. Network-staff access and configuration capabilities are provided (primarily) through combinations of Linux user-groups and Linux file-system-based file-permissions that complement and extend basic features designed into Syslogd2:

-- As a multi-homed Linux application, Syslogd2 supports the flexibility and security of Linux and Unix in network deployments. Syslogd2 considers each port on each IP address of a multi-homed system to be a completely separate and independent input source, capable of independent operation from other addresses or interfaces.

-- A single Linux hardware platform with multiple Network Interface Cards (NICs) can aggregate multiple NICs into a single logical interface (connected as a network-trunk) for the increased bandwidth required by high-speed data input (or output).
-- Alternatively (or additionally), any aggregated-interface (network trunk) can be assigned multiple vlans (via the Linux vconfig command) each with an assigned address applicable to that vlan. Use of this feature can significantly reduce the number of physical hardware platforms (and switch ports) that need to be deployed in support of low-traffic syslog-generating devices and hosts.
-- Multiple physical NICs can alternatively be assigned to different network segments (or to multiple switches in DMZ scenarios) to collect data from multiple network locations on a single platform.
-- For security purposes, every Linux input socket allows owner/group/mode specifications and can be (arbitrarily) specified as to file-system location, allowing them to be placed into "protected" directories that also have read/write/access permissions that are based on standard Linux file-system security rules. This reduces the chances of rogue applications logging arbitrary data or "spoofing" syslog events of sensitive applications. It also guards against abuse of the Syslogd2 "command-tool" (should that tool be deployed).
-- Traditional guidance is to configure devices to produce minimal syslog traffic in order to minimize the impact on network links and syslog receivers. This guidance is no longer needed with Syslogd2:

-- Syslogd2's ability to support multiple input threadpools (and multiple worker-threadpools) extends to dedicating a complete reader threadpool AND a processing threadpool to each socket. This mans that Syslogd2 can keep up with much heavier traffic levels than are traditionally supplied by network devices.
-- Multiple input threadpools applied to multiple, independent NICs allow for "sharing" of a single physical host platform across multiple network segments without loss of performance. This keeps overall costs down and performance up.
-- Dedicated syslog-collector systems (vs cetralized, virtualized hosts) provide the CPU cycles needed to process tremendous increases in syslog input traffic levels while simultaneously decreasing overall network levels of syslog traffic.
-- Syslogd2 filters provide control of overall wan/lan traffic levels by limiting the traffic that "gets past" the distributed edge-collectors.
-- Enhanced troubleshooting and root-cause analysis is provided when detailed syslog traffic is logged to local disks with only selected summary (error) traffic sent over the network.
-- Note: No syslog collector can help network devices with under-powered CPUs that are adversely impacted by generating higher syslog traffic levels.

-- The traditional prohibition on limiting the number of output definitions in the syslog configuration file is lifted by Syslogd2's implementation of output threadpools (especially when Syslogd2 is running on dedicated hardware that has the memory and CPU cycles to use). Output threadpools allow for as many output files, sockets or connections as the host-administrator desires and as the disk space can support.

-- Every Syslogd2 output file can be individually configured with owner and/or group ownership as well as file-mode permissions to allow read-only (recommended) access to output log files for searching and forensic (root-cause) analysis by network (or security) personnel without compromising the root access to the entire host.
-- These owner/group specifications can be specified numerically or by name and (if non-existant or invalid users/groups are given) will default to whatever ownership and group are provided in run-time configuration Shipped defaults are uid:0, gid:0 and mode:600.
-- Furthermore, by locating files in "special-access" directories, files intended for (requested by) different groups (such as network vs security (or database vs web-server application groups) can have different file permissions in common output directories.

As a Linux-based platform, Syslogd2 can be secured against intrusion:

-- Linux configurations should always disable ip forwarding to disable routing of traffic between interfaces (and network vlans). There are two steps to this:

(1) issue a dynamic command to disable forwarding until the next reboot; echo 0 > /proc/sys/net/ipv4/ip_forward
Alternatively (or if the /proc filesystem is not mounted): sysctl -w net.ipv4.ip_forward=0
(2) Add the following to the /etc/sysctl.conf file to make the change permanent after subsequent system reboots:
# disable IP forwarding
net.ipv4.ip_forward = 0

-- Dedicated Syslogd2 configurations should use the systemctl and other commands to disable all possible services that are not directly needed by Syslogd2. The list of services that Syslogd2 requires for support is limited to named (DNS), ntpd (network-time protocol), and sshd (for access to the host). Almost all other services that have been installed can be disabled. After all, any service that is not running cannot be hacked. -- Disable the desktop GUI and all the services that go with it Syslogd2 is a background daemon and has no interaction with the desktop GUI. The command-line tool for Linux is based on ncurses and also has no need for the desktop GUI. As the root user, issue init 3 to disable the destkop GUI. To make this change permanent, edit the /etc/inittab file and edit the initdefault line so it looks like: id:3:initdefault:. This will cause run-level 3 (generally "command-line multi-user with network support") to become the default run-level. If the initdefault value is already set to 2 or 3, you can leave it there as the desired value is already set. Some distributions distinguish between run-levels 2 and 3 -- some do not.

It is hoped that Syslogd2 can remove (or vastly reduce) the stigmas, and misconceptions surrounding Linux and surrounding syslog-based network-management systems by overcoming previous software shortcomings and by removing vendor lock-in based on the "resource investments" involved in network-management equipment deployment.

If Syslogd2 is used to "instrument" networks in a vendor-neutral manner, syslog-based products from multiple vendors can be directly compared or exchanged (even upgraded) as desired. Multiple vendor syslog-analysis products can be used concurrently for different purposes (or for comparison) and can even deployed in "layers" where sites with their own staff can have their own syslog-based reporting systems. Sites without technical staff can forward syslog traffic to larger sites where they can be managed from wherever the managing staff resides.

Some of the misconceptions that Syslogd2 can be used to correct are the end result of decades of Microsoft-thinking -- thinking that is limited to the (limited) capabilities of MS Windows and the (mostly) single-homed applications that run there.

To achieve the vision of Syslogd2 and to maximize the impact and effectiveness of Syslogd2 deployments, it may be necessary to procure exceptions to common corporate policies:

-- No network host may be assigned a network port on more than a single vlan at the same time. This is usually a reaction to the Microsoft network-stack limitations and the fact that they will happily route traffic between the network segments because they lack the ability to disable IP forwarding. Sometimes this is a result of Linux administrators not understanding the Linux operating system and not knowing that Linux can disable IP forwarding.
-- All Linux systems must be virtualized on centralized servers to avoid "wasting" hardware resources. This bit of "conventional wisdom" developed over time from the MS-Windows reputation of being unreliable and unstable. It has a corollary (also from MS Windows) that states that Linux systems must be rebooted once a week to clear memory and prevent memory leaks. It does not allow for the fact that most high-quality Linux applications are open-source and (not consumed by release schedules or limited developer resources) are extensively tested for memory leaks and instabilities prior to release to production. -- The idea that Linux and MS Windows are equivalent extends to bans on the use of Linux based on lack of virus-checkers, ignoring the fact that Linux viruses (and most other Linux malware) is so rare and easily defended against as to be virtually non-existent.
-- There is no point in deploying a Linux platform without an "officially-sanctioned" application. It is too expensive (or too inefficient) to multiple remote Linux systems. Implementation of such policies (given that Syslogd2 is described as a "syslog logger" or as a "component of the OS") defeats ALL the primary advantages of Syslogd2's advanced features which center around decentralized implementations. Putting all of a corporation's syslog collectors in a central location forces all the "original" syslog UDP traffic to go through firewalls, to flood network links and to congest host ports of the central servers with sheer traffic volume. It prevents Syslogd2's ability to filter messages to separate the storage of forensic data and relevant network notifications and is in many ways counterproductive because of the impact that transmission of such data has on network performance as that data crosses limited-bandwidth links between switches or sites. -- Deploying decentralized systems is too expensive. This is likely due to the historically-increasing cost of Windows licenses. Many Linux distributions can be downloaded at no cost (even the Red hat clone distribution CentOS). Syslogd2 is freely downloadable as open-source GPL-licensed code. The hardware used for servers can often be hardware that would otherwise be discarded / sold as "outdated" after upgrading Windows servers. -- The Linux GUI is central to Linux distributions (Linux cannot be deployed without a desktop GUI) so each remote machine needs to have its own monitor and keyboard. This is similar to "an Intel box cannot be run without a monitor and keyboard". As long as a monitor and keyboard are available to physically attach during installation / emergencies, this assumption is left over from MS-WIndows where a keyboard/monitor are necessary to clear the (all-too-frequent) system-modal error-windows in MS-Windows. In Linux, most remote admins utilize the command-line, reserving the CPU cycles that would otherwise be spend on screen-savers for application usage.

Other misconceptions are a direct result of Microsoft and big-name salesmanship that is designed to continue and reinforce misconceptions about Linux and Unix-based systems and to play on the fears of IT and Security managers to retain market-share and sales-figures as long as possible. Many of these fears and misconceptions have (over time) been codified and ossified into corporate policies:

Corporate policies that specify that all Linux hosts be virtualized and consolidated onto centralized VM machines will counteract most of the advantages of deploying Syslogd2 for network management:

Syslogd2 's filtering capability was designed to reduce the level of syslog traffic being transmitted over network links. The idea was that the less "unnecessary" syslog traffic is transmitted over network links, the more capacity is made available to user traffic and user applications. Centralizing all syslog receivers defeats this design criteria.

--Not only does centralizing the receivers cause excessive traffic levels over network links, but it also results in excessive data being received at centralized host ports (where it is lost due to the effects of congestion of the aggregated data-flows).

--Adding insult to injury, the process of virtualizing the Linux hosts that Syslogd2 is running on inherently limits the CPU cycles and memory available to the process, thereby defeating the purpose of using the Syslog high-performance design and features due to lack of virtual CPUs and memory cycles being allocated by the VM -- not to mention the limited bandwidth available to each individual shared VM that is running Syslogd2.

Syslogd2 was designed to allow TCP connections through firewalls and routers with back-up spooling for those "must-have" messages that absolutely, positively HAVE to get to the reporting / analysis servers. Centralizing all syslog receivers means that all traffic over network links and through firewalls and routers is via the default UDP protocol which defeats the purpose and benefits of implementing TCP links at all. The TCP features of Syslogd2 were explicitly provided because UDP is a "CPU-killer" for firewalls and most routers. Most firewalls and routers have to check their rule-sets / route-tables (using CPU cycles) for each UDP packet passing through them versus one check when establishing the initial TCP connection handshake.

There is no guarantee that all UDP data-packets transmitted by distributed network devices (and hosts) even reaches the centralized, virtualized syslog server host due to network-device congestion and "dirty link" corruption of the UDP traffic. Or that the traffic is not discarded by the VM host server after being read from thee network buffers (if it is not lost because the VM host is too slow) and before being delivered to the virtualized server running Syslogd2.

Return to top

Return to Home Page