must: A More Useful Syslog Test tool Wiki
A syslog message generator that replicates real syslog messages
Brought to you by:
jackalsdance
Menu
▾
▴
guide to creating templates
Template Design
Overall file structure
The template file is in XML format and the whole config is enclosed within a single pair of tags:
<log_generator <a="" class="alink notfound" href="options">options>
…
</log_generator>
Within these, there are three types of tag possible:
• Parameters: Enclosed in <parameter <a="" class="alink notfound" href="options">options /> one line tags, these configure general settings</parameter>
• Sequences: Enclosed in <sequence [options=""> … <</sequence> these tags contan STEP tags that define the logs used in a sequence
• Logs: Enclosed in <log <a="" class="alink notfound" href="options">options> … </log> these tags define the actual log messages generated
Thus, a basic file should look something like:
<log generator="" <a="" class="alink notfound" href="options">options>
<parameter <a="" class="alink notfound" href="options">options />
<parameter <a="" class="alink notfound" href="options">options />
<parameter <a="" class="alink notfound" href="options">options />
<sequence <a="" class="alink notfound" href="options">options>
<step <a="" class="alink notfound" href="options">options>
…
</step>
<step {options]="">
…
</step>
</sequence>
<sequence <a="" class="alink notfound" href="options">options>
<step <a="" class="alink notfound" href="options">options>
…
</step>
<step {options]="">
…
</step>
</sequence></parameter></parameter></parameter></log>
<log <a="" class="alink notfound" href="options">options>
…
</log>
<log <a="" class="alink notfound" href="options">options>
…
</log>
<log <a="" class="alink notfound" href="options">options>
…
</log>
As the logical way to write these templates is logs, then sequences, then parameters, the following sections will approach these sections in this order.
Log Messages
LOG Tag
Log messages are wrapped in <log <a="" class="alink notfound" href="options">options>…</log> tags, where the options are the following key=”value” pairs: which are all required
• Id=”integer” – these numbers themselves mean nothing but are references used when defining sequences. You can just number logs 1,2,3 or in any other way you prefer
• src=”ip” – the IP address of the device sending the log. This is the IP that the UDP packet will be sourced (spoofed) from, and also if you select RFC3164 compliance it is the IP that will be inserted in the message
• facility=”string” – log facility coded into the RFC3164 priority header. These can be:
kernel kernel messages
user user-level messages
mail mail system
system system daemons
security security/authorization messages
internal messages generated internally by syslogd
print line printer subsystem
news network news subsystem
uucp UUCP subsystem
clock clock daemon
security2 security/authorization messages
ftp FTP daemon
ntp NTP subsystem
logaudit log audit
logalert log alert
clock2 clock daemon
local0 local use 0
local1 local use 1
local2 local use 2
local3 local use 3
local4 local use 4
local5 local use 5
local6 local use 6
local7 local use 7
If you are not sure which to use, use one of local1 – local7
• severity=”string” – the log severity coded into the RFC3164 priority header. These can be:
emerg Emergency: system is unusable
alert Alert: action must be taken immediately
crit Critical: critical conditions
err Error: error conditions
warn Warning: warning conditions
notice Notice: normal but significant condition
info Informational: informational messages
debug Debug: debug-level messages
An example of a LOG tag is:
<log id="user-content-1" src="192.168.1.1" facility="local7" severity="info"></log>
FIELD Tag
Within the LOG tags is the definition of the log message. A log message template is basically just a string of text, where fields are defined within the text within {{field}} tage, and then subsequent <field <a="" class="alink notfound" href="options">options> … </field> blocks.
A log message is defined between <message> … </message> tags, an example being:
<message>Apache HTTP/GET 1.1 /{{PAGE}} requested by {{SRCIP}} 404 error</message>
In this example, there are two fields called PAGE and SRCIP. These are defined in FIELD tags, and the several field types are possible. The FIELD tags are defined as key=”value” pairs with the following options. All options are mandatory.
• name =”string”– name of the field
• type=”string” – the type of the field, as defined below
• value=”string” – the pattern of the generated value, as defined below
Fundamental to the value field are two types of definition:
• Number ranges can be defined as x-y where a number will be randomly selected between x and y for each log message
• Comma separated values, where one of the values will be selected for each log message. Note that blank entries can be included if they are required, e.g. 1,2,3,,5,6 will randomly select 1 or 2 or 3 or 5 or 6 or a blank value, due to the double comma in the middle
The following field types are possible, with the following associated values:
• ip – this is an IP Address which can defined in more than one way:
o you can use a ranges where octets can be fixed, or defined in an x-y format
eg 10.1.1.4-60 or 10.1.12-25.4-60 or1-254.1.2.1-254
o you can use a selection where comma separated IPs will be randomly selected from
eg 192.168.1.1,192.168.1.2,192.168.1.3
o You cannot mix these, use one format or the other
• int – a range of integer defined in the x-y format eg 200-400 for a range. Whole numbers only will be outputted.
o In order to get non integer numbers use two fields e.g. {{WHOLE}}.{{DEC}} where WHOLE is "1-1000" and DEC is "0-20000"
o To get leading zeros add a selection e.g. 0.{{LEADING}}{{DEC}} where LEADING is ",0,00,000,0000" and DEC is "0-20000"
• text –this allows a range of comma separated words or phrases. Most characters are allowed except " and some esoteric characters that may be misinterpreted
eg blue,green,red orange,grey,white@yellow
• time – time and date values based on the current server time (be aware of time zones, this will generate using the local time not UTC so check your server time zone). Possible values within this are:
o timehr24 – the current hour in 24hr format (1-24)
o timehr12 – the current hour in 12hr format (1-12)
o timemin – the current minute
o timesec – the current second.
• Due to randomisation using times does not guarantee they will arrive in 'time' order, just close (within a second). Think of it like ntp skew :)
o dateday – the numerical representation of the day, always 2 digits
o datemon - the numerical representation of the month, always 2 digits
o dateshortmon – shortened version of the month (eg Jan, Feb, Mar)
o datelongmon – full version of the month (eg January, February)
o dateyear – 2 digit representation of the year
o datelongyear – 4 digit representation of the year (20xx)
o epoch – time from unix epoch (seconds)
A sample complete LOG entry is:
<log id="user-content-5" src="192.168.1.1" facility="local7" severity="info">
<message>Apache HTTP/GET 1.1 /{{PAGE}} requested by {{SRCIP}}</message>
<field name="SRCIP" type="ip" value="31.100.1-10.1-5">
<field name="PAGE" type="text" value="index.htm,page1.htm,page2.htm">
</field></field></log>
RFC3164 Compliance
Even with the overall parameter ‘rfc3164=”true’ set, which appends the priority field (in the format <xxx> followed the date in format Day MonthName MonthNum, this does not fulfill the RFC criteria. Additionally, you must define the application that is associated to the log. As such, in order to be compliant the first word in the message must be followed by a colon then a space.
For example, this may be apache_access:
After this word, any message may be used. This initial word can be generated in the normal way using a set of comma-separated values, and you would define it as {{APP}}:</xxx>
Sequences
SEQUENCE Tag
A sequence of logs is defined by a <sequence <a="" class="alink notfound" href="options">options> … </sequence> tag set, with the following required key=”value” options:
• name=”string” – the name of the sequence – only to help understand the template when editing, no operational purpose
• weight=”integer” – the actual number provided is not strictly constrained, however across all sequences the weights are summed and the logs new sequences are started pseudo-randomly, with the chance of this sequence being generated being [sequence weight] / [sum of all sequence weights]
A sample SEQUENCE tag is:
<sequence name="Example Day to Day Web Page Access" weight="39"></sequence>
STEP Tag
Within SEQUENCE tags, individual steps are defined. When a sequence is started, the first step is run immediately (thus every sequence must have one or more steps) and then the next step is queued to be generated sometime soon.
Each step defines the log ID that should be generated, and optionally includes sets of <inherit <a="" class="alink notfound" href="options">options /> tags defining which data from the previous step should persist into this step. This means that, for example, if the previous step and this setup both have a field called SRCIP, the value of SRCIP in the last step and in this step will be the same. Otherwise, a new value will be generated for each field. </inherit>
Step tags have the following required option defined as a key=”value” pair:
• logid=”integer” – where integer is the id of the LOG tag containing the message to be generated,
Inherit tags have the following required option defined as a key=”value” pair:
• value=”string” – where the string is the name of the field to inherit.
A sample complete SEQUENCE tag is:
<sequence name="Example Day to Day Web Page Access" weight="39">
<step logid="4">
<step logid="6">
<inherit value="SRCIP">
<inherit value="DSTIP">
</inherit></inherit></step>
<step logid="3">
<inherit value="SRCIP">
</inherit></step>
</step></sequence>
Parameters and template settings
LOG_GENERATOR options
The template starts with a LOG_GENERATOR tag, which has the following key=”value” pairs that are entirely optional, and only help to manage templates:
• template_name=”string” – a name for the template
• template_version=”string” – a version name/number
• template_author=”string” – the author of the template
A sample LOG_GENERATOR tag is
<log_generator template_name="new_template1.gnr: Test template 2" template_ver="1.0" template_author="Dave"></log_generator>
PARAMETER Tags
Within LOG_GENERATOR tags, as well as the SEQUENCE and LOG tags, general parameters can be defined. Parameters are entered as <parameter <a="" class="alink notfound" href="option">option /> where one option can be set per parameter tag, in a key=”value’ format.</parameter>
• maxrate=”integer” – an integer between 1 and 30000 – this caps the rate of this message per second if and only if the frequency and machine speed generate more than the maxrate per second. Set to 0 to set no limit
• total=”integer” – the script will stop once this many messages have been sent (assuming this is reached before the ‘runtime’ parameter is reached). Set to 0 to set no limit.
• runtime=”integer” – the script will stop after this time (in seconds) assuming the ‘total’ parameter has not been reached first. Set to 0 to set no limit.
• rfc3164=”Boolean” – set with true or false (false is default if not included) to instruct the script to append the priority and date fields to all messages as defined by RFC3164. (See section in message generation for additional requirements to achieve compliance)
