Re: [Simple-evcorr-users] using variables learned in rule A in rule B's perlfunc: possible?

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

hi Michael,

thanks a lot for sharing examples from your rulebase! I am sure they will
be helpful for people who have to tackle similar tasks in the future, and
will be searching the mailing list for relevant examples.

kind regards,
risto

Risto-
>
>
>
> Thank you for taking time to respond so thoroughly.  Your examples were
> quite clear.  I would not have thought to try to use global variables.
> Ultimately I chose your cached/context approach, which worked great.
>
>
>
> I ended up taking a monolithic config file of nearly 400 hard to
> maintain/organize rules and spread them more logically across 29 (and
> counting) configuration files.  The impetus was needing to re-use some (but
> not all) of the rules in a second network that I admin.
>
>
>
> In an attempt to give back, I included below my general approach in case
> others find it useful.
>
>
>
> Thanks again,
>
> -Michael
>
>
>
> ==============/=============
>
>
>
> # 000-main.config
>
> # this file will be the same on each network
>
>
>
> type=Single
>
> ptype=SubStr
>
> pattern=SEC_STARTUP
>
> desc=Started SEC
>
> action=assign %a /destinationfile.log
>
>
>
> type=Single
>
> ptype=RegExp
>
> pattern=^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (.+?) .+
>
> varmap=SYSLOG; hostname=1
>
> desc=hostname
>
> action=none
>
> continue=TakeNext
>
>
>
> type=Jump
>
> ptype=cached
>
> pattern=SYSLOG
>
> context=$+{hostname} -> ( sub { return 1 if $_[0]; return 0 } )
>
> cfset=001-all
>
> continue=EndMatch
>
>
>
> # This is a catch-all rule to dump to the logfile anything that didn't
> match above..
>
> type=Single
>
> ptype=RegExp
>
> pattern=.*
>
> desc=$0
>
> action=write %a $0
>
>
>
> ---------/--------
>
>
>
> #001-all.config
>
> # this file will be different between networks due to differences in
> vendors, device naming standards, etc
>
>
>
> type=Options
>
> joincfset=001-all
>
> procallin=no
>
>
>
> type=Jump
>
> ptype=cached
>
> pattern=SYSLOG
>
> context=$+{hostname} -> ( sub { return 1 if $_[0] =~ m/^r-/ } )
>
> cfset=050-juniper
>
> continue=EndMatch
>
>
>
> type=Jump
>
> ptype=cached
>
> pattern=SYSLOG
>
> context=$+{hostname} -> ( sub { return 1 if $_[0] =~ m/^[ts]-/ } )
>
>
>
> # This is a catch-all rule to dump to the logfile anything that doesn't
> match above..
>
> type=Single
>
> ptype=RegExp
>
> pattern=.*
>
> desc=$0
>
> action=write %a $0
>
>
>
> ---------/--------
>
>
>
> # 050-juniper.config
>
> # these files will be the same between networks, and where most of the
> rules and re-use will come in.
>
>
>
> type=Options
>
> joincfset=050-juniper
>
> procallin=no
>
>
>
> # near top due to frequency..
>
> type=Jump
>
> ptype=RegExp
>
> pattern=.+
>
> cfset=110-juniper-mx104
>
> continue=TakeNext
>
>
>
> # two examples, I have many stanzas like this for individual JunOS daemons
> [hence many files]
>
> type=Jump
>
> ptype=RegExp
>
> pattern= mgd\[[0-9]+\]:
>
> cfset=150-juniper-mgd
>
> continue=EndMatch
>
>
>
> type=Jump
>
> ptype=RegExp
>
> pattern= rpd\[[0-9]+\]:
>
> cfset=150-juniper-rpd
>
> continue=EndMatch
>
>
>
> # things that don’t match specific daemons
>
> type=Jump
>
> ptype=RegExp
>
> pattern=.+
>
> cfset=100-juniper-nodaemon
>
> continue=TakeNext
>
>
>
> # This is a catch-all rule to dump to the logfile anything that survived
>
> type=Single
>
> ptype=RegExp
>
> pattern=.*
>
> desc=$0
>
> action=write %a $0
>
>
>
> [FIN]
>
>
>
> *From:* ris...@gm... <ris...@gm...>
> *Sent:* Saturday, October 17, 2020 5:13 PM
> *To:* Michael Hare <mic...@wi...>
> *Cc:* sim...@li...
> *Subject:* Re: [Simple-evcorr-users] using variables learned in rule A in
> rule B's perlfunc: possible?
>
>
>
> hi Michael,
>
>
>
> there are a couple of ways to address this problem. Firstly, instead of
> using sec match variables, one can set up Perl's native variables for
> sharing data between rules. For example, the regular expression pattern of
> the first rule can be easily converted into perlfunc pattern, so that the
> pattern would assign the hostname to Perl global variable $hostname. This
> global variable can then be accessed in perfunc patterns of other rules.
> Here is an example that illustrates the idea:
>
>
>
> type=Single
> ptype=perlfunc
> pattern=sub { if ($_[0] =~ /^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (.+?) /) \
>               { $hostname = $1; } else { $hostname = ""; } return 1; }
> desc=hostname
> action=none
> continue=TakeNext
>
> type=Jump
> ptype=perlfunc
> pattern=sub { return 1 if $hostname =~ m/^first-use-case/ }
> cfset=rules-for-this-match-1
>
> type=Jump
> ptype=perlfunc
> pattern=sub { return 1 if $hostname =~ m/^second-use-case/ }
> cfset=rules-for-this-match-2
>
>
>
> Since SEC supports caching the results of pattern matching, one could also
> store the matching result from the first rule into cache, and then retrieve
> the result from cache with the 'cached' pattern type. Since this pattern
> type assumes that the name of the cached entry is provided in the 'pattern'
> field, the hostname check with a perl function has to be implemented in a
> context expression (the expression is evaluated after the pattern match).
>
>
>
> Here is an example which creates an entry named SYSLOG in a pattern match
> cache, so that all match variables created in the first rule can be
> retrieved in later rules. Note that the entry is created in the 'varmap'
> field which also sets up named match variable $+{hostname}. In further
> rules, the 'cached' pattern type is used for retrieving the SYSLOG entry
> from cache, and creating all match variables from this entry. In order to
> check the hostname, the $+{hostname} variable that was originally set in
> the first rule is passed into perlfunc patterns in the second and third
> rule. Also, if you need to check more than just few match variables in
> perlfunc pattern, it is more efficient to pass a reference to the whole
> cache entry into the Perl function, so that individual cached match
> variables can be accessed through the reference (I have added an additional
> fourth rule into the example which illustrates this idea):
>
>
>
> type=Single
> ptype=RegExp
> pattern=^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (.+?) .+
> varmap=SYSLOG; hostname=1
> desc=hostname
> action=none
> continue=TakeNext
>
> type=Jump
> ptype=cached
> pattern=SYSLOG
> context=$+{hostname} -> ( sub { return 1 if $_[0] =~ m/^first-use-case/ } )
> cfset=rules-for-this-match-1
>
> type=Jump
> ptype=cached
> pattern=SYSLOG
> context=$+{hostname} -> ( sub { return 1 if $_[0] =~ m/^second-use-case/ }
> )
> cfset=rules-for-this-match-2
>
> type=Jump
> ptype=cached
> pattern=SYSLOG
> context=SYSLOG :> ( sub { return 1 if $_[0]->{"hostname"} =~
> m/^third-use-case/ } )
> cfset=rules-for-this-match-3
>
>
>
> A small side-note about the first rule -- you can also create the
> $+{hostname} variable in the regular expression itself by rewriting it as
> follows:
>
> ^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (?<hostname>.+?) .+
>
> And in that case, you can rewrite the 'varmap=SYSLOG; hostname=1'
> statement in the first rule simply as 'varmap=SYSLOG'.
>
>
>
> Hopefully these examples are useful and help you to tackle the problem.
>
>
>
> kind regards,
>
> risto
>
>
>
>
>
>
>
> Hi-
>
> I'm sorry to ask what is probably very basic question, but I have
> struggling with this for awhile (I have perused the manual a lot and the
> mailing list a bit) and could use some guidance.
>
> The short version is: is there a way to take the results of a pattern
> match in one rule and use that value in a perlfunc in another?
>
> More verbosely, at this time I use SEC for network syslog exclusion;
> nothing fancy.  I would like to start using Jump rules based on hostname.
> Hostname is derived from the incoming log line.
>
> I thought I would be clever and use a single rule to determine if there
> was a hostname or not, save it somewhere reusable, and then launch jump
> rules based on that.
>
> something like
>
> type=Single
> ptype=RegExp
> pattern=^\w+\s+[0-9]+ [0-9]+:[0-9]+:[0-9]+ (.+?) .+
> varmap= hostname=1
> desc=hostname
> action=assign %r $+{hostname}
> continue=TakeNext
>
> type=Jump
> ptype=perlfunc
> pattern=sub { return 1 if $+{hostname} =~ m/^first-use-case/ }
> cfset=rules-for-this-match-1
>
> type=Jump
> ptype=perlfunc
> pattern=sub { return 1 if $+{hostname} =~ m/^second-use-case/ }
> cfset=rules-for-this-match-2
>
> I know this doesn't work.  I understand that '%r' is not a perl hash, and
> is an action list variable, and that $+{hostname} is undef inside the
> type=Jump rule perlfunc.  I also know that %r is being set correctly, I see
> it in "variables -> r" if I do SIGUSR1 dump.
>
> So is it possible stash away a variable from one rule and use it in a Jump
> rule like above?  I can work around this easily by using a single rule like
> below, but if I have for example 20 jump permutations, it seems quite
> redundant to keep recalculating the hostname for comparison.
>
> type=Jump
> ptype=perlfunc
> pattern=sub { return 0 unless (defined($_[1]) && $_[0] =~ /^\w+\s+[0-9]+
> [0-9]+:[0-9]+:[0-9]+ (.+?) .+/); return 1 if $1 =~ m/^first-use-case/}
> cfset=all-rules
>
> Thanks in advance,
> -Michael
>
>
> _______________________________________________
> Simple-evcorr-users mailing list
> Sim...@li...
> https://lists.sourceforge.net/lists/listinfo/simple-evcorr-users
>
>