Re: [Nfsen-discuss] memory problem
Netflow visualisation and investigation tool
Brought to you by:
phaag
|
From: Milenko S. <mic...@gm...> - 2013-04-12 09:24:17
|
Hi Ryan, I hope I didn't forget something but these are steps that I did when installing botents: *1.* I copied backend file "Botnets.pm" to nfsen backend folder in my case "/usr/local/libexec/nfsen/plugins/" *2.* I created folder /usr/local/libexec/botnets/ where I stored all required files botnet.cron, botnet_filter, get_botnets_emerging-botcc *3.1* content of botnet.cron (I changed it a little bit) #!/bin/sh /usr/local/bin/wget -P /tmp/ "http://rules.emergingthreats.net/blockrules/emerging-botcc.rules" if [ -f /tmp/emerging-botcc.rules ]; then /usr/local/libexec/botnets/get_botnets_emerging-botcc /tmp/emerging-botcc.rules > /usr/local/libexec/botnets/filterrules /usr/local/libexec/botnets/botnet_filter < /usr/local/libexec/botnets/filterrules > /usr/local/libexec/botnets/alert-filter rm /tmp/emerging-botcc.rules else echo "$(date "+%b %d %H:%M:%S") botnets File emerging-botcc.rules was not downloaded. Check!!!" >> /var/log/nfsen.log fi *3.2* contet of botnet_filter #!/usr/bin/perl print "ip in ["; my $first = 1; foreach my $line (<STDIN>) { next if ($line =~ /^\s*\#.*$/ || $line =~ /^\s*$/); # skip empty and comment lines my ($ip, $a, $b) = split(/\|/, $line); print $ip." "; } print "]"; *3.3* content of get_botnets_emerging-botcc #!/usr/bin/perl sub get_version ($) { my ($filename) = @_; return `stat -t $filename | cut -f 13 -d" "`; } $filename = $ARGV[0]; my ($stamp) = get_version($filename); chomp($stamp); # c&c's are timed out one week after the lists age. This means that we don't accept c&c's from a list that is older than a week my $expire = $stamp + 86400*7; open(BOTNETS, "<$filename"); foreach my $line (<BOTNETS>) { next if ($line =~ /^\s*\#.*$/ || $line =~ /^\s*$/); # skip empty and comment lines my ($ip_list) = $line=~m/alert tcp \$HOME_NET any \-\> \[([^\]]+)\]/; foreach my $ip (split(/,/,$ip_list)) { print join('|',$ip, "", "", "emergingthreads", $stamp, $expire, "")."\n"; } } In this file a replaced "alert any" to "alert tcp". I got solution for this from this discussion: http://sourceforge.net/mailarchive/forum.php?thread_name=5081AA44.9080006%40unsane.co.uk&forum_name=nfsen-discuss *4.* Crontab 0 * * * * /usr/local/libexec/botnets/botnet.cron Running botnet.cron script will create files filterrules and alert-filter. *5.* my nfsen.conf for Botnets @plugins = ( [ "live", 'Events' ], [ "!", 'Botnets' ], ); %PluginConf = ( periodic_queries => [ { # Botnet events are timed out 8 hour after the last update "Type"=>"[eq]botnet", "UpdateTime"=>['[le]#$unix_time-28800#','#$unix_time#'], "StopTime"=>["[null]",'#$unix_time#'], }, { # Botnet events are upgraded if the bot contacted the controller more than 20 times "Type"=>"[eq]botnet", "Times"=>"[ge]20", "UpdateTime"=>'#$unix_time#', "Level"=>["[eq]notify","alarm"], }, ], events => { db_connection_string => "DBI:mysql:database=events;host=localhost;port=3306", db_user => "user", db_passwd => "pass", }, delete_queries => [ { # Delete events after 14 days (14 days*24 hour*60 minute*60 seconds=1209600 seconds) "StopTime"=>['[le]#$unix_time-1209600#'], } ], botnets => { import_cmd => "/bin/cat /usr/local/libexec/botnets/filterrules", }, ); *6. *I installed events plugin as described in README file. I hope that it will help you with your installation. Regards On Thursday 11 of April 2013 16:39:52 Ryan West wrote: Hey Mitja, I’m looking to get Botnets with Events configured for my nfsen installation. I was wondering if you can share of the steps you used? Thanks, -ryan *From:* Mitja Podlogar [mailto:mic...@gm...] *Sent:* Thursday, April 11, 2013 9:30 AM *To:* nfs...@li... *Subject:* Re: [Nfsen-discuss] memory problem Update: I have tested a little and found out that RAM/SWAP consumption is done by plugin "nfsight". Even though I configured nfsight to use 512M of RAM it goes beyond that. Problem is with networks that I added, two /16 and I guess this is to much for server to handle it. I now added only one /16 as an internal network and it works but most of the traffic is generated form other /16. I am using nfsight-beta-2013-03-23. I don't have much experience with this so I don't really know what is normal behavior for this plugin. Maybe someone who have used it for a while can tell theirs statistics. On Wed, Apr 10, 2013 at 1:00 PM, Mitja Podlogar <mic...@gm...[1]> wrote: I installed Nfsen 1.3.6p1 and nfdump 1.6.9 on FreeBSD 9.1. I'm using these plugins: SURFmap, PortTracker, Botnets (with Events) and nfsight. Everything was working great for more then a month. Today I noticed strange behavior of process "nfsend-comm" which started using 2 CPU and entire RAM + entire SWAP. At that time nothing really special was done except watching graphs. I tried restarting nfsen service and it was working for a minute and then again nfsend-comm starts using RAM and CPU. Yesterday I added plugin PortTracker and upgraded nfdump from 1.6.8 to 1.6.9. I don't really know if it has anything to do with this problem since everything was working fine for 24 hours. I tried and disabled PortTracker but it didn't help. If logs are needed I can provide them but I didn't see anything interesting in messages and nfsen log. I would be very grateful if anyone could tell me how to proceed with debugging this or if anyone already had this problem and knows a solution. Thank you in advance. -------- [1] mailto:mic...@gm... |