From: Ben J. <be...@in...> - 2012-12-04 22:38:34
|
Hello, Please forgive me if the solution to my problem is obvious, but I've done quite a bit of searching-around and nothing has resonated. I've been using fail2ban-0.8.6 on Ubuntu 10.04-1 LTS for at least a year without issue (as far as I know). But recently, it seems that some very persistent users/bots are not being banned when they should be. In particular, I see entries in my Linux Logwatch digests like this: --------------------- SSHD Begin ------------------------ Failed logins from: 85.91.136.121 (85-91-136-121.varna.homelan.bg): 860 times 109.163.239.115: 17 times 173.208.232.143: 64 times 180.166.11.211: 1448 times 199.101.51.153 (host1.dbxmedia.com): 1638 times 216.114.69.35: 602 times Illegal users from: 82.221.99.229: 8 times 85.91.136.121 (85-91-136-121.varna.homelan.bg): 1 time 173.208.232.143: 90 times 180.166.11.211: 37 times 216.114.69.35: 2 times [...] Received disconnect: 11: disconnected by user : 1 Time(s) **Unmatched Entries** PAM 3 more authentication failures; logname= uid=0 euid=0 tty=ssh ruser= rhost=109.163.239.115 user=root : 11 time(s) PAM service(sshd) ignoring max retries; 4 > 3 : 11 time(s) ---------------------- SSHD End ------------------------- 860 times, 1448 times, 1638 times, 602 times... why aren't these bots being banned after 3 times? I executed the following in an effort to make that determination: ---------------------------------------------------------- # fail2ban-regex /var/log/auth.log.0 /etc/fail2ban/filter.d/sshd.conf Running tests ============= Use regex file : /etc/fail2ban/filter.d/sshd.conf Use log file : /var/log/auth.log.0 Results ======= Failregex |- Regular expressions: | [1] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*(?:error: PAM: )?Authentication failure for .* from <HOST>\s*$ | [2] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*(?:error: PAM: )?User not known to the underlying authentication module for .* from <HOST>\s*$ | [3] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*Failed (?:password|publickey) for .* from <HOST>(?: port \d*)?(?: ssh\d*)?$ | [4] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*ROOT LOGIN REFUSED.* FROM <HOST>\s*$ | [5] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*[iI](?:llegal|nvalid) user .* from <HOST>\s*$ | [6] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*User .+ from <HOST> not allowed because not listed in AllowUsers$ | [7] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*authentication failure; logname=\S* uid=\S* euid=\S* tty=\S* ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$ | [8] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*refused connect from \S+ \(<HOST>\)\s*$ | [9] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*Address <HOST> .* POSSIBLE BREAK-IN ATTEMPT!*\s*$ | [10] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*User .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*$ | `- Number of matches: [1] 0 match(es) [2] 0 match(es) [3] 4762 match(es) [4] 0 match(es) [5] 134 match(es) [6] 0 match(es) [7] 0 match(es) [8] 0 match(es) [9] 0 match(es) [10] 0 match(es) Ignoreregex |- Regular expressions: | `- Number of matches: Summary ======= Addresses found: [1] [2] [3] [... thousands of matches printed here ...] [6] [7] [8] [9] [10] Date template hits: 148256 hit(s): MONTH Day Hour:Minute:Second 0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second Year 0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second 0 hit(s): Year/Month/Day Hour:Minute:Second 0 hit(s): Day/Month/Year Hour:Minute:Second 0 hit(s): Day/Month/Year Hour:Minute:Second 0 hit(s): Day/MONTH/Year:Hour:Minute:Second 0 hit(s): Month/Day/Year:Hour:Minute:Second 0 hit(s): Year-Month-Day Hour:Minute:Second 0 hit(s): Year.Month.Day Hour:Minute:Second 0 hit(s): Day-MONTH-Year Hour:Minute:Second[.Millisecond] 0 hit(s): Day-Month-Year Hour:Minute:Second 0 hit(s): TAI64N 0 hit(s): Epoch 0 hit(s): ISO 8601 0 hit(s): Hour:Minute:Second 0 hit(s): <Month/Day/Year@Hour:Minute:Second> Success, the total number of match is 4896 However, look at the above section 'Running tests' which could contain important information. ---------------------------------------------------------- The file /var/log/auth.log.0 contains log entries from Dec 2 03:33:48 to Dec 2 06:28:15. If I inspect fail2ban's log entries for the same period of time, I find only the following: ---------------------------------------------------------- 2012-12-02 00:26:23,443 fail2ban.server : INFO Changed logging target to /var/log/fail2ban.log for Fail2ban v0.8.6 2012-12-02 00:26:24,713 fail2ban.filter : INFO Log rotation detected for /var/log/apache2/error.log 2012-12-02 00:26:24,723 fail2ban.filter : INFO Log rotation detected for /var/log/apache2/error.log 2012-12-02 00:30:01,906 fail2ban.filter : INFO Log rotation detected for /var/log/apache2/other_vhosts_access.log 2012-12-02 01:02:37,839 fail2ban.filter : INFO Log rotation detected for /var/log/auth.log 2012-12-02 01:02:37,976 fail2ban.filter : INFO Log rotation detected for /var/log/syslog ---------------------------------------------------------- The SSHd jail configuration is: ---------------------------------------------------------- [ssh] enabled = true port = ssh filter = sshd logpath = /var/log/auth.log maxretry = 6 ---------------------------------------------------------- Might anyone know why fail2ban registered absolutely nothing in its logs during this ongoing login-per-second attempt, for hours on end, given the output from fail2ban-regex, above? Thanks for any pointers, -Ben |
From: Yaroslav H. <li...@on...> - 2012-12-06 02:48:53
|
what backend does it use (check reported in the log)? gamin might still be broken on ubuntu may be? enforce pulling or upgrade to 0.8.7.1 (soon I will push out 0.8.8) otherwise -- raise loglevel to debug and see if also you get all the lines for changes in the file being detected On Tue, 04 Dec 2012, Ben Johnson wrote: > Hello, > Please forgive me if the solution to my problem is obvious, but I've > done quite a bit of searching-around and nothing has resonated. > I've been using fail2ban-0.8.6 on Ubuntu 10.04-1 LTS for at least a year > without issue (as far as I know). > But recently, it seems that some very persistent users/bots are not > being banned when they should be. > In particular, I see entries in my Linux Logwatch digests like this: > --------------------- SSHD Begin ------------------------ > Failed logins from: > 85.91.136.121 (85-91-136-121.varna.homelan.bg): 860 times > 109.163.239.115: 17 times > 173.208.232.143: 64 times > 180.166.11.211: 1448 times > 199.101.51.153 (host1.dbxmedia.com): 1638 times > 216.114.69.35: 602 times > Illegal users from: > 82.221.99.229: 8 times > 85.91.136.121 (85-91-136-121.varna.homelan.bg): 1 time > 173.208.232.143: 90 times > 180.166.11.211: 37 times > 216.114.69.35: 2 times > [...] > Received disconnect: > 11: disconnected by user : 1 Time(s) > **Unmatched Entries** > PAM 3 more authentication failures; logname= uid=0 euid=0 tty=ssh > ruser= rhost=109.163.239.115 user=root : 11 time(s) > PAM service(sshd) ignoring max retries; 4 > 3 : 11 time(s) > ---------------------- SSHD End ------------------------- > 860 times, 1448 times, 1638 times, 602 times... why aren't these bots > being banned after 3 times? > I executed the following in an effort to make that determination: > ---------------------------------------------------------- > # fail2ban-regex /var/log/auth.log.0 /etc/fail2ban/filter.d/sshd.conf > Running tests > ============= > Use regex file : /etc/fail2ban/filter.d/sshd.conf > Use log file : /var/log/auth.log.0 > Results > ======= > Failregex > |- Regular expressions: > | [1] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*(?:error: > PAM: )?Authentication failure for .* from <HOST>\s*$ > | [2] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*(?:error: > PAM: )?User not known to the underlying authentication module for .* > from <HOST>\s*$ > | [3] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*Failed > (?:password|publickey) for .* from <HOST>(?: port \d*)?(?: ssh\d*)?$ > | [4] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*ROOT > LOGIN REFUSED.* FROM <HOST>\s*$ > | [5] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*[iI](?:llegal|nvalid) > user .* from <HOST>\s*$ > | [6] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*User > .+ from <HOST> not allowed because not listed in AllowUsers$ > | [7] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*authentication > failure; logname=\S* uid=\S* euid=\S* tty=\S* ruser=\S* > rhost=<HOST>(?:\s+user=.*)?\s*$ > | [8] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*refused > connect from \S+ \(<HOST>\)\s*$ > | [9] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*Address > <HOST> .* POSSIBLE BREAK-IN ATTEMPT!*\s*$ > | [10] ^\s*(?:\S+ )?(?:kernel: \[\d+\.\d+\] )?(?:@vserver_\S+ > )?(?:(?:\[\d+\])?:\s+[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?|[\[\(]?sshd(?:\(\S+\))?[\]\)]?:?(?:\[\d+\])?:)?\s*User > .+ from <HOST> not allowed because none of user's groups are listed in > AllowGroups\s*$ > `- Number of matches: > [1] 0 match(es) > [2] 0 match(es) > [3] 4762 match(es) > [4] 0 match(es) > [5] 134 match(es) > [6] 0 match(es) > [7] 0 match(es) > [8] 0 match(es) > [9] 0 match(es) > [10] 0 match(es) > Ignoreregex > |- Regular expressions: > `- Number of matches: > Summary > ======= > Addresses found: > [1] > [2] > [3] > [... thousands of matches printed here ...] > [6] > [7] > [8] > [9] > [10] > Date template hits: > 148256 hit(s): MONTH Day Hour:Minute:Second > 0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second Year > 0 hit(s): WEEKDAY MONTH Day Hour:Minute:Second > 0 hit(s): Year/Month/Day Hour:Minute:Second > 0 hit(s): Day/Month/Year Hour:Minute:Second > 0 hit(s): Day/Month/Year Hour:Minute:Second > 0 hit(s): Day/MONTH/Year:Hour:Minute:Second > 0 hit(s): Month/Day/Year:Hour:Minute:Second > 0 hit(s): Year-Month-Day Hour:Minute:Second > 0 hit(s): Year.Month.Day Hour:Minute:Second > 0 hit(s): Day-MONTH-Year Hour:Minute:Second[.Millisecond] > 0 hit(s): Day-Month-Year Hour:Minute:Second > 0 hit(s): TAI64N > 0 hit(s): Epoch > 0 hit(s): ISO 8601 > 0 hit(s): Hour:Minute:Second > 0 hit(s): <Month/Day/Year@Hour:Minute:Second> > Success, the total number of match is 4896 > However, look at the above section 'Running tests' which could contain > important > information. > ---------------------------------------------------------- > The file /var/log/auth.log.0 contains log entries from Dec 2 03:33:48 to > Dec 2 06:28:15. If I inspect fail2ban's log entries for the same period > of time, I find only the following: > ---------------------------------------------------------- > 2012-12-02 00:26:23,443 fail2ban.server : INFO Changed logging target > to /var/log/fail2ban.log for Fail2ban v0.8.6 > 2012-12-02 00:26:24,713 fail2ban.filter : INFO Log rotation detected > for /var/log/apache2/error.log > 2012-12-02 00:26:24,723 fail2ban.filter : INFO Log rotation detected > for /var/log/apache2/error.log > 2012-12-02 00:30:01,906 fail2ban.filter : INFO Log rotation detected > for /var/log/apache2/other_vhosts_access.log > 2012-12-02 01:02:37,839 fail2ban.filter : INFO Log rotation detected > for /var/log/auth.log > 2012-12-02 01:02:37,976 fail2ban.filter : INFO Log rotation detected > for /var/log/syslog > ---------------------------------------------------------- > The SSHd jail configuration is: > ---------------------------------------------------------- > [ssh] > enabled = true > port = ssh > filter = sshd > logpath = /var/log/auth.log > maxretry = 6 > ---------------------------------------------------------- > Might anyone know why fail2ban registered absolutely nothing in its logs > during this ongoing login-per-second attempt, for hours on end, given > the output from fail2ban-regex, above? > Thanks for any pointers, > -Ben > ------------------------------------------------------------------------------ > LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial > Remotely access PCs and mobile devices and provide instant support > Improve your efficiency, and focus on delivering more value-add services > Discover what IT Professionals Know. Rescue delivers > http://p.sf.net/sfu/logmein_12329d2d > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Ben J. <be...@in...> - 2013-02-18 19:53:10
|
On 2/18/2013 1:08 PM, Fabian Wenk wrote: > Hello Ben > > On 18.02.2013 17:57, Ben Johnson wrote: >> The file fail2ban.log.1 was created at exactly the same time that the >> service was stopped. This seems to imply that the log rotation process >> requests that the fail2ban service be stopped (or restarted). (Does >> calling "fail2ban-client set logtarget" restart fail2ban implicitly?) > > No it should not restart the fail2ban server process. See below. > >> Here's the script (presumably, this version is included in the 0.8.8 >> package that you made): >> >> -------------------------------------------------------------------- >> /var/log/fail2ban.log { >> >> weekly >> rotate 4 >> compress >> >> delaycompress >> missingok >> postrotate >> fail2ban-client set logtarget /var/log/fail2ban.log >/dev/null >> endscript > > A I am using fail2ban on FreeBSD and Gentoo systems and I had to > create my own lograte config file. I use the following postrotate > command line: > > postrotate > /path/to/fail2ban-client set logtarget > /var/log/fail2ban.log 1>/dev/null || true > endscript > > Adjust /path/to according where your fail2ban-client is installed. > > This does rotate the logfile and keeps fail2ban-server running > and working: > > ~ $ ps auxwww|grep fail2ban > root 14143 0.0 0.2 39848 10400 ?? S 2Feb13 8:59.41 > /usr/local/bin/python2.7 /usr/local/bin/fail2ban-server -b -s > /var/run/fail2ban/fail2ban.sock > ~ $ > > ~ $ ls -1tr /var/log/fail2ban.log* > /var/log/fail2ban.log-20130106.gz > /var/log/fail2ban.log-20130113.gz > /var/log/fail2ban.log-20130120.gz > /var/log/fail2ban.log-20130127.gz > /var/log/fail2ban.log-20130203.gz > /var/log/fail2ban.log-20130210.gz > /var/log/fail2ban.log-20130217 > /var/log/fail2ban.log > ~ $ > > Try with the added ' || true' at the end of your postrotate command. > > > bye > Fabian Hi, Fabian, Thanks for the tips. I have added the full path to fail2ban-client (even though it doesn't seem to necessary, given that fail2ban-client is in my PATH environment variable), as well as " || true" to the post-rotate command, per your example. Unfortunately, I am still unable to start fail2ban in the normal way (on Debian): # service fail2ban start My understanding is that this is equivalent to: # sh /etc/init.d/fail2ban start Executing this command requires about 10 seconds and no output is produced at all before the cursor drops back to the prompt. I tried some of the other daemon options: # sh /etc/init.d/fail2ban status * Status of authentication failure monitor * fail2ban is not running # sh /etc/init.d/fail2ban restart * Restarting authentication failure monitor fail2ban [fail] This is exactly what happened when I ran into this problem a few months ago. Once the log rotation "borks" fail2ban, I'm unable to start it by normal means. It seems that something is "stuck" in a non-sensible state. Last time this occurred, I reinstalled fail2ban and the service started, but I would really prefer not to have to go to that length to fix this. What temp files, if any, does fail2ban use? Any ideas? Thanks again, -Ben |
From: Ben J. <be...@in...> - 2013-02-19 18:37:54
|
On 2/19/2013 10:20 AM, Fabian Wenk wrote: > Hello Ben > > On 19.02.2013 16:02, Ben Johnson wrote: >> On 2/19/2013 9:13 AM, Fabian Wenk wrote: > >>> Gentoo is also doing the transition from /var/run to the tmpfs >>> /run and so /var/run is just a symlink pointing to /run. But as >>> /run is a tmpfs, it is empty again after a system reboot. >>> >>>> ah -- then I have no clue where /var/run/fail2ban is gone since since >>>> 0.8.2 init script creates it: >>> >>> @Ben, could you check with 'ls -l /var/run' (without a / after >>> run) if this already is a symlink to /run. if yes, check e.g. >>> with 'df -hl | grep /run' what kind of filesystem /run is. >> >> I checked this out and it's not a symlink; it's a real directory. > > Could you check if eventually /var/run is also a tmpfs? Did it > fail after you have rebooted your system? I don't see any indication that /var/run is a tmpfs. Here's the output from "df": # df Filesystem 1K-blocks Used Available Use% Mounted on /dev/vzfs 80000000 47984964 32015036 60% / /dev/simfs 80000000 47984964 32015036 60% /tmp /dev/simfs 80000000 47984964 32015036 60% /var/tmp The contents of /etc/fstab are: proc /proc proc defaults 0 0 none /dev/pts devpts rw 0 0 /dev/vzfs / reiserfs rw,usrquota,grpquota 0 0 >>>> so I would look for someone who destroyed it for you! ;) > >> A better question might be, why does fail2ban not recreate this >> directory if it's missing, instead of "failing" outright. This may be a >> clear-cut case of ban2fail! :) >> >> In all seriousness, though, is there some reason for which this >> directory is not recreated if necessary? How is it created in the first >> place? During installation? > > During the installation the directory is created from the .deb > package, e.g. aptitude or apt-get does create it according to the > post script in the .deb. > > On system boot, or start of fail2ban with '/etc/init.d/fail2ban > start' (which this newish 'service fail2ban start' also uses) > this script should check it. Eventually adding something like > this at top in the start section could help: > > if [ ! -d /var/run/fail2ban ] ; then > mkdir /var/run/fail2ban > fi Adding these lines to the init script seems like a good idea. I'll add them to my own copy right now, but it would be nice to have assurance that this change will be committed to the source code. The theory that a reboot is what broke fail2ban has merit. I dug back through my Logwatch reports and discovered that fail2ban stopped banning on Feb. 11. Take a look at the mod-dates for the directories in /var/run: # ls -lah /var/run total 76K drwxr-xr-x 12 root root 4.0K Feb 19 09:04 . drwxr-xr-x 17 root root 4.0K Dec 20 10:14 .. drwxr-xr-x 2 amavis amavis 4.0K Feb 11 21:24 amavis drwxr-xr-x 2 root root 4.0K Feb 11 21:24 apache2 -rw-r--r-- 1 root root 5 Feb 17 00:26 apache2.pid drwxr-xr-x 2 clamav root 4.0K Feb 17 00:26 clamav -rw-r--r-- 1 root root 6 Feb 11 21:22 crond.pid ---------- 1 root root 0 Feb 11 21:22 crond.reboot drwxr-xr-x 3 root root 4.0K Feb 11 21:22 dovecot drwxr-xr-x 2 root root 4.0K Feb 19 06:48 fail2ban -rw-r--r-- 1 root root 161 Feb 19 09:04 motd drwxr-xr-x 2 mysql root 4.0K Feb 11 21:22 mysqld drwxr-xr-x 2 root root 4.0K Feb 11 21:22 network -rw-r--r-- 1 root root 4 Feb 11 21:22 ntpd.pid drwx------ 2 root root 4.0K Feb 19 09:24 pure-ftpd drwxrwxr-x 2 root utmp 4.0K Feb 11 21:21 screen drwxr-xr-x 2 root root 4.0K Feb 11 21:21 sshd -rw-r--r-- 1 root root 6 Feb 11 21:22 sshd.pid -rw-r--r-- 1 root root 6 Feb 11 21:22 syslogd.pid -rw-rw-r-- 1 root utmp 3.0K Feb 18 17:29 utmp All evidence points to the fact that a reboot was performed Feb 11, shortly after 21:00, which sounds about right (I was doing some maintenance that required a reboot that night). The question remains: how was this directory deleted in the first place, if no tmpfs is at play? As I stated earlier, fail2ban *did* log its shutdown sequence when the system notified it was going down for reboot; is there anything in the source code that would remove the /var/run/fail2ban directory under certain shutdown conditions? Perhaps this point is moot, as long as your modification is added to the init script: if [ ! -d /var/run/fail2ban ] ; then mkdir /var/run/fail2ban fi Thanks for the viable workaround, Fabian! -Ben > > bye > Fabian > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_d2d_feb > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users > |
From: Yaroslav H. <li...@on...> - 2013-02-19 19:08:58
|
On Tue, 19 Feb 2013, Ben Johnson wrote: > > if [ ! -d /var/run/fail2ban ] ; then > > mkdir /var/run/fail2ban > > fi > Adding these lines to the init script seems like a good idea. they are there (have been since a while actually: # Assure that /var/run/fail2ban exists [ -d /var/run/fail2ban ] || mkdir -p /var/run/fail2ban * 32281ed - BF: Assure that /var/run/fail2ban exists upon start (LP: #222804, #223706) (4 years, 10 months ago) [Yaroslav Halchenko] that was at times of 0.8.2 but the problem with them that I better move them BEFORE the point of checking the status ;) now if directory doesn't exist, status fails (probably) and then init script doesn't get to the point of creating them... I would need to double check on that > The question remains: how was this directory deleted in the first place, > if no tmpfs is at play? As I stated earlier, fail2ban *did* log its > shutdown sequence when the system notified it was going down for reboot; > is there anything in the source code that would remove the > /var/run/fail2ban directory under certain shutdown conditions? I would first verify that it indeed happens (reboot again) and then possibly grep for 'var/run' among init scripts -- may be some of them perform some cleanup? ;)... eg. init.d/mountnfs-bootclean.sh: # Clean /tmp, /var/lock, /var/run although on my system that /lib/init/bootclean.sh has clean /run "! -xtype d ! -name utmp ! -name innd.pid" || ES=1 so I guess it should not remove subdirectories... but who knows -- may be yours is different ? ;) -- Yaroslav O. Halchenko http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Yaroslav H. <li...@on...> - 2013-02-18 23:52:43
|
I am a bit lost ... in part because no fail2ban.log is ever shared in this thread so we do not have any information... for a moment I thought that it might fail to start because .socket file remains present which precludes starting but on my debian system I am getting the proper announcement on that event: novo# service fail2ban start [ ok ] Starting authentication failure monitor: fail2ban. novo# ls /var/run/fail2ban fail2ban.pid fail2ban.sock novo# pkill -9 fail2ban-server novo# ls /var/run/fail2ban fail2ban.pid fail2ban.sock novo# service fail2ban start [FAIL] Starting authentication failure monitor: fail2ban[....] Socket file /var/run/fail2ban/fail2ban.sock is present ... failed! . ok what is your output if you start it via (which points me now to some shortcomming of the default jail.conf on debian ;) committed a fix so will be in the next debian package) novo# fail2ban-client start WARNING 'findtime' not defined in 'ssh'. Using default value WARNING 'findtime' not defined in 'dropbear'. Using default value WARNING 'findtime' not defined in 'pam-generic'. Using default value WARNING 'findtime' not defined in 'xinetd-fail'. Using default value WARNING 'findtime' not defined in 'ssh-ddos'. Using default value WARNING 'findtime' not defined in 'apache'. Using default value WARNING 'findtime' not defined in 'apache-multiport'. Using default value WARNING 'findtime' not defined in 'apache-noscript'. Using default value WARNING 'findtime' not defined in 'apache-overflows'. Using default value WARNING 'findtime' not defined in 'php-url-fopen'. Using default value WARNING 'findtime' not defined in 'lighttpd-fastcgi'. Using default value WARNING 'findtime' not defined in 'lighttpd-auth'. Using default value WARNING 'findtime' not defined in 'vsftpd'. Using default value WARNING 'findtime' not defined in 'proftpd'. Using default value WARNING 'findtime' not defined in 'pure-ftpd'. Using default value WARNING 'findtime' not defined in 'wuftpd'. Using default value WARNING 'findtime' not defined in 'postfix'. Using default value WARNING 'findtime' not defined in 'couriersmtp'. Using default value WARNING 'findtime' not defined in 'courierauth'. Using default value WARNING 'findtime' not defined in 'sasl'. Using default value WARNING 'findtime' not defined in 'dovecot'. Using default value WARNING 'findtime' not defined in 'named-refused-tcp'. Using default value WARNING 'findtime' not defined in 'asterisk-tcp'. Using default value WARNING 'findtime' not defined in 'asterisk-udp'. Using default value 2013-02-18 18:46:49,655 fail2ban.server : INFO Starting Fail2ban v0.8.8 2013-02-18 18:46:49,655 fail2ban.server : INFO Starting in daemon mode ERROR Could not start server. Maybe an old socket file is still present. Try to remove /var/run/fail2ban/fail2ban.sock. If you used fail2ban-client to start the server, adding the -x option will do it novo# so what is your output in this case? if anything is logged into fail2ban.log -- share On Mon, 18 Feb 2013, Ben Johnson wrote: > On 2/18/2013 1:08 PM, Fabian Wenk wrote: > > Hello Ben > > On 18.02.2013 17:57, Ben Johnson wrote: > >> The file fail2ban.log.1 was created at exactly the same time that the > >> service was stopped. This seems to imply that the log rotation process > >> requests that the fail2ban service be stopped (or restarted). (Does > >> calling "fail2ban-client set logtarget" restart fail2ban implicitly?) > > No it should not restart the fail2ban server process. See below. > >> Here's the script (presumably, this version is included in the 0.8.8 > >> package that you made): > >> -------------------------------------------------------------------- > >> /var/log/fail2ban.log { > >> weekly > >> rotate 4 > >> compress > >> delaycompress > >> missingok > >> postrotate > >> fail2ban-client set logtarget /var/log/fail2ban.log >/dev/null > >> endscript > > A I am using fail2ban on FreeBSD and Gentoo systems and I had to > > create my own lograte config file. I use the following postrotate > > command line: > > postrotate > > /path/to/fail2ban-client set logtarget > > /var/log/fail2ban.log 1>/dev/null || true > > endscript > > Adjust /path/to according where your fail2ban-client is installed. > > This does rotate the logfile and keeps fail2ban-server running > > and working: > > ~ $ ps auxwww|grep fail2ban > > root 14143 0.0 0.2 39848 10400 ?? S 2Feb13 8:59.41 > > /usr/local/bin/python2.7 /usr/local/bin/fail2ban-server -b -s > > /var/run/fail2ban/fail2ban.sock > > ~ $ > > ~ $ ls -1tr /var/log/fail2ban.log* > > /var/log/fail2ban.log-20130106.gz > > /var/log/fail2ban.log-20130113.gz > > /var/log/fail2ban.log-20130120.gz > > /var/log/fail2ban.log-20130127.gz > > /var/log/fail2ban.log-20130203.gz > > /var/log/fail2ban.log-20130210.gz > > /var/log/fail2ban.log-20130217 > > /var/log/fail2ban.log > > ~ $ > > Try with the added ' || true' at the end of your postrotate command. > > bye > > Fabian > Hi, Fabian, > Thanks for the tips. I have added the full path to fail2ban-client (even > though it doesn't seem to necessary, given that fail2ban-client is in my > PATH environment variable), as well as " || true" to the post-rotate > command, per your example. > Unfortunately, I am still unable to start fail2ban in the normal way (on > Debian): > # service fail2ban start > My understanding is that this is equivalent to: > # sh /etc/init.d/fail2ban start > Executing this command requires about 10 seconds and no output is > produced at all before the cursor drops back to the prompt. > I tried some of the other daemon options: > # sh /etc/init.d/fail2ban status > * Status of authentication failure monitor > * fail2ban is not running > # sh /etc/init.d/fail2ban restart > * Restarting authentication failure monitor fail2ban [fail] > This is exactly what happened when I ran into this problem a few months > ago. Once the log rotation "borks" fail2ban, I'm unable to start it by > normal means. It seems that something is "stuck" in a non-sensible state. > Last time this occurred, I reinstalled fail2ban and the service started, > but I would really prefer not to have to go to that length to fix this. > What temp files, if any, does fail2ban use? Any ideas? > Thanks again, > -Ben > ------------------------------------------------------------------------------ > The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, > is your hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials, tech docs, > whitepapers, evaluation guides, and opinion stories. Check out the most > recent posts - join the conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users -- Yaroslav O. Halchenko http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Ben J. <be...@in...> - 2012-12-06 16:21:20
|
On 12/5/2012 9:48 PM, Yaroslav Halchenko wrote: > what backend does it use (check reported in the log)? gamin might still > be broken on ubuntu may be? enforce pulling or upgrade to 0.8.7.1 (soon > I will push out 0.8.8) > > otherwise -- raise loglevel to debug and see if also you get all the > lines for changes in the file being detected > Thanks, Yaroslav. I just upgraded to 0.8.8, but I'm having some problems getting fail2ban started now. I had to install manually because I'm stuck on Ubuntu 10.04 for the time being, and the package cited in the release notes seems to require a newer version of Python than what I can install using the package manager: The following packages have unmet dependencies: fail2ban: Depends: python (>= 2.6.6-2ubuntu2~) but 2.6.5-0ubuntu1 is to be installed E: Broken packages In any event, I followed the installation instructions at http://www.fail2ban.org/wiki/index.php/MANUAL_0_8 . But when I test the configuration, errors result: # fail2ban-client -d WARNING 'logpath' not defined in 'apache-overflows'. Using default value WARNING 'filter' not defined in 'apache-overflows'. Using default value WARNING 'action' not defined in 'apache-overflows'. Using default value ERROR /etc/fail2ban/filter.d/.conf and /etc/fail2ban/filter.d/.local do not exist ERROR Unable to read the filter ERROR Errors in jail 'apache-overflows'. Skipping... ['set', 'loglevel', 3] ['set', 'logtarget', '/var/log/fail2ban.log'] Any thoughts as to why this is happening? I haven't moved or altered any of the jail configuration files. As a related corollary, I can't seem to control fail2ban as a service, even though I copied the Debian script from the Wiki article to /etc/init.d/fail2ban. # service fail2ban start does not produce any output (nor does stop). Have I mucked things up here? Thanks for any help, -Ben |
From: Yaroslav H. <li...@on...> - 2012-12-06 17:51:52
|
On Thu, 06 Dec 2012, Ben Johnson wrote: > I had to install manually because I'm stuck on Ubuntu 10.04 for the time and when I saw that build for 10.04 has failed I thought "who would use this old one???" -- so here you come ;-/ the reason for absent package is that I switched at some point in the back to use dh_python2 helper instead of deprecated pycentral... hold on there -- I will prep a special build for this elderly beastie in an hour or so > # service fail2ban start > does not produce any output (nor does stop). I would prefer not even to bother figuring this out but rather give you a nice package -- wait a bit -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Yaroslav H. <li...@on...> - 2012-12-06 18:11:05
|
ok -- now should be available for 10.04 from neuro.debian.net repository. here is the path to the .deb -- please try (via apt-get install fail2ban=0.8.8-1+lucid0~nd10.04+1 after you add that repository), since I did not ;) or just download / install http://neuro.debian.net/debian/pool/main/f/fail2ban/fail2ban_0.8.8-1+lucid0~nd10.04+1_all.deb cheers, On Thu, 06 Dec 2012, Yaroslav Halchenko wrote: > On Thu, 06 Dec 2012, Ben Johnson wrote: > > I had to install manually because I'm stuck on Ubuntu 10.04 for the time > and when I saw that build for 10.04 has failed I thought "who would use > this old one???" -- so here you come ;-/ the reason for absent package > is that I switched at some point in the back to use dh_python2 helper > instead of deprecated pycentral... hold on there -- I will prep a > special build for this elderly beastie in an hour or so > > # service fail2ban start > > does not produce any output (nor does stop). > I would prefer not even to bother figuring this out but rather give you > a nice package -- wait a bit -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Ben J. <be...@in...> - 2013-02-19 01:43:45
|
On 2/18/2013 6:52 PM, Yaroslav Halchenko wrote: > I am a bit lost ... in part because no fail2ban.log is ever shared in > this thread so we do not have any information... I didn't share anything from the log, aside from the shutdown sequence a few messages back, because, beyond that, fail2ban hasn't logged anything since it went belly-up after the log-rotation. I thought that perhaps the log-target was botched, so I ran: # fail2ban-client set logtarget /var/log/fail2ban.log >/dev/null This command produced no output, so I assumed that it had the intended effect. > for a moment I thought that it might fail to start because .socket file > remains present which precludes starting but on my debian system I am > getting the proper announcement on that event: > > novo# service fail2ban start > [ ok ] Starting authentication failure monitor: fail2ban. > novo# ls /var/run/fail2ban > fail2ban.pid fail2ban.sock > novo# pkill -9 fail2ban-server > novo# ls /var/run/fail2ban > fail2ban.pid fail2ban.sock > novo# service fail2ban start > [FAIL] Starting authentication failure monitor: fail2ban[....] Socket file /var/run/fail2ban/fail2ban.sock is present ... failed! > . ok Hmm. I'm on Debian (well, Ubuntu), too, and I don't even have the directory /var/run/fail2ban, no less the file /var/run/fail2ban/fail2ban.sock. On another similarly configured server, that directory and that file exist. Well, at least we know that not to be the problem! > what is your output if you start it via (which points me now to some > shortcomming of the default jail.conf on debian ;) committed a fix so will be > in the next debian package) > > novo# fail2ban-client start > WARNING 'findtime' not defined in 'ssh'. Using default value > WARNING 'findtime' not defined in 'dropbear'. Using default value > WARNING 'findtime' not defined in 'pam-generic'. Using default value > WARNING 'findtime' not defined in 'xinetd-fail'. Using default value > WARNING 'findtime' not defined in 'ssh-ddos'. Using default value > WARNING 'findtime' not defined in 'apache'. Using default value > WARNING 'findtime' not defined in 'apache-multiport'. Using default value > WARNING 'findtime' not defined in 'apache-noscript'. Using default value > WARNING 'findtime' not defined in 'apache-overflows'. Using default value > WARNING 'findtime' not defined in 'php-url-fopen'. Using default value > WARNING 'findtime' not defined in 'lighttpd-fastcgi'. Using default value > WARNING 'findtime' not defined in 'lighttpd-auth'. Using default value > WARNING 'findtime' not defined in 'vsftpd'. Using default value > WARNING 'findtime' not defined in 'proftpd'. Using default value > WARNING 'findtime' not defined in 'pure-ftpd'. Using default value > WARNING 'findtime' not defined in 'wuftpd'. Using default value > WARNING 'findtime' not defined in 'postfix'. Using default value > WARNING 'findtime' not defined in 'couriersmtp'. Using default value > WARNING 'findtime' not defined in 'courierauth'. Using default value > WARNING 'findtime' not defined in 'sasl'. Using default value > WARNING 'findtime' not defined in 'dovecot'. Using default value > WARNING 'findtime' not defined in 'named-refused-tcp'. Using default value > WARNING 'findtime' not defined in 'asterisk-tcp'. Using default value > WARNING 'findtime' not defined in 'asterisk-udp'. Using default value > 2013-02-18 18:46:49,655 fail2ban.server : INFO Starting Fail2ban v0.8.8 > 2013-02-18 18:46:49,655 fail2ban.server : INFO Starting in daemon mode > ERROR Could not start server. Maybe an old socket file is still present. Try to remove /var/run/fail2ban/fail2ban.sock. If you used fail2ban-client to start the server, adding the -x option will do it > novo# > > so what is your output in this case? > if anything is logged into fail2ban.log -- share It did occur to me to try this; here's the output: # fail2ban-client start WARNING 'findtime' not defined in 'pure-ftpd'. Using default value WARNING 'findtime' not defined in 'xinetd-fail'. Using default value WARNING 'findtime' not defined in 'asterisk-udp'. Using default value WARNING 'findtime' not defined in 'apache-multiport'. Using default value WARNING 'findtime' not defined in 'vsftpd'. Using default value WARNING 'findtime' not defined in 'wuftpd'. Using default value WARNING 'findtime' not defined in 'courierauth'. Using default value WARNING 'findtime' not defined in 'apache'. Using default value WARNING 'findtime' not defined in 'pam-generic'. Using default value WARNING 'findtime' not defined in 'lighttpd-fastcgi'. Using default value WARNING 'findtime' not defined in 'dropbear'. Using default value WARNING 'findtime' not defined in 'couriersmtp'. Using default value WARNING 'findtime' not defined in 'dovecot'. Using default value WARNING 'findtime' not defined in 'proftpd'. Using default value WARNING 'findtime' not defined in 'named-refused-tcp'. Using default value WARNING 'findtime' not defined in 'php-url-fopen'. Using default value WARNING 'findtime' not defined in 'apache-noscript'. Using default value WARNING 'findtime' not defined in 'asterisk-tcp'. Using default value WARNING 'findtime' not defined in 'ssh'. Using default value WARNING 'findtime' not defined in 'apache-overflows'. Using default value WARNING 'findtime' not defined in 'lighttpd-auth'. Using default value WARNING 'findtime' not defined in 'postfix'. Using default value WARNING 'findtime' not defined in 'ssh-ddos'. Using default value WARNING 'findtime' not defined in 'sasl'. Using default value WARNING 'findtime' not defined in 'apache-badbots'. Using default value WARNING 'findtime' not defined in 'pure-ftpd-mysql'. Using default value 2013-02-18 17:35:48,532 fail2ban.server : INFO Starting Fail2ban v0.8.8 2013-02-18 17:35:48,532 fail2ban.server : INFO Starting in daemon mode ERROR Could not start server. Maybe an old socket file is still present. Try to remove /var/run/fail2ban/fail2ban.sock. If you used fail2ban-client to start the server, adding the -x option will do it I checked again for the /var/run/fail2ban/ directory, and it still doesn't exist. I tried the suggestion from the error message: # fail2ban-client -x start But, of course, this produces the same output, because the socket file is not the problem. Could it be that the /var/run/fail2ban/ directory and socket file cannot be created for some reason? I'm executing these commands as "root", so, I'm not sure why that would be. Any other thoughts? Also, Fabian stated that the log-rotate command should not cause the server to restart. I believe him (hehe), but if this is true, then how does one explain the shutdown sequence that was logged just before the log file was renamed to "fail2ban.log.1" and the new "fail2ban.log" was created? Thanks for all your help, guys! -Ben > On Mon, 18 Feb 2013, Ben Johnson wrote: > > > >> On 2/18/2013 1:08 PM, Fabian Wenk wrote: >>> Hello Ben > >>> On 18.02.2013 17:57, Ben Johnson wrote: >>>> The file fail2ban.log.1 was created at exactly the same time that the >>>> service was stopped. This seems to imply that the log rotation process >>>> requests that the fail2ban service be stopped (or restarted). (Does >>>> calling "fail2ban-client set logtarget" restart fail2ban implicitly?) > >>> No it should not restart the fail2ban server process. See below. > >>>> Here's the script (presumably, this version is included in the 0.8.8 >>>> package that you made): > >>>> -------------------------------------------------------------------- >>>> /var/log/fail2ban.log { > >>>> weekly >>>> rotate 4 >>>> compress > >>>> delaycompress >>>> missingok >>>> postrotate >>>> fail2ban-client set logtarget /var/log/fail2ban.log >/dev/null >>>> endscript > >>> A I am using fail2ban on FreeBSD and Gentoo systems and I had to >>> create my own lograte config file. I use the following postrotate >>> command line: > >>> postrotate >>> /path/to/fail2ban-client set logtarget >>> /var/log/fail2ban.log 1>/dev/null || true >>> endscript > >>> Adjust /path/to according where your fail2ban-client is installed. > >>> This does rotate the logfile and keeps fail2ban-server running >>> and working: > >>> ~ $ ps auxwww|grep fail2ban >>> root 14143 0.0 0.2 39848 10400 ?? S 2Feb13 8:59.41 >>> /usr/local/bin/python2.7 /usr/local/bin/fail2ban-server -b -s >>> /var/run/fail2ban/fail2ban.sock >>> ~ $ > >>> ~ $ ls -1tr /var/log/fail2ban.log* >>> /var/log/fail2ban.log-20130106.gz >>> /var/log/fail2ban.log-20130113.gz >>> /var/log/fail2ban.log-20130120.gz >>> /var/log/fail2ban.log-20130127.gz >>> /var/log/fail2ban.log-20130203.gz >>> /var/log/fail2ban.log-20130210.gz >>> /var/log/fail2ban.log-20130217 >>> /var/log/fail2ban.log >>> ~ $ > >>> Try with the added ' || true' at the end of your postrotate command. > > >>> bye >>> Fabian > >> Hi, Fabian, > >> Thanks for the tips. I have added the full path to fail2ban-client (even >> though it doesn't seem to necessary, given that fail2ban-client is in my >> PATH environment variable), as well as " || true" to the post-rotate >> command, per your example. > >> Unfortunately, I am still unable to start fail2ban in the normal way (on >> Debian): > >> # service fail2ban start > >> My understanding is that this is equivalent to: > >> # sh /etc/init.d/fail2ban start > >> Executing this command requires about 10 seconds and no output is >> produced at all before the cursor drops back to the prompt. > >> I tried some of the other daemon options: > >> # sh /etc/init.d/fail2ban status >> * Status of authentication failure monitor >> * fail2ban is not running > >> # sh /etc/init.d/fail2ban restart >> * Restarting authentication failure monitor fail2ban [fail] > >> This is exactly what happened when I ran into this problem a few months >> ago. Once the log rotation "borks" fail2ban, I'm unable to start it by >> normal means. It seems that something is "stuck" in a non-sensible state. > >> Last time this occurred, I reinstalled fail2ban and the service started, >> but I would really prefer not to have to go to that length to fix this. > >> What temp files, if any, does fail2ban use? Any ideas? > >> Thanks again, > >> -Ben > >> ------------------------------------------------------------------------------ >> The Go Parallel Website, sponsored by Intel - in partnership with Geeknet, >> is your hub for all things parallel software development, from weekly thought >> leadership blogs to news, videos, case studies, tutorials, tech docs, >> whitepapers, evaluation guides, and opinion stories. Check out the most >> recent posts - join the conversation now. http://goparallel.sourceforge.net/ >> _______________________________________________ >> Fail2ban-users mailing list >> Fai...@li... >> https://lists.sourceforge.net/lists/listinfo/fail2ban-users > > |
From: Ben J. <be...@in...> - 2013-02-19 21:44:59
|
On 2/19/2013 2:08 PM, Yaroslav Halchenko wrote: > > On Tue, 19 Feb 2013, Ben Johnson wrote: > >>> if [ ! -d /var/run/fail2ban ] ; then >>> mkdir /var/run/fail2ban >>> fi > >> Adding these lines to the init script seems like a good idea. > > they are there (have been since a while actually: > > # Assure that /var/run/fail2ban exists > [ -d /var/run/fail2ban ] || mkdir -p /var/run/fail2ban In what file are these two lines? They definitely don't appear anywhere in my /etc/init.d/fail2ban script. Oh, wait, I found them: /etc/init.d/fail2ban.dpkg-dist: # Assure that /var/run/fail2ban exists /etc/init.d/fail2ban.dpkg-dist: [ -d /var/run/fail2ban ] || mkdir -p /var/run/fail2ban /etc/init.d/fail2ban.dpkg-dist: chown "$FAIL2BAN_USER" /var/run/fail2ban I think I found the bug report for Ubuntu ( https://bugs.launchpad.net/ubuntu/+source/fail2ban/+bug/222804 ), and its status is "Fix Released", but that doesn't explain why this is still a problem on my system, with fail2ban-0.8.8. Is there any chance that the 0.8.8 package that you built for Ubuntu 10.04 copies the wrong (obsoleted) script to /etc/init.d/fail2ban and copies the correct (current) script to /etc/init.d/fail2ban.dpkg-dist? Here's the URL to the package I downloaded: http://neuro.debian.net/debian/pool/main/f/fail2ban/fail2ban_0.8.8-1+lucid0~nd10.04+1_all.deb > * 32281ed - BF: Assure that /var/run/fail2ban exists upon start (LP: #222804, #223706) (4 years, 10 months ago) [Yaroslav Halchenko] > > that was at times of 0.8.2 but the problem with them that I better move > them BEFORE the point of checking the status ;) > > now if directory doesn't exist, status fails (probably) and then init > script doesn't get to the point of creating them... I would need to > double check on that You are a hero! >> The question remains: how was this directory deleted in the first place, >> if no tmpfs is at play? As I stated earlier, fail2ban *did* log its >> shutdown sequence when the system notified it was going down for reboot; >> is there anything in the source code that would remove the >> /var/run/fail2ban directory under certain shutdown conditions? > > I would first verify that it indeed happens (reboot again) and then > possibly grep for 'var/run' among init scripts -- may be some of them > perform some cleanup? ;)... eg. Yes indeed. Rebooting causes the /var/run/fail2ban directory to be deleted. Cursory research indicates that the potential for anything in /var/run to be deleted upon reboot is well-understood. A quick "grep -ir "/var/run" /etc/init.d" reveals that several other init scripts check for their respective directories and create them if they don't exist. A handful of examples: /etc/init.d/bind9:PIDFILE=/var/run/named/named.pid /etc/init.d/bind9: # dirs under /var/run can go away on reboots. /etc/init.d/bind9: mkdir -p /var/run/named /etc/init.d/bind9: chmod 775 /var/run/named /etc/init.d/bind9: chown root:bind /var/run/named >/dev/null 2>&1 || true /etc/init.d/fetchmail: mkdir /var/run/fetchmail /etc/init.d/fetchmail: chown -h $USER:nogroup /var/run/fetchmail /etc/init.d/fetchmail: chmod 700 /var/run/fetchmail /etc/init.d/mailman:PIDFILE=/var/run/mailman/mailman.pid /etc/init.d/mailman:if ! [ -d /var/run/mailman ]; then /etc/init.d/mailman: install -d -o list -g list /var/run/mailman /etc/init.d/ssh: if [ ! -d /var/run/sshd ]; then /etc/init.d/ssh: mkdir /var/run/sshd /etc/init.d/ssh: chmod 0755 /var/run/sshd So, it seems that the fix that you already implemented is the "correct" way to do this. At this point, I think this is a simple case of "the wrong init script made it into the unofficially-back-ported Ubuntu 10.04 package". Copying the correct script over the incorrect one seems to fix the problem; fail2ban starts correct even if the directory does not exist. > init.d/mountnfs-bootclean.sh: # Clean /tmp, /var/lock, /var/run > > although on my system that /lib/init/bootclean.sh has > > clean /run "! -xtype d ! -name utmp ! -name innd.pid" || ES=1 > > so I guess it should not remove subdirectories... but who knows -- may be yours > is different ? ;) > It seems so! I don't have the file /lib/init/bootclean.sh, but some other script must be doing this. We may never know! Thanks for everything! -Ben |
From: Yaroslav H. <li...@on...> - 2013-02-20 00:18:14
|
On Tue, 19 Feb 2013, Ben Johnson wrote: > I think I found the bug report for Ubuntu ( > https://bugs.launchpad.net/ubuntu/+source/fail2ban/+bug/222804 ), and > its status is "Fix Released", but that doesn't explain why this is still > a problem on my system, with fail2ban-0.8.8. > Is there any chance that the 0.8.8 package that you built for Ubuntu > 10.04 copies the wrong (obsoleted) script to /etc/init.d/fail2ban and > copies the correct (current) script to /etc/init.d/fail2ban.dpkg-dist? if you ever introduced manual changes to that init file and then said to keep your copy upon upgrades -- here you come ;) > Here's the URL to the package I downloaded: > http://neuro.debian.net/debian/pool/main/f/fail2ban/fail2ban_0.8.8-1+lucid0~nd10.04+1_all.deb > > * 32281ed - BF: Assure that /var/run/fail2ban exists upon start (LP: #222804, #223706) (4 years, 10 months ago) [Yaroslav Halchenko] > > that was at times of 0.8.2 but the problem with them that I better move > > them BEFORE the point of checking the status ;) > > now if directory doesn't exist, status fails (probably) and then init > > script doesn't get to the point of creating them... I would need to > > double check on that > You are a hero! yeay! ;) > So, it seems that the fix that you already implemented is the "correct" > way to do this. > At this point, I think this is a simple case of "the wrong init script > made it into the unofficially-back-ported Ubuntu 10.04 package". nah -- I think it is that you just kept your custom file from some elderly buggy version... that would also explain why noone else recently ran into this situation ;-) > Copying the correct script over the incorrect one seems to fix the > problem; fail2ban starts correct even if the directory does not exist. good > > init.d/mountnfs-bootclean.sh: # Clean /tmp, /var/lock, /var/run > > although on my system that /lib/init/bootclean.sh has > > clean /run "! -xtype d ! -name utmp ! -name innd.pid" || ES=1 > > so I guess it should not remove subdirectories... but who knows -- may be yours > > is different ? ;) > It seems so! I don't have the file /lib/init/bootclean.sh, but some > other script must be doing this. We may never know! and it doesn't really matter now ;) > Thanks for everything! Cheers! -- Yaroslav O. Halchenko http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Yaroslav H. <li...@on...> - 2013-02-19 02:02:34
|
On Mon, 18 Feb 2013, Ben Johnson wrote: > > [FAIL] Starting authentication failure monitor: fail2ban[....] Socket file /var/run/fail2ban/fail2ban.sock is present ... failed! > > . ok > Hmm. I'm on Debian (well, Ubuntu), too, and I don't even have the > directory /var/run/fail2ban, no less the file > /var/run/fail2ban/fail2ban.sock. On another similarly configured server, > that directory and that file exist. Well, at least we know that not to > be the problem! ah.. right -- we all (debian and ubuntu) decided to go for /run, I forgotten about that, so indeed you might be cherishing the fruits of such changes... Myself I have not run into this since on Debian systems we do still maintain compatibility with a symlink... so as a quick workaround you could try doing sudo ln -s /run /var/ So I guess all my "backports" for the recent ubuntus out there are broken on those systems if they did not maintain this level of compatibility... -- Yaroslav O. Halchenko http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Yaroslav H. <li...@on...> - 2012-12-06 18:11:22
|
and make sure you remove your 'manually installed one' first ;) On Thu, 06 Dec 2012, Yaroslav Halchenko wrote: > On Thu, 06 Dec 2012, Ben Johnson wrote: > > I had to install manually because I'm stuck on Ubuntu 10.04 for the time > and when I saw that build for 10.04 has failed I thought "who would use > this old one???" -- so here you come ;-/ the reason for absent package > is that I switched at some point in the back to use dh_python2 helper > instead of deprecated pycentral... hold on there -- I will prep a > special build for this elderly beastie in an hour or so > > # service fail2ban start > > does not produce any output (nor does stop). > I would prefer not even to bother figuring this out but rather give you > a nice package -- wait a bit -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Ben J. <be...@in...> - 2012-12-06 20:46:36
|
On 12/6/2012 1:11 PM, Yaroslav Halchenko wrote: > and make sure you remove your 'manually installed one' first ;) > > On Thu, 06 Dec 2012, Yaroslav Halchenko wrote: > > >> On Thu, 06 Dec 2012, Ben Johnson wrote: > >>> I had to install manually because I'm stuck on Ubuntu 10.04 for the time > >> and when I saw that build for 10.04 has failed I thought "who would use >> this old one???" -- so here you come ;-/ the reason for absent package >> is that I switched at some point in the back to use dh_python2 helper >> instead of deprecated pycentral... hold on there -- I will prep a >> special build for this elderly beastie in an hour or so > > >>> # service fail2ban start > >>> does not produce any output (nor does stop). > >> I would prefer not even to bother figuring this out but rather give you >> a nice package -- wait a bit Thanks, Yaroslav! You're a good man. So, I downloaded the package and attempted to install it: ------------------------------------------- # dpkg --install fail2ban_0.8.8-1+lucid0~nd10.04+1_all.deb Selecting previously deselected package fail2ban. (Reading database ... 48504 files and directories currently installed.) Unpacking fail2ban (from fail2ban_0.8.8-1+lucid0~nd10.04+1_all.deb) ... Setting up fail2ban (0.8.8-1+lucid0~nd10.04+1) ... Traceback (most recent call last): File "/usr/bin/pycentral", line 2300, in <module> main() File "/usr/bin/pycentral", line 2294, in main rv = action.run(global_options) File "/usr/bin/pycentral", line 1481, in run pkg.read_version_info() File "/usr/bin/pycentral", line 897, in read_version_info raise PyCentralError, "package has no field Python-Version" __main__.PyCentralError: package has no field Python-Version dpkg: error processing fail2ban (--install): subprocess installed post-installation script returned error exit status 1 Errors were encountered while processing: fail2ban ------------------------------------------- It seems as though fail2ban was installed successfully, though. The service was started automatically and can be stopped and started with "service fail2ban start|stop". Is this a non-issue? Cheers, and thanks again for whipping-up a package that upon cursory inspection seems to do the job! -Ben |
From: Ben J. <be...@in...> - 2013-02-19 03:07:18
|
On 2/18/2013 9:02 PM, Yaroslav Halchenko wrote: > > On Mon, 18 Feb 2013, Ben Johnson wrote: >>> [FAIL] Starting authentication failure monitor: fail2ban[....] Socket file /var/run/fail2ban/fail2ban.sock is present ... failed! >>> . ok > >> Hmm. I'm on Debian (well, Ubuntu), too, and I don't even have the >> directory /var/run/fail2ban, no less the file >> /var/run/fail2ban/fail2ban.sock. On another similarly configured server, >> that directory and that file exist. Well, at least we know that not to >> be the problem! > > ah.. right -- we all (debian and ubuntu) decided to go for /run, I > forgotten about that, so indeed you might be cherishing the fruits of > such changes... Myself I have not run into this since on Debian systems we do > still maintain compatibility with a symlink... so as a quick workaround you > could try doing You lost me there. ;) To be clear, I do indeed have the directory /var/run. It just doesn't contain a "fail2ban" sub-directory (it contains plenty of other items). I'm on an older OS, Ubuntu 10.04 (you declared it an "elderly beastie", quite aptly), and as such, shouldn't be affected by Debian/Ubuntu transitioning from /var/run to /run. As I mentioned, I have another nearly identically-configured server, with fail2ban 0.8.6 (the latest version available from the 10.04 repos), and the /var/run/fail2ban directory, and socket file within it, do exist there. fail2ban has been running fine on that server since the day I installed it (1.5 years ago, if memory serves). Also, this installation of fail2ban-0.8.8 has functioned without issue since you created the Ubuntu 10.04 backport .deb package for me several months ago. I thought we were "out of the woods", so to speak. But, alas, this seems to be the same problem that I described back in December, 2012, on this list ( http://sourceforge.net/mailarchive/forum.php?thread_name=50BE77A2.4060300%40indietorrent.org&forum_name=fail2ban-users ). Oddly, it appears that several log-rotations have happened successfully since then. I'm not sure why this most recent one "broke" fail2ban. I don't know if it's relevant, but when I try to start fail2ban with "fail2ban-client start", it takes about 30 seconds for the error, "ERROR Could not start server. Maybe an old socket file is still present. Try to remove /var/run/fail2ban/fail2ban.sock. If you used fail2ban-client to start the server, adding the -x option will do it". What's taking so long? I do find it a bit odd that fail2ban doesn't log its start-up sequence, in detail, especially at maximum verbosity. It seems that this is not simply a matter of increasing the log-level; I had it at 3, and just changed it to 4, but absolutely nothing is written to /var/log/fail2ban.log. If the client executable can spit-out an error message, one would think it capable of writing other (more useful) information to the log-file, too. I haven't changed anything recently, either. In /etc/fail2ban/fail2ban.conf: logtarget = /var/log/fail2ban.log socket = /var/run/fail2ban/fail2ban.sock Those look correct, and as I said, shouldn't have changed any time recently. What else might we check? > sudo ln -s /run /var/ > > So I guess all my "backports" for the recent ubuntus out there are broken > on those systems if they did not maintain this level of compatibility... > I'm still a bit hazy on the issue here, or if that was just a miscommunication. As expected, this command yields the following on the affected system: "ln: creating symbolic link `/var/run': File exists". Thanks! -Ben |
From: Yaroslav H. <li...@on...> - 2013-02-19 03:38:19
|
On Mon, 18 Feb 2013, Ben Johnson wrote: > >>> [FAIL] Starting authentication failure monitor: fail2ban[....] Socket file /var/run/fail2ban/fail2ban.sock is present ... failed! > >>> . ok > >> Hmm. I'm on Debian (well, Ubuntu), too, and I don't even have the > >> directory /var/run/fail2ban, no less the file > >> /var/run/fail2ban/fail2ban.sock. On another similarly configured server, > >> that directory and that file exist. Well, at least we know that not to > >> be the problem! > > ah.. right -- we all (debian and ubuntu) decided to go for /run, I > > forgotten about that, so indeed you might be cherishing the fruits of > > such changes... Myself I have not run into this since on Debian systems we do > > still maintain compatibility with a symlink... so as a quick workaround you > > could try doing > You lost me there. ;) we are even then! ;-) > To be clear, I do indeed have the directory /var/run. It just doesn't > contain a "fail2ban" sub-directory (it contains plenty of other items). > I'm on an older OS, Ubuntu 10.04 (you declared it an "elderly beastie", > quite aptly), and as such, shouldn't be affected by Debian/Ubuntu > transitioning from /var/run to /run. ah -- then I have no clue where /var/run/fail2ban is gone since since 0.8.2 init script creates it: $> git lg debian/fail2ban.init * 86ae7d2 - debian/fail2ban.init: Should-(start|stop): iptables-persistent (Closes: #598109), ferm (Closes: #604843) (7 months ago) [Yaroslav Halchenko] * 7154b3a - Since we are exiting with non-0 -- all msgs should be failures (1 year ago) [Yaroslav Halchenko] * 5cac32e - Making sure that the status command gives the correct exit code when fail2ban is not running. (1 year ago) [Glenn Aaldering] * de502cf - NF: run as different user (disabled by default) (1 year, 1 month ago) [Zbigniew Jędrzejewski-Szmek] * d1b9e71 - Adding arno-iptables-firewall (no deprecation of ipmasq per Joey Hess mentioning, which still could be used on lenny systems) (2 years, 9 months ago) [Yaroslav Halchenko] * 32281ed - BF: Assure that /var/run/fail2ban exists upon start (LP: #222804, #223706) (4 years, 10 months ago) [Yaroslav Halchenko] ... $> git describe --contains 32281ed debian/0.8.2-3~1^2 so I would look for someone who destroyed it for you! ;) > As I mentioned, I have another nearly identically-configured server, > with fail2ban 0.8.6 (the latest version available from the 10.04 repos), > and the /var/run/fail2ban directory, and socket file within it, do exist > there. as they should! ;-) good > Also, this installation of fail2ban-0.8.8 has functioned without issue > since you created the Ubuntu 10.04 backport .deb package for me several > months ago. I thought we were "out of the woods", so to speak. succesful start means that /var/run/fail2ban was there!;-) > But, alas, this seems to be the same problem that I described back in > December, 2012, on this list ( > http://sourceforge.net/mailarchive/forum.php?thread_name=50BE77A2.4060300%40indietorrent.org&forum_name=fail2ban-users > ). sorry -- my memory span getting shorter and shorter with every email I read ;-) > Could not start server. Maybe an old socket file is still present. Try > to remove /var/run/fail2ban/fail2ban.sock. If you used fail2ban-client > to start the server, adding the -x option will do it". > What's taking so long? hope that server would come up -- look at __waitOnServer function in -client and its comment on 30 sec there ;) > I do find it a bit odd that fail2ban doesn't log its start-up sequence, > in detail, especially at maximum verbosity. It seems that this is not > simply a matter of increasing the log-level; I had it at 3, and just > changed it to 4, but absolutely nothing is written to > /var/log/fail2ban.log. yeah -- that is how it was designed... -client dumps stuff only to stdout, not to the .log file... not sure if it would function just fine with two processes (-client and -server) trying to dump into the same file > If the client executable can spit-out an error > message, one would think it capable of writing other (more useful) > information to the log-file, too. > I haven't changed anything recently, either. In /etc/fail2ban/fail2ban.conf: > logtarget = /var/log/fail2ban.log > socket = /var/run/fail2ban/fail2ban.sock > Those look correct, and as I said, shouldn't have changed any time recently. > What else might we check? check on who killed /var/run/fail2ban ;-) meanwhile just mkdir /var/run/fail2ban/ and see if that helps (it should!) > I'm still a bit hazy on the issue here, or if that was just a > miscommunication. As expected, this command yields the following on the > affected system: "ln: creating symbolic link `/var/run': File exists". right -- my wild guess was wrong due to misinterpretation of your email... just mkdir -p /var/run/fail2ban and be happy again ;-) -- Yaroslav O. Halchenko http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Fabian W. <fa...@we...> - 2013-02-19 14:13:38
|
Hello Ben Hello Yaroslav On 19.02.2013 04:38, Yaroslav Halchenko wrote: > On Mon, 18 Feb 2013, Ben Johnson wrote: >> To be clear, I do indeed have the directory /var/run. It just doesn't >> contain a "fail2ban" sub-directory (it contains plenty of other items). >> I'm on an older OS, Ubuntu 10.04 (you declared it an "elderly beastie", >> quite aptly), and as such, shouldn't be affected by Debian/Ubuntu >> transitioning from /var/run to /run. Gentoo is also doing the transition from /var/run to the tmpfs /run and so /var/run is just a symlink pointing to /run. But as /run is a tmpfs, it is empty again after a system reboot. > ah -- then I have no clue where /var/run/fail2ban is gone since since > 0.8.2 init script creates it: @Ben, could you check with 'ls -l /var/run' (without a / after run) if this already is a symlink to /run. if yes, check e.g. with 'df -hl | grep /run' what kind of filesystem /run is. > so I would look for someone who destroyed it for you! ;) Maybe the above checks will resolve this. bye Fabian |
From: Ben J. <be...@in...> - 2013-02-19 15:02:27
|
On 2/19/2013 9:13 AM, Fabian Wenk wrote: > Hello Ben > Hello Yaroslav > > On 19.02.2013 04:38, Yaroslav Halchenko wrote: >> On Mon, 18 Feb 2013, Ben Johnson wrote: > >>> To be clear, I do indeed have the directory /var/run. It just doesn't >>> contain a "fail2ban" sub-directory (it contains plenty of other items). >>> I'm on an older OS, Ubuntu 10.04 (you declared it an "elderly beastie", >>> quite aptly), and as such, shouldn't be affected by Debian/Ubuntu >>> transitioning from /var/run to /run. > > Gentoo is also doing the transition from /var/run to the tmpfs > /run and so /var/run is just a symlink pointing to /run. But as > /run is a tmpfs, it is empty again after a system reboot. > >> ah -- then I have no clue where /var/run/fail2ban is gone since since >> 0.8.2 init script creates it: > > @Ben, could you check with 'ls -l /var/run' (without a / after > run) if this already is a symlink to /run. if yes, check e.g. > with 'df -hl | grep /run' what kind of filesystem /run is. I checked this out and it's not a symlink; it's a real directory. >> so I would look for someone who destroyed it for you! ;) That will be the hard part. ;) I'm the only one with access to this server, and I'm sufficiently seasoned so as not to go around deleting directories in /var/run. Something (or someone) else deleted the /var/run/fail2ban directory. The question is, what or who? A better question might be, why does fail2ban not recreate this directory if it's missing, instead of "failing" outright. This may be a clear-cut case of ban2fail! :) In all seriousness, though, is there some reason for which this directory is not recreated if necessary? How is it created in the first place? During installation? -Ben > Maybe the above checks will resolve this. > > > bye > Fabian > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_d2d_feb > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users > |
From: Fabian W. <fa...@we...> - 2013-02-19 15:20:43
|
Hello Ben On 19.02.2013 16:02, Ben Johnson wrote: > On 2/19/2013 9:13 AM, Fabian Wenk wrote: >> Gentoo is also doing the transition from /var/run to the tmpfs >> /run and so /var/run is just a symlink pointing to /run. But as >> /run is a tmpfs, it is empty again after a system reboot. >> >>> ah -- then I have no clue where /var/run/fail2ban is gone since since >>> 0.8.2 init script creates it: >> >> @Ben, could you check with 'ls -l /var/run' (without a / after >> run) if this already is a symlink to /run. if yes, check e.g. >> with 'df -hl | grep /run' what kind of filesystem /run is. > > I checked this out and it's not a symlink; it's a real directory. Could you check if eventually /var/run is also a tmpfs? Did it fail after you have rebooted your system? >>> so I would look for someone who destroyed it for you! ;) > A better question might be, why does fail2ban not recreate this > directory if it's missing, instead of "failing" outright. This may be a > clear-cut case of ban2fail! :) > > In all seriousness, though, is there some reason for which this > directory is not recreated if necessary? How is it created in the first > place? During installation? During the installation the directory is created from the .deb package, e.g. aptitude or apt-get does create it according to the post script in the .deb. On system boot, or start of fail2ban with '/etc/init.d/fail2ban start' (which this newish 'service fail2ban start' also uses) this script should check it. Eventually adding something like this at top in the start section could help: if [ ! -d /var/run/fail2ban ] ; then mkdir /var/run/fail2ban fi bye Fabian |
From: Fabian W. <fa...@we...> - 2013-02-19 19:00:34
|
Hello Ben On 19.02.2013 19:37, Ben Johnson wrote: >> Could you check if eventually /var/run is also a tmpfs? Did it >> fail after you have rebooted your system? > > I don't see any indication that /var/run is a tmpfs. Here's the output > from "df": > > # df > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/vzfs 80000000 47984964 32015036 60% / > /dev/simfs 80000000 47984964 32015036 60% /tmp > /dev/simfs 80000000 47984964 32015036 60% /var/tmp > > The contents of /etc/fstab are: > > proc /proc proc defaults 0 0 > none /dev/pts devpts rw 0 0 > /dev/vzfs / reiserfs rw,usrquota,grpquota 0 0 Yes, this does not indicate that /var/run is on a tmpfs, and as you already wrote, it is not symlinked to somewhere else. >> On system boot, or start of fail2ban with '/etc/init.d/fail2ban >> start' (which this newish 'service fail2ban start' also uses) >> this script should check it. Eventually adding something like >> this at top in the start section could help: >> >> if [ ! -d /var/run/fail2ban ] ; then >> mkdir /var/run/fail2ban >> fi > > Adding these lines to the init script seems like a good idea. I'll add > them to my own copy right now, but it would be nice to have assurance > that this change will be committed to the source code. Sure this is a work around for now. But I am not sure if this is the proper way for doing it on Ubuntu, probably checking some other init scripts will tell. > The theory that a reboot is what broke fail2ban has merit. I dug back > through my Logwatch reports and discovered that fail2ban stopped banning > on Feb. 11. Take a look at the mod-dates for the directories in /var/run: > All evidence points to the fact that a reboot was performed Feb 11, > shortly after 21:00, which sounds about right (I was doing some > maintenance that required a reboot that night). Yes it looks like this way. > The question remains: how was this directory deleted in the first place, > if no tmpfs is at play? As I stated earlier, fail2ban *did* log its > shutdown sequence when the system notified it was going down for reboot; > is there anything in the source code that would remove the > /var/run/fail2ban directory under certain shutdown conditions? I guess not from fail2ban, but could be some system shutdown / restart cleanup script which is doing something like 'rm -rf /var/run/*'. Because most often it can cause problems if there are .pid files left after as system crash on system boot up. I have seen daemons not being able to start because the .pid file was still there but with the PID noted in there was already used from an other process, which then on 'restart' could not be killed from the starting services because the init script had already dropped his privileged to a regular user. I hope this was not written to complicated. ;) > Perhaps this point is moot, as long as your modification is added to the > init script: > > if [ ! -d /var/run/fail2ban ] ; then > mkdir /var/run/fail2ban > fi > > Thanks for the viable workaround, Fabian! You're welcome! bye Fabian |
From: Ben J. <be...@in...> - 2013-02-19 21:01:36
|
On 2/19/2013 2:00 PM, Fabian Wenk wrote: > Hello Ben > > On 19.02.2013 19:37, Ben Johnson wrote: >>> Could you check if eventually /var/run is also a tmpfs? Did it >>> fail after you have rebooted your system? >> >> I don't see any indication that /var/run is a tmpfs. Here's the output >> from "df": >> >> # df >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/vzfs 80000000 47984964 32015036 60% / >> /dev/simfs 80000000 47984964 32015036 60% /tmp >> /dev/simfs 80000000 47984964 32015036 60% /var/tmp >> >> The contents of /etc/fstab are: >> >> proc /proc proc defaults 0 0 >> none /dev/pts devpts rw 0 0 >> /dev/vzfs / reiserfs rw,usrquota,grpquota 0 0 > > Yes, this does not indicate that /var/run is on a tmpfs, and as > you already wrote, it is not symlinked to somewhere else. > >>> On system boot, or start of fail2ban with '/etc/init.d/fail2ban >>> start' (which this newish 'service fail2ban start' also uses) >>> this script should check it. Eventually adding something like >>> this at top in the start section could help: >>> >>> if [ ! -d /var/run/fail2ban ] ; then >>> mkdir /var/run/fail2ban >>> fi >> >> Adding these lines to the init script seems like a good idea. I'll add >> them to my own copy right now, but it would be nice to have assurance >> that this change will be committed to the source code. > > Sure this is a work around for now. But I am not sure if this is > the proper way for doing it on Ubuntu, probably checking some > other init scripts will tell. > >> The theory that a reboot is what broke fail2ban has merit. I dug back >> through my Logwatch reports and discovered that fail2ban stopped banning >> on Feb. 11. Take a look at the mod-dates for the directories in /var/run: > >> All evidence points to the fact that a reboot was performed Feb 11, >> shortly after 21:00, which sounds about right (I was doing some >> maintenance that required a reboot that night). > > Yes it looks like this way. > >> The question remains: how was this directory deleted in the first place, >> if no tmpfs is at play? As I stated earlier, fail2ban *did* log its >> shutdown sequence when the system notified it was going down for reboot; >> is there anything in the source code that would remove the >> /var/run/fail2ban directory under certain shutdown conditions? > > I guess not from fail2ban, but could be some system shutdown / > restart cleanup script which is doing something like 'rm -rf > /var/run/*'. Because most often it can cause problems if there > are .pid files left after as system crash on system boot up. > I have seen daemons not being able to start because the .pid file > was still there but with the PID noted in there was already used > from an other process, which then on 'restart' could not be > killed from the starting services because the init script had > already dropped his privileged to a regular user. I hope this was > not written to complicated. ;) Your explanation makes sense. And this seems to be even more reason for fail2ban to create this directory when the service is started if it does not already exist. (Yaroslav acknowledged this need in a subsequent reply; I'll respond to his message separately.) I put this reboot theory to the test, and sure enough, the /var/run/fail2ban directory was gone again! So, something is definitely deleting the directory during shutdown or startup. I have a pretty "stock" Ubuntu 10.04 system. If this is happening to me, it's happening to tens of thousands of other people. Why this doesn't seem to occur on another similarly-configured server is another mystery. I rebooted that server around the same time, and fail2ban came back up without issue. Perhaps this is largely irrelevant, though, as fail2ban has no control over what the operating system does with /var/run (or /run). Thanks for the clear explanation, Fabian. -Ben >> Perhaps this point is moot, as long as your modification is added to the >> init script: >> >> if [ ! -d /var/run/fail2ban ] ; then >> mkdir /var/run/fail2ban >> fi >> >> Thanks for the viable workaround, Fabian! > > You're welcome! > > > bye > Fabian > > ------------------------------------------------------------------------------ > Everyone hates slow websites. So do we. > Make your web apps faster with AppDynamics > Download AppDynamics Lite for free today: > http://p.sf.net/sfu/appdyn_d2d_feb > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users > |
From: Yaroslav H. <li...@on...> - 2012-12-06 23:32:05
|
On Thu, 06 Dec 2012, Ben Johnson wrote: > It seems as though fail2ban was installed successfully, though. The > service was started automatically and can be stopped and started with > "service fail2ban start|stop". > Is this a non-issue? most probably ;-) let's just keep it at this... if more people run into it I would look into it > Cheers, and thanks again for whipping-up a package that upon cursory > inspection seems to do the job! you are welcome -- enjoy -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Yaroslav H. <li...@on...> - 2013-02-19 04:12:29
|
uff -- even not sure what to say, I thought that I have replicated your situation (and creating /var/run/fail2ban didn't help) but then it stopped "replicating" ;) so if creating that directory doesn't help, report back meanwhile -- I have added more debug messages printed by client so that fail2ban-client -v -v -v start should be more informative now also check if you somehow do not have some other fail2ban-client laying around (i.e. "which fail2ban-client") and that you still have fail2ban-server in the same directory (hopefully /usr/bin/) as that -client. Cheers On Mon, 18 Feb 2013, Yaroslav Halchenko wrote: > On Mon, 18 Feb 2013, Ben Johnson wrote: > > >>> [FAIL] Starting authentication failure monitor: fail2ban[....] Socket file /var/run/fail2ban/fail2ban.sock is present ... failed! > > >>> . ok > > >> Hmm. I'm on Debian (well, Ubuntu), too, and I don't even have the > > >> directory /var/run/fail2ban, no less the file > > >> /var/run/fail2ban/fail2ban.sock. On another similarly configured server, > > >> that directory and that file exist. Well, at least we know that not to > > >> be the problem! > > > ah.. right -- we all (debian and ubuntu) decided to go for /run, I > > > forgotten about that, so indeed you might be cherishing the fruits of > > > such changes... Myself I have not run into this since on Debian systems we do > > > still maintain compatibility with a symlink... so as a quick workaround you > > > could try doing > > You lost me there. ;) > we are even then! ;-) > > To be clear, I do indeed have the directory /var/run. It just doesn't > > contain a "fail2ban" sub-directory (it contains plenty of other items). > > I'm on an older OS, Ubuntu 10.04 (you declared it an "elderly beastie", > > quite aptly), and as such, shouldn't be affected by Debian/Ubuntu > > transitioning from /var/run to /run. > ah -- then I have no clue where /var/run/fail2ban is gone since since > 0.8.2 init script creates it: > $> git lg debian/fail2ban.init > * 86ae7d2 - debian/fail2ban.init: Should-(start|stop): iptables-persistent (Closes: #598109), ferm (Closes: #604843) (7 months ago) [Yaroslav Halchenko] > * 7154b3a - Since we are exiting with non-0 -- all msgs should be failures (1 year ago) [Yaroslav Halchenko] > * 5cac32e - Making sure that the status command gives the correct exit code when fail2ban is not running. (1 year ago) [Glenn Aaldering] > * de502cf - NF: run as different user (disabled by default) (1 year, 1 month ago) [Zbigniew Jędrzejewski-Szmek] > * d1b9e71 - Adding arno-iptables-firewall (no deprecation of ipmasq per Joey Hess mentioning, which still could be used on lenny systems) (2 years, 9 months ago) [Yaroslav Halchenko] > * 32281ed - BF: Assure that /var/run/fail2ban exists upon start (LP: #222804, #223706) (4 years, 10 months ago) [Yaroslav Halchenko] > ... > $> git describe --contains 32281ed > debian/0.8.2-3~1^2 > so I would look for someone who destroyed it for you! ;) > > As I mentioned, I have another nearly identically-configured server, > > with fail2ban 0.8.6 (the latest version available from the 10.04 repos), > > and the /var/run/fail2ban directory, and socket file within it, do exist > > there. > as they should! ;-) good > > Also, this installation of fail2ban-0.8.8 has functioned without issue > > since you created the Ubuntu 10.04 backport .deb package for me several > > months ago. I thought we were "out of the woods", so to speak. > succesful start means that /var/run/fail2ban was there!;-) > > But, alas, this seems to be the same problem that I described back in > > December, 2012, on this list ( > > http://sourceforge.net/mailarchive/forum.php?thread_name=50BE77A2.4060300%40indietorrent.org&forum_name=fail2ban-users > > ). > sorry -- my memory span getting shorter and shorter with every email I > read ;-) > > Could not start server. Maybe an old socket file is still present. Try > > to remove /var/run/fail2ban/fail2ban.sock. If you used fail2ban-client > > to start the server, adding the -x option will do it". > > What's taking so long? > hope that server would come up -- look at __waitOnServer function in > -client and its comment on 30 sec there ;) > > I do find it a bit odd that fail2ban doesn't log its start-up sequence, > > in detail, especially at maximum verbosity. It seems that this is not > > simply a matter of increasing the log-level; I had it at 3, and just > > changed it to 4, but absolutely nothing is written to > > /var/log/fail2ban.log. > yeah -- that is how it was designed... -client dumps stuff only to stdout, not > to the .log file... not sure if it would function just fine with two processes > (-client and -server) trying to dump into the same file > > If the client executable can spit-out an error > > message, one would think it capable of writing other (more useful) > > information to the log-file, too. > > I haven't changed anything recently, either. In /etc/fail2ban/fail2ban.conf: > > logtarget = /var/log/fail2ban.log > > socket = /var/run/fail2ban/fail2ban.sock > > Those look correct, and as I said, shouldn't have changed any time recently. > > What else might we check? > check on who killed /var/run/fail2ban ;-) > meanwhile just mkdir /var/run/fail2ban/ and see if that helps (it should!) > > I'm still a bit hazy on the issue here, or if that was just a > > miscommunication. As expected, this command yields the following on the > > affected system: "ln: creating symbolic link `/var/run': File exists". > right -- my wild guess was wrong due to misinterpretation of your email... just > mkdir -p /var/run/fail2ban > and be happy again ;-) -- Yaroslav O. Halchenko http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik |
From: Ben J. <be...@in...> - 2013-02-19 14:54:19
|
On 2/18/2013 11:12 PM, Yaroslav Halchenko wrote: > uff -- even not sure what to say, I thought that I have replicated your > situation (and creating /var/run/fail2ban didn't help) but then it > stopped "replicating" ;) > > so if creating that directory doesn't help, report back Actually, that solved it! Creating the directory manually allowed fail2ban to start normally. The question then becomes, how the heck did this directory disappear in the first place? I'm the only one with access to this server, and surely I didn't delete it. Given that this has happened before (this is the second time with this issue), some process seems to be removing this directory. An "attack" of some kind seems unlikely, although, anything is possible. Is there any possibility that fail2ban itself is deleting this directory in error? Thanks for getting me back up-and-running here, Yaroslav (and Fabian)! -Ben P.S. Any chance of a fix for that Python issue in the back-ported 10.04 .deb package? ;) > meanwhile -- I have added more debug messages printed by client so that > > fail2ban-client -v -v -v start > > should be more informative now > > also check if you somehow do not have some other fail2ban-client laying > around (i.e. "which fail2ban-client") and that you still have > fail2ban-server in the same directory (hopefully /usr/bin/) as that > -client. # which fail2ban-client /usr/bin/fail2ban-client So, just one instance of the application. > Cheers > > On Mon, 18 Feb 2013, Yaroslav Halchenko wrote: > > >> On Mon, 18 Feb 2013, Ben Johnson wrote: >>>>>> [FAIL] Starting authentication failure monitor: fail2ban[....] Socket file /var/run/fail2ban/fail2ban.sock is present ... failed! >>>>>> . ok > >>>>> Hmm. I'm on Debian (well, Ubuntu), too, and I don't even have the >>>>> directory /var/run/fail2ban, no less the file >>>>> /var/run/fail2ban/fail2ban.sock. On another similarly configured server, >>>>> that directory and that file exist. Well, at least we know that not to >>>>> be the problem! > >>>> ah.. right -- we all (debian and ubuntu) decided to go for /run, I >>>> forgotten about that, so indeed you might be cherishing the fruits of >>>> such changes... Myself I have not run into this since on Debian systems we do >>>> still maintain compatibility with a symlink... so as a quick workaround you >>>> could try doing > >>> You lost me there. ;) > >> we are even then! ;-) > >>> To be clear, I do indeed have the directory /var/run. It just doesn't >>> contain a "fail2ban" sub-directory (it contains plenty of other items). >>> I'm on an older OS, Ubuntu 10.04 (you declared it an "elderly beastie", >>> quite aptly), and as such, shouldn't be affected by Debian/Ubuntu >>> transitioning from /var/run to /run. > >> ah -- then I have no clue where /var/run/fail2ban is gone since since >> 0.8.2 init script creates it: > >> $> git lg debian/fail2ban.init >> * 86ae7d2 - debian/fail2ban.init: Should-(start|stop): iptables-persistent (Closes: #598109), ferm (Closes: #604843) (7 months ago) [Yaroslav Halchenko] >> * 7154b3a - Since we are exiting with non-0 -- all msgs should be failures (1 year ago) [Yaroslav Halchenko] >> * 5cac32e - Making sure that the status command gives the correct exit code when fail2ban is not running. (1 year ago) [Glenn Aaldering] >> * de502cf - NF: run as different user (disabled by default) (1 year, 1 month ago) [Zbigniew Jędrzejewski-Szmek] >> * d1b9e71 - Adding arno-iptables-firewall (no deprecation of ipmasq per Joey Hess mentioning, which still could be used on lenny systems) (2 years, 9 months ago) [Yaroslav Halchenko] >> * 32281ed - BF: Assure that /var/run/fail2ban exists upon start (LP: #222804, #223706) (4 years, 10 months ago) [Yaroslav Halchenko] >> ... >> $> git describe --contains 32281ed >> debian/0.8.2-3~1^2 > >> so I would look for someone who destroyed it for you! ;) > >>> As I mentioned, I have another nearly identically-configured server, >>> with fail2ban 0.8.6 (the latest version available from the 10.04 repos), >>> and the /var/run/fail2ban directory, and socket file within it, do exist >>> there. > >> as they should! ;-) good > >>> Also, this installation of fail2ban-0.8.8 has functioned without issue >>> since you created the Ubuntu 10.04 backport .deb package for me several >>> months ago. I thought we were "out of the woods", so to speak. > >> succesful start means that /var/run/fail2ban was there!;-) > >>> But, alas, this seems to be the same problem that I described back in >>> December, 2012, on this list ( >>> http://sourceforge.net/mailarchive/forum.php?thread_name=50BE77A2.4060300%40indietorrent.org&forum_name=fail2ban-users >>> ). > >> sorry -- my memory span getting shorter and shorter with every email I >> read ;-) > >>> Could not start server. Maybe an old socket file is still present. Try >>> to remove /var/run/fail2ban/fail2ban.sock. If you used fail2ban-client >>> to start the server, adding the -x option will do it". > >>> What's taking so long? > >> hope that server would come up -- look at __waitOnServer function in >> -client and its comment on 30 sec there ;) > >>> I do find it a bit odd that fail2ban doesn't log its start-up sequence, >>> in detail, especially at maximum verbosity. It seems that this is not >>> simply a matter of increasing the log-level; I had it at 3, and just >>> changed it to 4, but absolutely nothing is written to >>> /var/log/fail2ban.log. > >> yeah -- that is how it was designed... -client dumps stuff only to stdout, not >> to the .log file... not sure if it would function just fine with two processes >> (-client and -server) trying to dump into the same file > >>> If the client executable can spit-out an error >>> message, one would think it capable of writing other (more useful) >>> information to the log-file, too. > >>> I haven't changed anything recently, either. In /etc/fail2ban/fail2ban.conf: > >>> logtarget = /var/log/fail2ban.log >>> socket = /var/run/fail2ban/fail2ban.sock > >>> Those look correct, and as I said, shouldn't have changed any time recently. > >>> What else might we check? > >> check on who killed /var/run/fail2ban ;-) >> meanwhile just mkdir /var/run/fail2ban/ and see if that helps (it should!) > >>> I'm still a bit hazy on the issue here, or if that was just a >>> miscommunication. As expected, this command yields the following on the >>> affected system: "ln: creating symbolic link `/var/run': File exists". > >> right -- my wild guess was wrong due to misinterpretation of your email... just > >> mkdir -p /var/run/fail2ban > >> and be happy again ;-) |