Re: [Sshguard-users] SSHguard for HTTP

[Peter B. <be...@an...>]

On Thu, 11 Jun 2009, Peter Beckman wrote:

> Would this work?
>
>     tail -n0 -F httpd.log | grep ' 404 ' | sshguard -a 100 -s 60 -p 1200

  Unfortunately this doesn't work.  The problem, however, is not SSHguard,
  but pipes.  Once you run

     tail -n0 -F httpd.log | grep ' 404 '

  It outputs as expected to stdout.  However, when you add pipe number two,
  piping to sshguard, the output doesn't continue as tail processes.  I'm
  not sure if it gets buffered somewhere or what, but SOMETHING prevents
  the output you can see from grep to getting to sshguard.  Try it out:

     tail -n0 -F httpd.log | grep ' 200 ' | cat

  If you just do:

     cat httpd.log | grep ' 200 ' | cat

  Works just fine.  But there is something about tail that screws up
  multiple pipes.  Anyone know what's up here?  I tried installing gtail
  (didn't work), tried to figure out how to configure lighttpd to spit only
  404's to a certain local0 syslog facility so I could pipe it to lighttpd,
  I even googled "'tail -f' multiple pipes" and read a bunch of stuff.  I've
  looked for unbuffering functionality in grep, egrep, sed, tail, gtail and
  others.  Most solutions I did find simply worked around the issue of
  multiple pipes by combining commands into a single pipe after tail -F.
  People doing:

     tail -n0 -F httpd.log | grep 'foo' | grep -v 'bar'

  were told to use awk and a single pipe.

  So what's the deal?  Why does tail not play nice with multiple pipes?

  In theory, something like this would work like a charm:

     tail -n0 -F httpd.log | sed -n -E 's/^(.+?) .+ 404 .+$/\1 404 access denied/p' | sshguard -a 100 -s 60 -p 1200

  (if it only worked) to only get the 404's out of the log file, and then
  rewrite the log entry to meet sshguard's criteria for blocking.  The '-n'
  and trailing 'p' flag in s// allow me to NOT pipe non-replaced lines to
  sshguard, for efficiency.  But this doesn't work (tested with sshguard -d).

  If you can think of how to use SSHguard to block people who attempt brute
  force HTTP scans of 404 links and get around the multiple pipes issue, I'd
  love to hear it.  Lighttpd doesn't log 404 errors to the error log, and it
  doesn't seem to be able to only send 404 errors to a different file than
  the set access log file.  I'd still need to pipe access log entries sent
  to syslog through sed and then sshguard, which MIGHT work, but then I lose
  access logs, which are kinda important.  Plus I'm not sure what kind of
  overhead that might generate.

Beckman
---------------------------------------------------------------------------
Peter Beckman                                                  Internet Guy
be...@an...                                 http://www.angryox.com/
---------------------------------------------------------------------------