sshguard-users — User questions and answers, bugs, and general support

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Felix S. <fel...@os...>]

Am 27.08.20 um 18:52 schrieb Kevin Zheng:
>> Right now SSHguard log output about blocked IP addresses is delayed
>> by ~4-7 minutes.
> 
> What do you mean by this? Is it that sshg-blocker warns about the
> attacker 4-7 minutes after the attack has begun, or is it simply that
> attacks are not blocked until 4-7 minutes later?

I opened "journalctl -f -u sshguard" and "watch systemctl status sshguard" in
a a terminal. The times shown in these log outputs lagged ~4-7 minutes while
they are near real time otherwise. (And there was a continuous stream of new
login attempts
so there should have been a many messages about new blocks.)

The firewalld ipset contained ~600 ip addresses at some point but after ~16
hours the flooding eventually stopped so I can not check all the details anymore.

However:
> If your `top` or `ps` shows wait states, could you check if sshg-blocker
> is running, idle, or being blocked something by a pipe write?

"systemctl status sshguard" shows also child processes and what I could see is
that there was always a call to "firewall-cmd" visible. Also my manual calls
to "firewall-cmd" in a separate terminal took pretty long (a few seconds per
invocation). Usually these commands are pretty quick.

I guess I should try to create a test scenario where I call add random IP
addresses via firewall-cmd and check if I also see high CPU load. Ideally I'd
see much a lower CPU load - though I'm a bit swamped currently so it'll take a
few days.

Felix

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Christopher E. <ce...@lc...>]

On 27.08.20 19:23, Kevin Zheng wrote:
> I wonder how firewalld talks to its underlying backends? Could that be
> the culprit? SSHGuard does install iptables and nft backends, too.

nftables has python bindings for libnftables, and firewalld uses those.
iptables & ipset are accessed via their commandline utilities, I think.

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Kevin Z. <kev...@gm...>]

On 8/27/20 10:14 AM, Christopher Engelhard wrote:
> firewall-cmd itself talks to firewalld via the DBus interface, so one
> could maybe save some time by using DBus directly, if firewall-cmd is in
> fact the culprit.
> 
> The direct interface bypasses firewalld and sends stuff directly to the
> underlying firewall backend, which might be a problem because that
> backend could be iptables or nftables or whatever else firewalld supports.

I see, firewalld is already one layer of indirection.

I wonder how firewalld talks to its underlying backends? Could that be
the culprit? SSHGuard does install iptables and nft backends, too.

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Christopher E. <ce...@lc...>]

On 27.08.20 18:52, Kevin Zheng wrote:
> I suspect, without any measurement to back my suspicion, that the
> slowness comes from trying to invoke a separate firewall-cmd process so
> many times. Are there other ways to talk to firewalld without spinning
> up a process? I don't use firewalld, but some searching shows that
> there's a D-Bus interface and a "direct" interface. How does the
> firewalld GUI talk to firewalld? Through firewall-cmd or one of the
> interfaces I mentioned?

firewall-cmd itself talks to firewalld via the DBus interface, so one
could maybe save some time by using DBus directly, if firewall-cmd is in
fact the culprit.

The direct interface bypasses firewalld and sends stuff directly to the
underlying firewall backend, which might be a problem because that
backend could be iptables or nftables or whatever else firewalld supports.

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Kevin Z. <kev...@gm...>]

It's been a while since we've heard of an attack like this on the
SSHGuard mailing list.

> Right now SSHguard log output about blocked IP addresses is delayed
> by ~4-7 minutes.

What do you mean by this? Is it that sshg-blocker warns about the
attacker 4-7 minutes after the attack has begun, or is it simply that
attacks are not blocked until 4-7 minutes later?

If your `top` or `ps` shows wait states, could you check if sshg-blocker
is running, idle, or being blocked something by a pipe write?

If it is blocked by sshg-fw, then this suggests what you expect is true,
that sshg-fw is inefficient.

>> I think SSHguard uses firewalld's API inefficiently as it seems to add/remove
>> only a single IP per CLI call. I suspect this leads to high CPU usage by
>> firewalld when SSHguard needs to block many addresses.
>>
>> firewalld also offers options to add/remove many items at once. Do you think
>> SSHguard could use these options?
> 
> One could maybe modify the loop in sshg-fw.in [1] to collect the
> addresses etc. in a bash array:
> 
> args=( addr1 addrtype1 cidr1 addr2 addrtype2 cidr2 ...)
> 
> and pass that to the fw_block()-etc functions if the previous line was
> read more than a given interval ago, like a second or so:

Like discussed here, one approach could be to collect multiple IP's and
run firewall-cmd once per second.

I also want to mention that sshg-fw is just an ordinary program; you can
write a drop-in substitute, not necessarily in Bourne shell.

I suspect, without any measurement to back my suspicion, that the
slowness comes from trying to invoke a separate firewall-cmd process so
many times. Are there other ways to talk to firewalld without spinning
up a process? I don't use firewalld, but some searching shows that
there's a D-Bus interface and a "direct" interface. How does the
firewalld GUI talk to firewalld? Through firewall-cmd or one of the
interfaces I mentioned?

Unfortunately, I don't have a Fedora installation around and won't be
able to test. But this is something we should keep in mind for all the
other firewall backends.

(I use pf, and sshg-fw-pf does the expensive thing of spinning up a
pfctl process for each address that it blocks. It would be more
efficient if I rewrote it using the ioctl(2) interface to talk directly
to the /dev/pf device node.)

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Christopher E. <ce...@lc...>]

On 27.08.20 10:21, Christopher Engelhard wrote:
> One could maybe modify the loop in sshg-fw.in [1] to collect the
> addresses etc. in a bash array [...] and pass that to the fw_block()-etc functions

... or even simpler, collect in a string and shift through them in the
receiving functions.

Christopher

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Christopher E. <ce...@lc...>]

Hi,

On 27.08.20 09:14, Felix Schwarz wrote:
> Hi,
> 
> short version:
> I think SSHguard uses firewalld's API inefficiently as it seems to add/remove
> only a single IP per CLI call. I suspect this leads to high CPU usage by
> firewalld when SSHguard needs to block many addresses.
> 
> firewalld also offers options to add/remove many items at once. Do you think
> SSHguard could use these options?

One could maybe modify the loop in sshg-fw.in [1] to collect the
addresses etc. in a bash array:

args=( addr1 addrtype1 cidr1 addr2 addrtype2 cidr2 ...)

and pass that to the fw_block()-etc functions if the previous line was
read more than a given interval ago, like a second or so:

<add vars to argarray>
if [[ $(( $CURRENTTIME - $LASTTIME )) -gt 1 ]]; then
  fw_block <argarray>
  <clear argarray>
fi
LASTTIME=$CURRENTTIME

The backends would then be modified to loop over the arg array instead
of using the parameters directly:

arg=($@)
for ((i=0; i < $#; i+=3)); do
  addr="${arg[i]}"
  addrtype="${arg[i+1]}"
  cidr="${arg[i+2]}"

  <do stuff>
done

If the backend can not do multiple addresses, <do stuff> would be the
same as before, but if it can, it could use the loop to assemble the
actual command to run and then only trigger the backend once.

> Even though the server is probed for more than 10 hours now SSHguard still
> sees new IP addresses so I don't dare to hope that the attacker will be
> running out of new IP addresses soon.

Ouch.


Christopher

[1] https://bitbucket.org/sshguard/sshguard/src/master/src/fw/sshg-fw.in

[SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Felix S. <fel...@os...>]

Hi,

short version:
I think SSHguard uses firewalld's API inefficiently as it seems to add/remove
only a single IP per CLI call. I suspect this leads to high CPU usage by
firewalld when SSHguard needs to block many addresses.

firewalld also offers options to add/remove many items at once. Do you think
SSHguard could use these options?

Felix


background:
I'm a satisfied user of SSHguard. So far it really works great and I found it
easy to set up with CentOS and Fedora.

This morning however one of my servers is targeted by some kind of distributed
brute force "attack". I see roughly 10-20 SSH login attempts per second from
various IP addresses (I guess a few hundred but well below 1000).

SSHguard is happily blocking IPs and still working as intended but the CPU of
that poor little server is maxed out at 100% (it is a very tiny instance).
When using "top" I see that firewalld needs a lot of CPU over longer periods
of time.

I can see that SSHguard uses
"firewall-cmd [...] --add-entry=.../32" and seems to add/remove only a single
IP at a time.

Based on experience with other software I suspect that inefficient use of the
firewalld API might contribute to this high CPU usage. Right now SSHguard log
output about blocked IP addresses is delayed by ~4-7 minutes.

Even though the server is probed for more than 10 hours now SSHguard still
sees new IP addresses so I don't dare to hope that the attacker will be
running out of new IP addresses soon.

Re: [SSHGuard-users] Patches, pull requests welcome for SSHGuard website

[Kevin B. <kev...@gm...>]

On 2020/08/18 01:51, Kevin Zheng wrote:
> 
> I'm inviting anyone interested to submit issues, patches, and pull
> requests for the SSHGuard website. The sources for the current version
> of the website that's online is here:
> 
> https://bitbucket.org/sshguard/website-static/
> 

1)
If there was a top-level README in the repo, that would be displayed
automatically when browing the top-level of the repo, then it would
be able to display a link to the live website.

I keep forgetting it is .net and not .org!

2)
On the front-page, I think

SSHGUARD GEARS

would be better written as

SSHGUARD FEATURES


Kevin

[SSHGuard-users] Patches, pull requests welcome for SSHGuard website

[Kevin Z. <kev...@gm...>]

Hi there,

I'm inviting anyone interested to submit issues, patches, and pull
requests for the SSHGuard website. The sources for the current version
of the website that's online is here:

https://bitbucket.org/sshguard/website-static/

You might be surprised to see M4 files in this repository. The reason
this is requires some background:

SSHGuard's website was once written in Django. One day the server
running it died, and since the Django was outdated it would have been a
lot of work to get it working on a new installation. I scraped the
themes and HTML and cobbled together a static version of the site as a
stopgap measure. The stopgap measure continues to exist to this day.

As a result, the website still looks approximately the same, and now
changes are made directly to the HTML that's M4'd to generate the
header. I am looking for a long term solution, probably my favorite new
static site generator, but it'll be some time before I can sit down and
port the themes.

For now, the in-progress RSS feed is written by hand (news.xml). I'll
probably de-link the existing news.html so I don't have to maintain two
copies of this. Patches and pull requests against the content on the
other pages are welcome.

Regards,
Kevin

Re: [SSHGuard-users] 2.4.1 announcment?

[Kevin Z. <kev...@gm...>]

On 8/16/20 3:14 PM, Jungle Boogie wrote:
> Just curious, do you plan on updating the news.html page?
> As lbutlr pointed out, the latest release is from July 9 for 2.2.0,
> which was actually from JUly 2018.

Yes, and to resurrect the RSS feed, eventually.

Re: [SSHGuard-users] 2.4.1 announcment?

[Jungle B. <jun...@gm...>]

Hi Kevin,

On 8/16/2020 10:35 AM, Kevin Zheng wrote:
> On 8/16/20 7:57 AM, @lbutlr wrote:
>> I saw there was a new version of sshguard so, wondering what the new
>> features were, I went to https://www.sshguard.net/news.html which
>> told me 2.2 was just released in "July" so, looks like that site is
>> not maintained. I then went and looked around the SF pages, but there
>> is no announcement there nor a changes list that I could find, just
>> the tgz of the new version.
>>
>> I didn't see anything on this list and the sshguard-maintainers
>> archive shows the last post was almost a year ago.
>>
>> I mean, it's not really important, and I assume there is a changeling
>> inside the tgz file, but I am unlikely to see that when I update via
>> a package manager.
>>
>> Short of manually downloading the package and such, where would I
>> find the changelog?
> 
> This one is totally my fault: I cut the release but didn't get around to
> making the release announcement.

Just curious, do you plan on updating the news.html page?
As lbutlr pointed out, the latest release is from July 9 for 2.2.0, 
which was actually from JUly 2018.

At any rate, thank you for another release and for continuing to 
maintain sshguard.

> 
> I'll do this now.

[SSHGuard-users] SSHGuard 2.4.1 release announcement

[Kevin Z. <kev...@gm...>]

Dear SSHGuard users,

SSHGuard 2.4.1 is now available.

(The release was cut July 31st; this release announcement is late.)

**Added**

- Recognize RFC 5424 syslog banners
- Recognize busybox syslog -S banners
- Recognize rsyslog banners
- Recognize web services TYPO3, Contao, and Joomla
- Update signatures for Dovecot
- Update signatures for OpenSSH

**Changed**

- Whitelist entire 127.0.0.0/8 and ::1 block
- Whitelist file allows inline comments

**Fixed**

- Fix FILES and LOGREADER configuration file options

Regards,
Kevin

Re: [SSHGuard-users] 2.4.1 announcment?

[Kevin Z. <kev...@gm...>]

On 8/16/20 7:57 AM, @lbutlr wrote:
> I saw there was a new version of sshguard so, wondering what the new
> features were, I went to https://www.sshguard.net/news.html which
> told me 2.2 was just released in "July" so, looks like that site is
> not maintained. I then went and looked around the SF pages, but there
> is no announcement there nor a changes list that I could find, just
> the tgz of the new version.
> 
> I didn't see anything on this list and the sshguard-maintainers
> archive shows the last post was almost a year ago.
> 
> I mean, it's not really important, and I assume there is a changeling
> inside the tgz file, but I am unlikely to see that when I update via
> a package manager.
> 
> Short of manually downloading the package and such, where would I
> find the changelog?

This one is totally my fault: I cut the release but didn't get around to
making the release announcement.

I'll do this now.

[SSHGuard-users] 2.4.1 announcment?

[@lbutlr <kr...@kr...>]

I saw there was a new version of sshguard so, wondering what the new features were, I went to https://www.sshguard.net/news.html which told me 2.2 was just released in "July" so, looks like that site is not maintained. I then went and looked around the SF pages, but there is no announcement there nor a changes list that I could find, just the tgz of the new version.

I didn't see anything on this list and the sshguard-maintainers archive shows the last post was almost a year ago.

I mean, it's not really important, and I assume there is a changeling inside the tgz file, but I am unlikely to see that when I update via a package manager.

Short of manually downloading the package and such, where would I find the changelog?




-- 
"Are you pondering what I'm pondering?"
Yeah, but I thought Madonna already had a steady bloke!"

[SSHGuard-users] Blacklist - Double entries

[Jos C. <ssh...@cl...>]

Hello team,

Just saw that my blacklist contains several identical ip numbers:

1594465182|260|4|193.35.51.13
1594482213|260|4|5.188.206.194
1594545561|260|4|46.38.145.5 *
1594545565|260|4|46.38.145.4
1594545807|260|4|46.38.145.5  *
1594545891|260|4|46.38.145.254
1594560837|260|4|185.143.73.33
/1594636209|260|4|78.128.113.114 //
//1594636209|260|4|78.128.113.114//
/1594765288|100|4|211.118.42.219
1594991575|260|4|46.38.145.253
/1595278864|260|4|45.95.168.77//
//1595283662|260|4|45.95.168.77//
//1595283662|260|4|45.95.168.77//
//1595408787|260|4|45.148.10.98 //
//1595408787|260|4|45.148.10.98//
//1595408787|260|4|45.148.10.98/
/1595801798|260|4|185.253.217.78//
//1595801798|260|4|185.253.217.78//
//1595805119|260|4|185.253.217.78//
/1596154612|260|4|91.191.209.188
1596398311|260|4|193.56.28.20
1596880381|210|4|185.210.217.116

Can you tell what I should check in order to solve this?
Thanks, Jos

-- With both feed on the ground you will never make a step forward

Re: [SSHGuard-users] firewall doesn't seem to be updating

[@lbutlr <kr...@kr...>]

On 15 Jul 2020, at 01:58, Kevin Zheng <kev...@gm...> wrote:
> On 7/14/20 2:57 PM, @lbutlr wrote:
>> auth.log contains entries like:
>> 
>> sshd[81715] error: PAM: Authentication error for root from 116.98.172.159
>> sshd[81715] Connection closed by authenticating user root 116.98.172.159 port 49832 [preauth]
> 
> Are these the exact lines from your auth.log? SSHGuard is not detecting
> these because it does not recognize your log header. Is this syslogd?

Other than the time stamp and server name, yes. I thought that information was not relevant.

> This is not recognized by 2.4.0 because OpenSSH recently changed their
> log format. The parser in Git does recognize this as an attack. I will
> cut a release soon.
> 
>> So, I try to manually trigger it by manually appending the above bloglines back to auth.log from another session:
>> 
>> # which sshguard
>> /usr/local/sbin/sshguard
>> # env SSHGUARD_DEBUG=foo /usr/local/sbin/sshguard 
>> /usr/local/sbin/sshguard: cannot create : No such file or directory
> 
> The version in FreeBSD ports has a non-standard addition that checks for
> the existence of the PID file. I think it's just saying it can't write
> to the PID file, because $PID_FILE is empty? I'll investigate further.
> 
>> sshguard 94135 - - whitelist: add '***' as plain IPv4.
>> sshguard 94135 - - whitelist: add plain IPv4 ***.
>> sshguard 94135 - - whitelist: add IPv4 block: ***.
>> sshguard 94135 - - whitelist: add IPv4 block: ***.
>> sshguard 94135 - - blacklist: blocking 4832 addresses
>> sshguard 94135 - - whitelist: add '127.0.0.1' as plain IPv4.
>> sshguard 94135 - - whitelist: add plain IPv4 127.0.0.1.
>> sshguard 94135 - - Now monitoring attacks.
> 
> SSHGuard does not accept attacks from standard input.

No, I added the log lines to auth.log in another session (this works fine on a different machine, at least in terms of when I logged in it showed that the IP used was in the whitelist.

> There are several things you can do to troubleshoot SSHGuard, but an
> easy starter is to syscall trace it:
> 
> $ truss -f /usr/local/sbin/sshguard

I'll give that a shot.

What are some other steps to troubleshoot.

-- 
'They say that whoever pays the piper calls the tune.' 'But,
gentlemen,' said Mr Saveloy, 'whoever holds a knife to the
piper's throat writes the symphony.' --Interesting Times

Re: [SSHGuard-users] firewall doesn't seem to be updating

[Kevin Z. <kev...@gm...>]

Hi there,

So, a few things:

On 7/14/20 2:57 PM, @lbutlr wrote:
> auth.log contains entries like:
> 
> sshd[81715] error: PAM: Authentication error for root from 116.98.172.159
> sshd[81715] Connection closed by authenticating user root 116.98.172.159 port 49832 [preauth]

Are these the exact lines from your auth.log? SSHGuard is not detecting
these because it does not recognize your log header. Is this syslogd?

You can test the parser by itself (on FreeBSD) using:

$ /usr/local/libexec/sshg-parser

It reads input line by line, does nothing if it does not detect an
attack, and prints something if it does.

You have an example attack below:

Jul 14 14:07:05 mail.covisp.net sshd[81715] error: PAM: Authentication
error for root from 116.98.172.159

This is not recognized by 2.4.0 because OpenSSH recently changed their
log format. The parser in Git does recognize this as an attack. I will
cut a release soon.

> So, I try to manually trigger it by manually appending the above bloglines back to auth.log from another session:
> 
>  # which sshguard
> /usr/local/sbin/sshguard
>  # env SSHGUARD_DEBUG=foo /usr/local/sbin/sshguard 
> /usr/local/sbin/sshguard: cannot create : No such file or directory

The version in FreeBSD ports has a non-standard addition that checks for
the existence of the PID file. I think it's just saying it can't write
to the PID file, because $PID_FILE is empty? I'll investigate further.

> sshguard 94135 - - whitelist: add '***' as plain IPv4.
> sshguard 94135 - - whitelist: add plain IPv4 ***.
> sshguard 94135 - - whitelist: add IPv4 block: ***.
> sshguard 94135 - - whitelist: add IPv4 block: ***.
> sshguard 94135 - - blacklist: blocking 4832 addresses
> sshguard 94135 - - whitelist: add '127.0.0.1' as plain IPv4.
> sshguard 94135 - - whitelist: add plain IPv4 127.0.0.1.
> sshguard 94135 - - Now monitoring attacks.

SSHGuard does not accept attacks from standard input.

So, seems like some things are wrong; some things that should be fixed
when we cut a release.

There are several things you can do to troubleshoot SSHGuard, but an
easy starter is to syscall trace it:

$ truss -f /usr/local/sbin/sshguard

If you could trace it for 10 seconds or so and attach the output, I
might be able to see what's going on.

Good luck,
Kevin

-- 
Kevin Zheng
kev...@gm... | ke...@be...
XMPP: ke...@ee...

[SSHGuard-users] firewall doesn't seem to be updating

[@lbutlr <kr...@kr...>]

sshguard-2.4.0_2,1 on FreeBSD 12.1

If I check my sshguard table, it returns no entries

 # pfctl -t sshguard -T show
 # 

auth.log contains entries like:

sshd[81715] error: PAM: Authentication error for root from 116.98.172.159
sshd[81715] Connection closed by authenticating user root 116.98.172.159 port 49832 [preauth]

I can manually add an IP to the sshguard table, but I cannot see any evidence that sshguard is doing anything and "sshg" appears in no log files in /var/log/.

 # pfctl -t badguys -T show | grep 116.98.172.159  
 # pfctl -t sshguard -T add 116.98.172.159
1/1 addresses added.
 # pfctl -t sshguard -T show
   116.98.172.159

So, PF appears to be fine.

So, I try to manually trigger it by manually appending the above bloglines back to auth.log from another session:

 # which sshguard
/usr/local/sbin/sshguard
 # env SSHGUARD_DEBUG=foo /usr/local/sbin/sshguard 
/usr/local/sbin/sshguard: cannot create : No such file or directory
sshguard 94135 - - whitelist: add '***' as plain IPv4.
sshguard 94135 - - whitelist: add plain IPv4 ***.
sshguard 94135 - - whitelist: add IPv4 block: ***.
sshguard 94135 - - whitelist: add IPv4 block: ***.
sshguard 94135 - - blacklist: blocking 4832 addresses
sshguard 94135 - - whitelist: add '127.0.0.1' as plain IPv4.
sshguard 94135 - - whitelist: add plain IPv4 127.0.0.1.
sshguard 94135 - - Now monitoring attacks.
Jul 14 14:07:05 mail.covisp.net sshd[81715] error: PAM: Authentication error for root from 116.98.172.159
Jul 14 14:07:08 mail.covisp.net sshd[81715] Connection closed by authenticating user root 116.98.172.159 port 49832 [preauth]

Nothing. ??

ps output
root       843   0.0  0.1   4884   1928  -  Is    7Jun20      0:00.00 /bin/sh /usr/local/sbin/sshguard -b /usr/local/etc/sshguard.blacklist -w /usr/local/etc/sshguard.whitelist -b 120:/var/db/sshguard/blacklist.db -i /var/run/sshguard.pid
root       848   0.0  0.1   5560   2692  -  IC    7Jun20      0:00.15 /usr/local/libexec/sshg-blocker -a 30 -b 120:/var/db/sshguard/blacklist.db -p 1200 -s 18000 -w /usr/local/etc/sshguard.whitelist

/usr/local/etc/sshguard.conf:
BACKEND="/usr/local/libexec/sshg-fw-pf"
FILES="/var/log/auth.log /var/log/mail.log /var/log/debug.log /var/log/xferlog"
THRESHOLD=30
BLOCK_TIME=1200
DETECTION_TIME=18000
BLACKLIST_FILE=30:/var/db/sshguard/blacklist.db
WHITELIST_FILE=/usr/local/etc/sshguard.whitelist
#EOF

/etc/pf.conf:
ext=em0

table <goodguys> { **someIPs** } persist
table <badguys> { } persist
table <sshguard> persist

block in quick on $ext from <sshguard> label "sshguardblock"
block in quick on $ext from <badguys> label "COUNTRY BLOCKS"

pass in quick on $ext proto tcp from <goodguys> to ($ext) port ssh keep state
pass in on $ext proto tcp from any to ($ext) port ssh keep state (max-src-conn 5, max-src-conn-rate 4/300, overload <badguys> flush global)
#EOF

[SSHGuard-users] [Proposal] Handle last octet range in Whitelists

[Kevin B. <kev...@gm...>]

Hi there,

in an ideal world, groups of related hosts would always be
assigned IP addresses that saw all of the group lying within
"nice" sets of CIDR boundary ranges.

Well, it's not an ideal world, and a lot people still aren't
falling asleep at night thinking about CIDR boundaries.

With that in mind, I was hoping to use the SSHGuard codebase
so as to handle a set of IPv4 addresses that some people only
ever think of, in terms of

  aaa.bbb.ccc.[ddd.eee]

and found it didn't handle them, however, I thought it could.

It can.

The attached patch adds a block of code to

   src/blocker/sshguard_whitelist.c

that does allow people, who have better things to fall asleep
at night thinking about to, to have entries in whitelist files
that adhere to that non-CIDR format.

I tried it against various instances of aaa.bbb.ccc.[143-220]
and, whilst I wouldn't recommend anyone should do that, I did
see the expected

sshguard[12345]: whitelist: add plain IPv4 aaa.bbb.cc.143.
...
sshguard[12345]: whitelist: add plain IPv4 146.118.38.220.

set of entries being added.


It's yours if you want it: it may need some extra checking.

Note that this is IPv4 only: after all, who falls asleep at
night thinking about IPv6 addresses?

Kevin

Re: [SSHGuard-users] [Proposal] Handle End-of-line comments in Whitelists

[Kevin Z. <kev...@gm...>]

On 5/14/20 9:14 PM, Kevin Buckley wrote:
> The various log messages already in SSHGuard go out via syslog, vis:
> 
> $ cat sshguard_log.h
> ...
> #define sshguard_log syslog
> $
> 
> which is a nod in the direction of SSHGuard being run as a daemon.
> 
> 
> In the kind of debug message that I was thinkng about when suggesting
> "adding a debug flag", I was thinking about the case where one runs
> the blocker from the command line and so might want to see messages
> directly on the console. (The typical run daemon in non-detached mode)

Yes, you read that part correctly.

However, the blocker also passes the SSHGUARD_DEBUG environment variable
to init_log, which calls openlog() with LOG_PERROR. If your syslog
supports it, then:

LOG_PERROR    Write the message to standard error output as well to the
              system log.

I believe running sshg-blocker directly from the terminal does correctly
log to standard error.

-- 
Kevin Zheng
kev...@gm... | ke...@be...
XMPP: ke...@ee...

Re: [SSHGuard-users] [Proposal] Handle End-of-line comments in Whitelists

[Kevin B. <kev...@gm...>]

On 2020/05/15 02:36, Kevin Zheng wrote:
> 
> Currently, LOG_DEBUG lines are only displayed when SSHGUARD_DEBUG is set
> in the environment, see sshguard(8).

Another observation on that (now that I'm aware of it, that is!).

The various log messages already in SSHGuard go out via syslog, vis:

$ cat sshguard_log.h
...
#define sshguard_log syslog
$

which is a nod in the direction of SSHGuard being run as a daemon.


In the kind of debug message that I was thinkng about when suggesting
"adding a debug flag", I was thinking about the case where one runs
the blocker from the command line and so might want to see messages
directly on the console. (The typical run daemon in non-detached mode)

I had found myself sprinkling a few "fprintf(stderr, ...)" across
the codebase and so was more thinking of the "debug flag" as a way
to wrap those.

Hoping that serves to inform the discussion a little more,
Kevin

Re: [SSHGuard-users] [Proposal] Handle End-of-line comments in Whitelists

[Kevin B. <kev...@gm...>]

On 2020/05/15 02:36, Kevin Zheng wrote:
>
> Do you mean to say that you're running in an environment where setting
> environment variables is not possible?

No.

> Currently, LOG_DEBUG lines are only displayed when SSHGUARD_DEBUG is set
> in the environment, see sshguard(8).

So that's the missing piece for that then.

>> * What we originally found, after adding EOL comments to a whitelist file
>> was that SSHGuard "appeared" to work OK, except that it wasnt, and it was
>> only when we added a comment along the lines of:
>> 
>> 
>> nnn.nnn.nnn.nnn   # Host at some org but covered by range
>> nnn.nnn.nnn.0/30 above
>> 
>> that we realised that the whitelist parser (whitelist_add() function)
>> was seeing the "/" in the comment as indication of a range and then
>> "not quite getting there" when parsing the full line.
> 
> Sounds like what we should really have is a better-written parser.
> Perhaps we should write a better IP list parser ...


In all fairness, if one is happy to ONLY have comments on lines of
their own, so above or below individual entries, then the existing
parser is fine.

Furthermore, none of the example files show End-of-line comments.

However, if you do not wish to have your IP addresses and ranges
surrounded by comments, and it could make the file a bit unwiedly,
and/or the IP addresses and ranges a little harder to discern, then
the existing parser doesn't always do the right thing.

Making the latter work is what the addition of the strsep() proposal
addresses, not any percieved issue with the existing parser when
used against whitelists that dont have EOL comments.


> ... and combine the
> whitelist and blacklist parsers to support comments?

The ability to annotate a blacklist with EOL comments might add
some operational benefit, as well as providing consistency in
the way the black and white lists are handled, which then reduces
any potential for confusion and errors in the input files.

Kevin

Re: [SSHGuard-users] Asterisk support

[David M. <dm...@gm...>]

Hi Kevin,

On Thu, May 14, 2020 at 1:19 PM Kevin Zheng <kev...@gm...> wrote:
>
> Just curious -- why do you have a mandate to replace fail2ban?
>

The short version is 'it does not work reliably' - there are multiple
reasons for this, some of which are unknown to me. Asterisk patterns
in fail2ban have changed drastically between versions, python version
differences impact functionality, etc. - many moving parts to
orchestrate. I appreciate the SSHGuard preference for a small, fast,
robust application without so many external dependencies.

>
> Steps 3-6 could use some fleshing-out; I'm also available to help.
>

Thank you very much!

> Let me know how you'd like to try to proceed.

I'm going to review the documentation and familiarize myself with the
code. I've also been pointed to the CSF/LFD project, which is another
possible candidate for replacing fail2ban in my environment. I
appreciate your response and I will follow up with you once I've had
time to digest my options.

David

Re: [SSHGuard-users] [Proposal] Handle End-of-line comments in Whitelists

[Kevin Z. <kev...@gm...>]

On 5/13/20 6:45 PM, Kevin Buckley wrote:
> Other than neededing to bump up the log level to LOG_ERR to see this
> working on my test system, that would clearly be a useful addition,
> however, our testing would suggest that, when the whitelist parser
> "gets it wrong", then it really gets it wrong, and just segfaults*.
> 
> My "extra flag for debuging" idea would have just seen everything
> dumped into the log stream by adding in some
> 
> if( debug_flag_used) then
>   sshguard_log(LOG_ERR, "stuff")
> fi
> 
> guards, on the understanding that, if you invoked SSHGuard with the
> debug flag, you would want to see some output without needing to alter
> anything else on the system outside of SSHGuard.

Do you mean to say that you're running in an environment where setting
environment variables is not possible?

Currently, LOG_DEBUG lines are only displayed when SSHGUARD_DEBUG is set
in the environment, see sshguard(8).

> * What we originally found, after adding EOL comments to a whitelist file
> was that SSHGuard "appeared" to work OK, except that it wasnt, and it was
> only when we added a comment along the lines of:
> 
> 
> nnn.nnn.nnn.nnn   # Host at some org but covered by range
> nnn.nnn.nnn.0/30 above
> 
> that we realised that the whitelist parser (whitelist_add() function)
> was seeing the "/" in the comment as indication of a range and then
> "not quite getting there" when parsing the full line.

Sounds like what we should really have is a better-written parser.
Perhaps we should write a better IP list parser and combine the
whitelist and blacklist parsers to support comments?

Regards,
Kevin

-- 
Kevin Zheng
kev...@gm... | ke...@be...
XMPP: ke...@ee...