sshguard-users — User questions and answers, bugs, and general support

[SSHGuard-users] "Stale" and "Pardon" Times

[Jim S. <jse...@Li...>]

Hi All,

I've been doing a bit of work with sshguard, lately, which has caused
me to observe attack patterns more closely.

The result of doing so had me increasing the "stale" parameter (-s)
from the default of twenty minutes (set in /etc/default/sshguard) to
35 minutes.  (See below for why.)

Then it occurred to me the initial "pardon" value was 840 seconds (14
minutes), which meant the first two blockings of an offending IP
address would have no effect if their retry rate was out at the edge
of the stale value I set.

That led to me looking at this, in sshguard_options.c:

Perhaps there should be a subsequent test, after the options are all
processed, to make sure pardon > stale and issue a warning if not?
Perhaps also automatically bump pardon by, say, 120 seconds over
stale if that happens?

I increased sshguard's stale argument because the attack pattern was
multiple IPs, with repeated IPs retrying every 32-33 minutes.  The
attackers are becoming more devious.  (I have an idea for countering
that, but I need to give it further thought.)

Regards,
Jim
-- 
Note: My mail server employs *very* aggressive anti-spam
filtering.  If you reply to this email and your email is
rejected, please accept my apologies and let me know via my
web form at <http://jimsun.LinxNet.com/contact/scform.php>.

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[lists <li...@la...>]

It won't help because I still want to block hosting services, servers, VPS, etc. I need the ability to have a large static block list. 

Incidentally if the developers want something else to block, tor exit nodes are documented. I had a script to block them too but dropped it due to the same firewalld issue. There is an API for the exit nodes. 

Nothing I serve needs to have the viewers IP hidden. Every time I read about some hacking I look at the IP and I usually have it blocked. 





	  Original Message  	


From: joh...@as...
Sent: January 19, 2022 9:31 PM
To: ssh...@li...
Subject: Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once


I set up a cron job to zero out the blacklist file every week. When the
system reboots (usually for kernel update), it can rebuild the smaller
file in a timely manner. Since the IPs are ever-changing anyway, the
current "bad" ones get re-added to the list quickly. To keep the file
size even smaller, I also have the block subnet size set to 24 (which
I'm sure is not a preferable option for everyone).

It's not elegant, but it works for me.

Chris

On 1/19/22 19:51, lists wrote:
> I dropped using sshguard specifically for the load cause by adding IPs to firewalld. It was less of a load to allow failed ssh attempts than to block them. I use PKI so I think the odds of a breach are small.  More likely than not some software vulnerability will lead to a breach than someone hacking ssh.


_______________________________________________
sshguard-users mailing list
ssh...@li...
https://lists.sourceforge.net/lists/listinfo/sshguard-users

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Chris J. <joh...@as...>]

I set up a cron job to zero out the blacklist file every week. When the 
system reboots (usually for kernel update), it can rebuild the smaller 
file in a timely manner. Since the IPs are ever-changing anyway, the 
current "bad" ones get re-added to the list quickly. To keep the file 
size even smaller, I also have the block subnet size set to 24 (which 
I'm sure is not a preferable option for everyone).

It's not elegant, but it works for me.

Chris

On 1/19/22 19:51, lists wrote:
> I dropped using sshguard specifically for the load cause by adding IPs to firewalld. It was less of a load to allow failed ssh attempts than to block them. I use PKI so I think the odds of a breach are small.  More likely than not some software vulnerability will lead to a breach than someone hacking ssh.

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[lists <li...@la...>]

I dropped using sshguard specifically for the load cause by adding IPs to firewalld. It was less of a load to allow failed ssh attempts than to block them. I use PKI so I think the odds of a breach are small.  More likely than not some software vulnerability will lead to a breach than someone hacking ssh. 

I realize sshguard also can protect your email. What I did is block all out of country use on all email ports other than 25 in addition to the servers list that I block. Totally impractical except if you have a personal web server. I see very little hacking on my email and I use anvil to throttle was little does occur. 

My situation is I am using a one CPU VPS.

Now to complete the story, I have a very large list of IP space that  I block. I assume firewalld processes the cidrs into some hash table that it efficiently searches. That is the overhead of a static IP blocking list is nil. It does use a fair amount of memory but that isn't an issue for the VPS.

At times my VPS would stall due to the CPU load. Probably at moments when sshguard was altering the block list and the VPS wasn't getting much CPU juice.

Sshguard is a fine program but firewalld doesn't handle dynamic blocking. 

Given the state of centos, I probably will have to change to a new disty in a year or so. I stay on the list since the traffic is light and I may learn something for the next time I install it. 

Blathering on a bit more, something like a two step firewall is what you need. Use the big list first then follow up with the small list. I have over 30k cidrs in the big list. I never had more than a few hundred at a time in the sshguard block list. 



  Original Message  


From: ssh...@li...
Sent: January 19, 2022 3:10 PM
To: ssh...@li...
Reply-to: rr...@ro...
Subject: Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once


On 1/10/2022 8:36 PM, Kevin Zheng wrote:
> On 1/9/22 9:22 PM, Kevin Buckley wrote:
>> 2h 00min 13916
>> 1h 45min 12116  15mins 1800    120.00/m   2.00/s
>> 1h 12min  8405  33mins 3711    112.45/m   1.87/s
>> 1h  8min  7936   4mins  469    117.25/m   1.95/s
>
> These numbers don't look good.
>
> I don't have a system handy where I can test firewall-cmd, but is
> there an interface (or another command) that lets you bulk-add address
> to an ipset without invoking firewall-cmd once per address?

firewall-cmd --ipset=ipset --add-entries-from-file=filename seems to be
possibly what you are looking for but that may in fact just be
programmed to invoke firewall-cmd one per address in the file as opposed
to a bulk load.  I would read carefully that documentation and run some
tests to see if it's faster for you and works for you.  A script to read
a line from a file you prepare and invoke the firewall-cmd command like
SSHGuard currently does as you are using it timed and then another
script to invoke the firewall-cmd command with the
--add-entries-from-file options times should give you benchmarks.

You also could investigate other BACKENDS as SSHGuard calls them and see
if others happen to be faster.

>
> I took a cursory look at what our "competitor" fail2ban does:
I don't think many would agree that fail2ban is a competitor of
SSHGuard.  At least to me they seem to have very different strategies
both of which could be integrated into the same system or used on a
network for different systems in a way where they are not competitors
per se but both used in different situations depending on their
strengths in solving particular problems.
>
> https://github.com/fail2ban/fail2ban/blob/80805cabfcf57dd0454d47d7f86d43c6738ce629/config/action.d/firewallcmd-ipset.conf
>
>
> Which, to summarize, seems to be:
>
> actionban = firewall-cmd --ipset=<ipmset> --add-entry=<ip>
>
> actionunban = firewall-cmd --ipset=<ipmset> --remove-entry=<ip>
>
> Which seems to be exactly what SSHGuard is doing.
>
> Anyone with more Linux firewall experience who could tell us if
> there's a faster way to add to a firewalld ipset? Is it time to teach
> SSHGuard how to use the dbus interface?
I think if you want to stick to the firewalld ipset option with SSHGuard
carefully reading their documentation and in particular for your
questions information about firewall-cmd options such as the one listing
the ability to add ip addresses from a file on their web site if you
have access to that would be a good idea.  Documentation about
firewall-cmd is available at the web site
https://firewalld.org/documentation/man-pages/firewall-cmd.html if you
don't have access to a machine to run something like 'man
firewall-cmd'.  As I said above though, I'm not sure it would be
faster.  It would seem it would but I haven't done any testing.
>
> Regards,
> Kevin
>
>
> _______________________________________________
> sshguard-users mailing list
> ssh...@li...
> https://lists.sourceforge.net/lists/listinfo/sshguard-users

--
This email has been checked for viruses by AVG.
https://www.avg.com



_______________________________________________
sshguard-users mailing list
ssh...@li...
https://lists.sourceforge.net/lists/listinfo/sshguard-users

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Kevin B. <kev...@gm...>]

On 2022/01/19 19:34, Christopher Engelhard wrote:
> 
> On 19.01.22 10:10, Kevin Buckley wrote:
>> That suggests that one could take the blacklist lines
>> and simply pre-populate the
>> 
>> sshguard4.xml
>> 
>> IPSet file, before starting SSHGuard, however I am not sure
>> what SSHGuard would do with existing entries if, on starting,
>> it finds that the sshguard4 IPSet exists, and already has
>> entries.
> 
> This would be another option, but seems rather brittle to me. a) it
> messes with firewalld-internal config files that might or might not stay
> in that location/format and b) actually getting firewalld to read those
> files means triggering a global reload which might or might not
> influence other things the user has been doing in the meantime.

Yes.

It didn't "feel quite right" to me either, but it does separate
all ongoing blocks, initiated by SSHGuard, from already known to
FirewallD drops.

Christpher also writes

> IMO there is no reason not to use the --permanent flag when setting the 
> entries, but that should be checked to be sure. Main difference is that 
> the banned IPs will survive a restart, while sshguards banlist doesn't?, 
> which might lead to IPs not being unbanned. But that is easily remedied 
> by letting SSHGuard wipe the sets on startup. It might already, I can check.


So here's my experience with that:

I start SSHGuard, against a FirewallD that has never seen SSHGUard,
with a blaklist containing 28793, and with the "--permanent" option
added to the SSHGuard's ads/remove block functions, as detailed
previously.

It took around SEVEN hours for SSHGuard to read the entries from the
blacklist and add them into FirewallD.

Jan 19 13:02:11 host systemd[1]: Started SSHGuard - blocks brute-force login at>
Jan 19 13:02:11 host sshguard[25082]: blacklist: blocking 28793 addresses
Jan 19 20:07:32 host sshguard[25082]: Now monitoring attacks.

As may have been already documented, each addition sees the exiting IPSet
file backed up on disk, before the new entry is added.

I then stop SSHGuard.

The sshguard4 IPSet, as known to FirewallD is still there.

I remove the SSHGuard blacklist.

I restart SSHGuard.

10:51:40 host systemd[1]: Started SSHGuard - blocks brute-force login attempts.
10:51:40 host sshguard[15354]: blacklist: blocking 0 addresses
10:51:40 host sshguard[15354]: Now monitoring attacks.

The sshguard4 IPSet, as known to FirewallD is still there.

So now we have a situation where FirewallD "knows about" all 28793 original
blacklisted entries, plus any new blocks before SSHGuard was stopped, but
SSHGuard has no record of anything in the sshguard4 IPSet.

Then again, if FirewallD is dropping connections from all 28793 original
blacklisted entries, plus any new blocks, does SSHGuard need to know that
it was the original source of a given block/drop ?

I think this comes down to whether SSHGuard, or more correctly its blacklist,
has to be the "sole arbiter" of what gets blocked, or whether it is seen as
something that merely adds to, or removes from, whatever system firewalling
mechanisms are in place.

The issue for me now becomes one of removing blocks known to SSHGuard
that are also known to FirewallD.

If this can only occur via a re-reading of a blacklist file from which
"to be unblocked" entries have been removed, then it's going to take
too long to be operationally viable.

If, on other hand, unblocking is handled via direct interaction with
FirewallD, and so without needing to re-read its blacklist file, it
becomes workable.

Not sure what that means for the SSHGuard blacklist though.

Maybe there's a case for two blacklists, one that only gets read at
startup and which gets compared with existing rules in the OS firewall
and an "operational" blacklist that collects any new blocks after the
SSHGuard starts.

Kevin

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Ryan M. R. <rr...@ro...>]

On 1/10/2022 8:36 PM, Kevin Zheng wrote:
> On 1/9/22 9:22 PM, Kevin Buckley wrote:
>> 2h 00min 13916
>> 1h 45min 12116  15mins 1800    120.00/m   2.00/s
>> 1h 12min  8405  33mins 3711    112.45/m   1.87/s
>> 1h  8min  7936   4mins  469    117.25/m   1.95/s
>
> These numbers don't look good.
>
> I don't have a system handy where I can test firewall-cmd, but is 
> there an interface (or another command) that lets you bulk-add address 
> to an ipset without invoking firewall-cmd once per address?

firewall-cmd --ipset=ipset --add-entries-from-file=filename seems to be 
possibly what you are looking for but that may in fact just be 
programmed to invoke firewall-cmd one per address in the file as opposed 
to a bulk load.  I would read carefully that documentation and run some 
tests to see if it's faster for you and works for you.  A script to read 
a line from a file you prepare and invoke the firewall-cmd command like 
SSHGuard currently does as you are using it timed and then another 
script to invoke the firewall-cmd command with the 
--add-entries-from-file options times should give you benchmarks.

You also could investigate other BACKENDS as SSHGuard calls them and see 
if others happen to be faster.

>
> I took a cursory look at what our "competitor" fail2ban does:
I don't think many would agree that fail2ban is a competitor of 
SSHGuard.  At least to me they seem to have very different strategies 
both of which could be integrated into the same system or used on a 
network for different systems in a way where they are not competitors 
per se but both used in different situations depending on their 
strengths in solving particular problems.
>
> https://github.com/fail2ban/fail2ban/blob/80805cabfcf57dd0454d47d7f86d43c6738ce629/config/action.d/firewallcmd-ipset.conf 
>
>
> Which, to summarize, seems to be:
>
> actionban = firewall-cmd --ipset=<ipmset> --add-entry=<ip>
>
> actionunban = firewall-cmd --ipset=<ipmset> --remove-entry=<ip>
>
> Which seems to be exactly what SSHGuard is doing.
>
> Anyone with more Linux firewall experience who could tell us if 
> there's a faster way to add to a firewalld ipset? Is it time to teach 
> SSHGuard how to use the dbus interface?
I think if you want to stick to the firewalld ipset option with SSHGuard 
carefully reading their documentation and in particular for your 
questions information about firewall-cmd options such as the one listing 
the ability to add ip addresses from a file on their web site if you 
have access to that would be a good idea.  Documentation about 
firewall-cmd is available at the web site 
https://firewalld.org/documentation/man-pages/firewall-cmd.html if you 
don't have access to a machine to run something like 'man 
firewall-cmd'.  As I said above though, I'm not sure it would be 
faster.  It would seem it would but I haven't done any testing.
>
> Regards,
> Kevin
>
>
> _______________________________________________
> sshguard-users mailing list
> ssh...@li...
> https://lists.sourceforge.net/lists/listinfo/sshguard-users

-- 
This email has been checked for viruses by AVG.
https://www.avg.com

Re: [SSHGuard-users] Are the FirewallD IPSet commands actually doing the right/any/thing?

[Christopher E. <ce...@lc...>]

On 13.01.22 06:44, Kevin Buckley wrote:
> I tried running the same commands, and saw the same thing, however,
> it's possible I have worked out what is going on
> 
> 
> # firewall-cmd --info-ipset=sshguard4
> Error: INVALID_IPSET: sshguard4
> 
> # firewall-cmd --ipset=sshguard4 --add-entry=192.102.251.102
> Error: INVALID_IPSET: sshguard4
> 
> BUT
> 
> if those commands have the "--permanent" flag added to them, so:
> 
> 
> # firewall-cmd --permanent --ipset=sshguard4 --add-entry=192.102.251.102
> success
> # firewall-cmd --permanent --info-ipset=sshguard4
> sshguard4
>    type: hash:net
>    options:
>    entries: 192.102.251.102
> #
> 
> so are the Error-s that are being seen the result of SSHGuard's commands
> lacking the "--permanent" flag when targetting IPSets, for example:
> 
> 
> fw_block() {
>      ${FIREW_CMD} --ipset="sshguard$2" --add-entry="$1/$3"
> }

That's interesting. I wonder if this is something that changed in 
firewalld. I'm using firewalld myself so I'll take a look if I see the 
same behaviour.

IMO there is no reason not to use the --permanent flag when setting the 
entries, but that should be checked to be sure. Main difference is that 
the banned IPs will survive a restart, while sshguards banlist doesn't?, 
which might lead to IPs not being unbanned. But that is easily remedied 
by letting SSHGuard wipe the sets on startup. It might already, I can check.

Christopher

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Christopher E. <ce...@lc...>]

I had originally looked into this when the topic first came up. The main 
overhead seems to be not so much the adding but rather the call to 
firewall-cmd. It is possible to collect addresses and add them all in 
one call, which dramatically speeds up the process.

The dbus-interface might be another option to speed things up, as might 
be bypassing firewall-cmd and interacting with the underlying library 
directly. I wanted to look into that but haven't so far.

On 19.01.22 10:10, Kevin Buckley wrote:
> That suggests that one could take the blacklist lines
> and simply pre-populate the
> 
> sshguard4.xml
> 
> IPSet file, before starting SSHGuard, however I am not sure
> what SSHGuard would do with existing entries if, on starting,
> it finds that the sshguard4 IPSet exists, and already has
> entries.

This would be another option, but seems rather brittle to me. a) it 
messes with firewalld-internal config files that might or might not stay 
in that location/format and b) actually getting firewalld to read those 
files means triggering a global reload which might or might not 
influence other things the user has been doing in the meantime.


There is one fundamental tradeoff to consider when collecting addresses 
for a single call: You're either inefficient (collect very few 
addresses) or slow (addresses are collected for a long time before send 
to the firewall).
Example: IP x gets put in the UNBAN collection at the end of a 10 minute 
timeout. That collection will be send to firewalld every y minutes or z 
entries. But since the backend script is only called when sshguard 
notices something, this happens the next time that happens AT THE 
EARLIEST, no matter what we choose for y,z. If that time is a day (low 
traffic server which only gets attacked occasionally) then x gets 
unbanned after a day.

The only way around that is letting sshguard trigger the backend 
periodically as well as on events.


Christopher

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Kevin B. <kev...@gm...>]

On 2022/01/11 12:36, Kevin Zheng wrote:
> On 1/9/22 9:22 PM, Kevin Buckley wrote:
>> 2h 00min 13916
>> 1h 45min 12116  15mins 1800    120.00/m   2.00/s
>> 1h 12min  8405  33mins 3711    112.45/m   1.87/s
>> 1h  8min  7936   4mins  469    117.25/m   1.95/s
> 
> These numbers don't look good.
> 
> I don't have a system handy where I can test firewall-cmd, but is there
> an interface (or another command) that lets you bulk-add address to an
> ipset without invoking firewall-cmd once per address?
> 
> I took a cursory look at what our "competitor" fail2ban does:
> 
> https://github.com/fail2ban/fail2ban/blob/80805cabfcf57dd0454d47d7f86d43c6738ce629/config/action.d/firewallcmd-ipset.conf
> 
> Which, to summarize, seems to be:
> 
> actionban = firewall-cmd --ipset=<ipmset> --add-entry=<ip>
> 
> actionunban = firewall-cmd --ipset=<ipmset> --remove-entry=<ip>
> 
> Which seems to be exactly what SSHGuard is doing.
> 
> Anyone with more Linux firewall experience who could tell us if there's
> a faster way to add to a firewalld ipset? Is it time to teach SSHGuard
> how to use the dbus interface?
> 

I am not going to claim vast firewalld experience, but

one thing worth noting is that adding IP addresses, or ranges, read
from an existing blakclist file at SSHGuard startup, results in every
line in

var/lib/sshguard/blacklist

being added to

/etc/firewalld/ipsets/sshguard4.xml

as entries between these lines


<?xml version="1.0" encoding="utf-8"?>
<ipset type="hash:net">
   <option name="family" value="inet"/>
   ... entries here ...
</ipset>

so, for example,

   <entry>192.102.251.102/32</entry>

That suggests that one could take the blacklist lines
and simply pre-populate the

sshguard4.xml

IPSet file, before starting SSHGuard, however I am not sure
what SSHGuard would do with existing entries if, on starting,
it finds that the sshguard4 IPSet exists, and already has
entries.

I can't currently speak to the speed with which a pre-populated
IPSet file is read into FirewallD, because my instance is still
slowly churning through the ingest of my blacklist, after the
most recent restart but once, I have the IPSet file, I intend
to take a look at that.

Kevin

[SSHGuard-users] Are the FirewallD IPSet commands actually doing the right/any/thing?

[Kevin B. <kev...@gm...>]

On a SLES 15 VM, with

firewalld-0.5.5-4.24.9.noarch

which matches the one on the production system I am looking
to deploy SSHGuard on,

I just ran the command that I can see inside SSHGuard's

usr/lib/sshguard/sshg-fw-firewalld

script, which purports to initialise the FirewallD IPSets.


sles-15-02:~ # firewall-cmd --permanent --new-ipset="sshguard4" --type="hash:net" --option="family=inet"
success
sles-15-02:~ #

Deep joy!

However,

sles-15-02:~ # firewall-cmd --info-ipset=sshguard4
Error: INVALID_IPSET: sshguard4
sles-15-02:~ #

Deep sorrow!


That confirms what I saw on the production system (see threads passim)
where the logs were full of

Error: INVALID_IPSET: sshguard4

messages.


On an openSUSE Leap 15.3 box that we are using for something else,
which has

firewalld-0.9.3-1.1.noarch

I tried running the same commands, and saw the same thing, however,
it's possible I have worked out what is going on


# firewall-cmd --info-ipset=sshguard4
Error: INVALID_IPSET: sshguard4

# firewall-cmd --ipset=sshguard4 --add-entry=192.102.251.102
Error: INVALID_IPSET: sshguard4

BUT

if those commands have the "--permanent" flag added to them, so:


# firewall-cmd --permanent --ipset=sshguard4 --add-entry=192.102.251.102
success
# firewall-cmd --permanent --info-ipset=sshguard4
sshguard4
   type: hash:net
   options:
   entries: 192.102.251.102
#

so are the Error-s that are being seen the result of SSHGuard's commands
lacking the "--permanent" flag when targetting IPSets, for example:


fw_block() {
     ${FIREW_CMD} --ipset="sshguard$2" --add-entry="$1/$3"
}


Kevin

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Felix S. <fel...@os...>]

Am 11.01.22 um 05:36 schrieb Kevin Zheng:
> I don't have a system handy where I can test firewall-cmd, but is there an 
> interface (or another command) that lets you bulk-add address to an ipset 
> without invoking firewall-cmd once per address?

I just checked the man page but "--add-entries-from-file=filename" looks 
interesting (there is also "--remove-entries-from-file=filename").

Of course firewalld also has a DBUS interface.

Felix

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Kevin B. <kev...@gm...>]

On 2022/01/11 12:36, Kevin Zheng wrote:
>> I don't have a system handy where I can test firewall-cmd, but is there
> an interface (or another command) that lets you bulk-add address to an
> ipset without invoking firewall-cmd once per address?

Sadly, can't claim to have huge experience (yet?) with firewalld,
and, as you may have seen in another recent post to the list,
into this thread

   Firewalld backend: do I need to create the two ipsets ?

I'm having issues with the ipset that SSHGuard creates anyway, but
am yet to get around to looking at why I'm seeing what I am, vis:

Jan 10 11:03:44 ln01 firewalld[48409]: WARNING: sshguard4: INVALID_TYPE: 'hash:net' is not supported by ipset., ignoring for run-time.
Jan 10 11:03:44 ln01 firewalld[48409]: WARNING: sshguard6: INVALID_TYPE: 'hash:net' is not supported by ipset., ignoring for run-time.
Jan 10 11:03:45 ln01 firewalld[48409]: WARNING: INVALID_IPSET: sshguard6
Jan 10 11:03:45 ln01 firewalld[48409]: WARNING: INVALID_IPSET: sshguard4
Jan 10 11:03:45 ln01 firewalld[48409]: ERROR: INVALID_IPSET: sshguard4
Jan 10 11:03:46 ln01 firewalld[48409]: ERROR: INVALID_IPSET: sshguard4
Jan 10 11:03:46 ln01 firewalld[48409]: ERROR: INVALID_IPSET: sshguard4

so, that might be part of the slow rate of ingest I've seen so far,
and/or an indication that nothing is actually being ingested?

To expand on that last point. SSHGuard is clearly working, even to the
extent of blocking (again) two IP addresses that were in the blacklist,
but which  presumably hadn't been ingested from it by the time the latest
"attack" passed the threshold, or, had been read from the file but just
not added to the ipset.

Kevin

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Kevin Z. <kev...@gm...>]

On 1/9/22 9:22 PM, Kevin Buckley wrote:
> 2h 00min 13916
> 1h 45min 12116  15mins 1800    120.00/m   2.00/s
> 1h 12min  8405  33mins 3711    112.45/m   1.87/s
> 1h  8min  7936   4mins  469    117.25/m   1.95/s

These numbers don't look good.

I don't have a system handy where I can test firewall-cmd, but is there 
an interface (or another command) that lets you bulk-add address to an 
ipset without invoking firewall-cmd once per address?

I took a cursory look at what our "competitor" fail2ban does:

https://github.com/fail2ban/fail2ban/blob/80805cabfcf57dd0454d47d7f86d43c6738ce629/config/action.d/firewallcmd-ipset.conf

Which, to summarize, seems to be:

actionban = firewall-cmd --ipset=<ipmset> --add-entry=<ip>

actionunban = firewall-cmd --ipset=<ipmset> --remove-entry=<ip>

Which seems to be exactly what SSHGuard is doing.

Anyone with more Linux firewall experience who could tell us if there's 
a faster way to add to a firewalld ipset? Is it time to teach SSHGuard 
how to use the dbus interface?

Regards,
Kevin

Re: [SSHGuard-users] Firewalld backend: do I need to create the two ipsets ?

[Kevin B. <kev...@gm...>]

On 2022/01/06 23:30, Christopher Engelhard wrote:
> On 06.01.22 06:18, Kevin Buckley wrote:
> 
>> though my expeience with the IPTables backend suggests that I should
>> read the above as saying that
>> 
>>      You need to create the two ipsets in the default zone
>> 
>> but, is that the case, or does SSHGuard do /some things/everything/
>> for you in firewalld-land, as long as you use the default (as in zone
>> in effect when SHHGuard starts) zone ?
> 
> For both firewalld and nft backends SSHGuard should take care of setting
> up the rules.
> 
> For firewalld it does so in the default zone, so if you're not using
> that you might need to change that, otherwise things should Just Work(TM).
> 
> Christopher

I note the (TM), Christopher !

having started a 2.4.2 instance up, and watched the slow crawl through
the ingest of an existing blacklist, I have seen a new IP address blocked
forever, as well as seeing a few addresses blocked temporarily and then
ublocked, but have since seen that there was something amis with the
ipset, right from the start:

Jan 10 11:03:41 ln01 sshguard[66617]: blacklist: blocking 28793 addresses
Jan 10 11:03:44 ln01 firewalld[48409]: WARNING: sshguard4: INVALID_TYPE: 'hash:net' is not supported by ipset., ignoring for run-time.
Jan 10 11:03:44 ln01 firewalld[48409]: WARNING: sshguard6: INVALID_TYPE: 'hash:net' is not supported by ipset., ignoring for run-time.
Jan 10 11:03:45 ln01 firewalld[48409]: WARNING: INVALID_IPSET: sshguard6
Jan 10 11:03:45 ln01 firewalld[48409]: WARNING: INVALID_IPSET: sshguard4
Jan 10 11:03:45 ln01 firewalld[48409]: ERROR: INVALID_IPSET: sshguard4
Jan 10 11:03:46 ln01 firewalld[48409]: ERROR: INVALID_IPSET: sshguard4
Jan 10 11:03:46 ln01 firewalld[48409]: ERROR: INVALID_IPSET: sshguard4
...

where I think those ERRORs, close to the startup, are coming from each
attempt to ingest an address from the blacklist.

Looking at the ipset itself shows:

# firewall-cmd --permanent --info-ipset=sshguard4
sshguard4
   type: hash:net
   options: family=inet
   entries:
#

Perhaps because this is my first time with SSHGuard using firewalld,
I was expecting to have seen some entries there. given the standard
SSHGuard activity I had been seeing in the systemctl status outputs,
but have seen nothing as yet.

This suggests to me that, despite what the logs say, nothing has
actually been blocked?

Kevin

Re: [SSHGuard-users] performance when using firewalld: adding/removing many entries at once

[Kevin B. <kev...@gm...>]

On 2020/09/01 16:36, Christopher Engelhard wrote:
> 
> Execution speed is probably not really a focus for them, but I'm sort of
> hoping that there are some straightforward bottlenecks that simply
> nobody has bothered to identify yet. And to be fair, only being able to
> add ~5 IPs per second to a set is REALLY slow.
> 

Although I saw that the thread dried up in September last
year, I thought I'd pitch a bit more info into it.

I have just started up an SSHGuard 2.4.2 instance, on a
SLES 15-based Cray/HPE box, and have looked at the time
it appears to be taking to ingest IP addresses from the
blacklist that was placed there ahead of process starting.

I have assumed that the ingest is done a line at a time,
hence the grep -n of the blacklist for the IP address
that a systemctl status shows was being processed, so
as to give a rough timeline:


    Active: active (running) since Mon 2022-01-10 11:03:41 AWST; 1h 8min ago
            `-93206 /usr/bin/python3 -Es /usr/bin/firewall-cmd --quiet \
            --ipset=sshguard4 --add-entry=119.63.84.130/32

# grep -n 119.63.84.130  /var/lib/sshguard/blacklist
7936:1616965857|100|4|119.63.84.130

    Active: active (running) since Mon 2022-01-10 11:03:41 AWST; 1h 12min ago
            `-95043 /usr/bin/python3 -Es /usr/bin/firewall-cmd --quiet \
            --ipset=sshguard4 --add-entry=140.249.197.153/32

# grep -n 140.249.197.153  /var/lib/sshguard/blacklist
8405:1617116224|100|4|140.249.197.153

    Active: active (running) since Mon 2022-01-10 11:03:41 AWST; 1h 45min ago
            `-108270 /usr/bin/python3 -Es /usr/bin/firewall-cmd --quiet \
            --ipset=sshguard4 --add-entry=188.166.237.18/32

# grep -n 188.166.237.18  /var/lib/sshguard/blacklist
12116:1620694731|100|4|188.166.237.18

    Active: active (running) since Mon 2022-01-10 11:03:41 AWST; 2h 0min ago
            |-61184 /usr/bin/python3 -Es /usr/bin/firewall-cmd --quiet \
            --ipset=sshguard4 --add-entry=150.138.114.102/32

# grep -n 150.138.114.102  /var/lib/sshguard/blacklist
13916:1622407810|100|4|150.138.114.102


To summarise that data:


2h 00min 13916
1h 45min 12116  15mins 1800    120.00/m   2.00/s
1h 12min  8405  33mins 3711    112.45/m   1.87/s
1h  8min  7936   4mins  469    117.25/m   1.95/s


It does seem a bit slow, especially when compared to some
SLES 12-based instances here that use the IPTables backend.
(Apologies, I don't have any comparative figures to hand,
  at time of writing, but will look to get some)


Probably doesn't tell you anything you didn't already know,
Kevin

Re: [SSHGuard-users] Firewalld backend: do I need to create the two ipsets ?

[Christopher E. <ce...@lc...>]

On 06.01.22 06:18, Kevin Buckley wrote:

> though my expeience with the IPTables backend suggests that I should
> read the above as saying that
> 
>      You need to create the two ipsets in the default zone
> 
> but, is that the case, or does SSHGuard do /some things/everything/
> for you in firewalld-land, as long as you use the default (as in zone
> in effect when SHHGuard starts) zone ?

For both firewalld and nft backends SSHGuard should take care of setting 
up the rules.

For firewalld it does so in the default zone, so if you're not using 
that you might need to change that, otherwise things should Just Work(TM).

Christopher

[SSHGuard-users] Firewalld backend: do I need to create the two ipsets ?

[Kevin B. <kev...@gm...>]

Been happily running SSHGuard 2.4.2 against IPTables, but now
have some boxes that are likely to be running FirewallD.

I read, in the sshguard-setup manpage

    firewalld
        Blocked attackers are added to two ipsets named sshguard4 and
        ssh- guard6.  The entries in the ipsets are blocked by default
        in the default firewall zone.  Additional firewall zones can be
        configured using:

           # firewall-cmd --zone=zone-name --permanent \
               --add-rich-rule="rule source ipset=sshguard4 drop"
           # firewall-cmd --zone=zone-name --permanent \
               --add-rich-rule="rule source ipset=sshguard6 drop"

        You can inspect the entries in the two ipsets using:

           # firewall-cmd --permanent --info-ipset=sshguard4
           # firewall-cmd --permanent --info-ipset=sshguard6

though my expeience with the IPTables backend suggests that I should
read the above as saying that

     You need to create the two ipsets in the default zone

but, is that the case, or does SSHGuard do /some things/everything/
for you in firewalld-land, as long as you use the default (as in zone
in effect when SHHGuard starts) zone ?

Kevin

[SSHGuard-users] openvpn messages via journalctl not showing up

[Brandon A. <vo...@gm...>]

Hello,

This one is a bit funky - because it's not really an sshguard issue per se.  I suppose it could be a Debian/openvpn/sshguard issue depending on how you look at it.

I have a Debian 9.x box, running their package of openvpn as well as their package of sshguard.  It looks like the default config here is for sshguard to read from stdin - reading in from journalctl.

On my box - this catches SSH just fine.  However, it doesn’t seem that openvpn messages ever show up.  If I manually look for them, you will see that they are indeed there:

###
root@ice:/home/vom# journalctl -o cat | grep 'TLS handshake failed' | wc -l
5145
###

However, the Debian 9 script that launches sshguard uses this:

###
/bin/journalctl -afb -p info -n1 -o cat SYSLOG_FACILITY=4 SYSLOG_FACILITY=10 | /usr/sbin/sshguard "$@“
###

So I’m assuming that combination of arguments there is “filtering” the openvpn messages.  Not intentionally of course, but rather those arguments are defining a stream that doesn’t include the openvpn messages.  I’m not a systemd expert either, so my journalctl-fu is rather weak.

Anyone ran into this ?  In a more general sense - what can I tweak on the openvpn/systemd side to get the messages into the “channel” that sshguard sees ?

Thanks.

Re: [SSHGuard-users] simple install of sshguard on ubuntu 18 20 versions

[Amit D. <ami...@gm...>]

Hi Kevin,

After adding the ufw before rules for sshguard as mentioned in
https://wiki.archlinux.org/title/Sshguard#UFW sshguard works as expected
blocking unauthorized attempts.

This option i have tried before rasing the question here, not sure maybe
because of sshguard version 1.7.3 version on ubuntu 20 initially used.
For debian 10 , ubuntu 16.04 they are stright fwd with apt commands.

Anyway thanks for your help.

Thanks,
Amit




On Fri, Dec 3, 2021 at 4:43 PM Amit Das <ami...@gm...> wrote:

>
> Hi Kevin,
>
> I am getting the output as seen in the attachment. Did u mean pipe or
> redirect  not clear?.
>
> Also my sshguard version is 1.7.3 (tried on other 2.3.1 , 2.3.4  versions
> too). I have tried on multiple vms , aws vms and dedicated servers. First
> unauthorized attempt is blocked by sshguard but later on its not blocking
> as seen in the auth logs. Not seen in journalctl logs its blocking.   Dont
> understand why sshguard drops after blocking first time and passed to sshd
> in auth logs.eventhough sshguard service is running all time.
>
> Not sure whats wrong my backend config or from ubuntu maintainer  or the
> version issues on latest ubuntu 18, ubuntu 20.
>
> Thanks,
> Amit
>
>
> On Fri, Dec 3, 2021 at 12:27 AM Kevin Zheng <kev...@gm...> wrote:
>
>> Hi Amit,
>>
>> On 12/1/21 11:41 AM, Amit Das wrote:
>> > # Log reader command (optional, no default)
>> > LOGREADER="LANG=C /usr/bin/journalctl -afb -p info -n1 -t sshd -t
>> vsftpd
>> > -o cat"
>>
>> Could you check that your LOGREADER command is actually giving you the
>> log output from sshd?
>>
>> That is, run this command at the command line, and see if any failed
>> login messages are coming through:
>>
>> $ /usr/bin/journalctl -afb -p info -n1 -t sshd -t vsftpd -o cat
>>
>> If they are coming through, pipe the output to `sshg-parser -a` and make
>> sure the attacks you expect to be recognized are marked with an asterisk.
>>
>> Regards,
>> Kevin
>>
>

Re: [SSHGuard-users] simple install of sshguard on ubuntu 18 20 versions

[Kevin Z. <kev...@gm...>]

Hi Amit,

On 12/1/21 11:41 AM, Amit Das wrote:
> # Log reader command (optional, no default)
> LOGREADER="LANG=C /usr/bin/journalctl -afb -p info -n1 -t sshd -t vsftpd 
> -o cat"

Could you check that your LOGREADER command is actually giving you the 
log output from sshd?

That is, run this command at the command line, and see if any failed 
login messages are coming through:

$ /usr/bin/journalctl -afb -p info -n1 -t sshd -t vsftpd -o cat

If they are coming through, pipe the output to `sshg-parser -a` and make 
sure the attacks you expect to be recognized are marked with an asterisk.

Regards,
Kevin

Re: [SSHGuard-users] simple install of sshguard on ubuntu 18 20 versions

[Kevin Z. <kev...@gm...>]

Hi Amit,

It sounds like Ubuntu 18 and 20 broke SSHGuard for many users, not just 
you. I don't have the resources (an Ubuntu installation and experience 
using it) to troubleshoot.

Do you know who the maintainer for SSHGuard on Ubuntu is and put us in 
touch, maybe in this thread?

Thanks,
Kevin

Re: [SSHGuard-users] simple install of sshguard on ubuntu 18 20 versions

[Amit D. <ami...@gm...>]

Hi Kevin,

Thanks for the response.

Could you please describe exactly what troubleshooting steps you tried?

On ubuntu 16.04 apt install sshguard. Sucessfully works no issues on tests
also.
On ubuntu 18 and ubuntu 20  did "apt install sshguard".

Version installed 1.7.3.
sshguard status running.

Did multiple attempts (more than 30)  to ssh from different ip address but
in auth logs i see sessions closing everytime
Expected auth logs should be sshgurad to block the unauthrized attempts.

As i have already install on more than 50 vms without testing on ubuntu 18
and 20 its difficult to build from source code and run on all machines.

For example if i want to install sshguard on 50 vms (ubuntu 20) do i need
compile build from source code on all VMs. " apt get install sshguard"
will not work?


Can you give more details about your setup and configuration? We can't
help you if you don't.


In sshguard troubleshooting it says:

Check the paths first: where is iptables, or pfctl, or ipfw? You may need
to specify their path explicitly from ./configure if they are not in
standard paths nor in system's PATH..


I have ufw enabled and added the backend as

# Full path to backend executable (required, no default)
BACKEND="/usr/lib/sshguard/sshg-fw-iptables"

# Log reader command (optional, no default)
LOGREADER="LANG=C /usr/bin/journalctl -afb -p info -n1 -t sshd -t vsftpd -o
cat"

# How many problematic attempts trigger a block
THRESHOLD=20
# Blocks last at least 180 seconds
BLOCK_TIME=180
# The attackers are remembered for up to 3600 seconds
DETECTION_TIME=3600

# Blacklist threshold and file name
BLACKLIST_FILE=100:/var/db/sshguard/blacklist.db

# IPv6 subnet size to block. Defaults to a single address, CIDR notation.
(optional, default to 128)
IPV6_SUBNET=64
# IPv4 subnet size to block. Defaults to a single address, CIDR notation.
(optional, default to 32)
IPV4_SUBNET=24


Restarted sshguard and ufw.

Followed this document also which didnt work.

 Finally installed iptables. and updated the backend.



This are the steps i followed.

Please suggest me what i have missed here.

Thanks
amit

On Wed, Dec 1, 2021 at 2:02 AM Kevin Zheng <kev...@gm...> wrote:

> Hi Amit,
>
> On 11/30/21 11:51 AM, Amit Das wrote:
> > I have used sshguard on ubuntu 16 without any issues. Recently i
> > installed on ubuntu 18 and ubuntu 20 servers which is not working as
> > expected. Went through few threads like
> >
> https://askubuntu.com/questions/1245543/how-do-i-configure-sshguard-in-ubuntu-20-04
> > <
> https://askubuntu.com/questions/1245543/how-do-i-configure-sshguard-in-ubuntu-20-04>
>
> > which didn't help much. In my auth logs i can see closed sessions for
> > unauthorized users sshd sessions  but not blocking me even after
> > multiple attempts.
>
> Could you please describe exactly what troubleshooting steps you tried?
> Can you give more details about your setup and configuration? We can't
> help you if you don't.
>
> Have you looked at the troubleshooting section of the sshguard-setup man
> page? Which troubleshooting steps have you tried, if any?
>
> Regards,
> Kevin
>

Re: [SSHGuard-users] simple install of sshguard on ubuntu 18 20 versions

[Kevin Z. <kev...@gm...>]

Hi Amit,

On 11/30/21 11:51 AM, Amit Das wrote:
> I have used sshguard on ubuntu 16 without any issues. Recently i 
> installed on ubuntu 18 and ubuntu 20 servers which is not working as 
> expected. Went through few threads like
> https://askubuntu.com/questions/1245543/how-do-i-configure-sshguard-in-ubuntu-20-04 
> <https://askubuntu.com/questions/1245543/how-do-i-configure-sshguard-in-ubuntu-20-04>  
> which didn't help much. In my auth logs i can see closed sessions for 
> unauthorized users sshd sessions  but not blocking me even after 
> multiple attempts.

Could you please describe exactly what troubleshooting steps you tried? 
Can you give more details about your setup and configuration? We can't 
help you if you don't.

Have you looked at the troubleshooting section of the sshguard-setup man 
page? Which troubleshooting steps have you tried, if any?

Regards,
Kevin

[SSHGuard-users] simple install of sshguard on ubuntu 18 20 versions

[Amit D. <ami...@gm...>]

Hi,

I have used sshguard on ubuntu 16 without any issues. Recently i installed
on ubuntu 18 and ubuntu 20 servers which is not working as expected. Went
through few threads like
https://askubuntu.com/questions/1245543/how-do-i-configure-sshguard-in-ubuntu-20-04
which didn't help much. In my auth logs i can see closed sessions for
unauthorized users sshd sessions  but not blocking me even after multiple
attempts.

Also *backends *i dont see few packages /usr/lib/sshguard/sshd-fw  or
netfiler or iptables path. I am using ufw rules.
There is a bug stating ip tables links broken.
https://bugs.launchpad.net/ubuntu/+source/sshguard/+bug/1884848

Is this what i am missing here ?

I have installed sshguard on 100s vms without testing on latest OS.

Is there any simple way like may be apt commands to install latest version
without compiling from source code.

Please let me know simple solution.

Thanks,
amit

Re: [SSHGuard-users] sshd and connections resulting in "kex_exchange_identification" errors

[Burton S. <Bu...@Bu...>]

I think you will need to patch sshd...

Find the line that generates that message and add the source IP to it. Then
you'll be able to add the signature line to sshguard.



-----Burton 

-----Original Message-----
Message: 1
Date: Sat, 20 Nov 2021 07:13:19 +1100
From: Greg Bell <gbe...@ya...>
To: ssh...@li...
Subject: [SSHGuard-users] sshd and connections resulting in
	"kex_exchange_identification" errors
Message-ID: <894...@ya...>
Content-Type: text/plain; charset=utf-8; format=flowed

Hi,

My sshd server sits on port 443 so I can get to it from behind corp
firewalls. So it gets a lot of http requests, which result in things like:

 ??? Nov 20 02:12:12 server sshd[1170601]: error: 
kex_exchange_identification: banner line contains invalid characters

No IP is reported, so sshguard can't do anything about these. I'd like to
block them - seems reasonable that a hack, or at least a DOS, could happen
at that early point in sshd's protocol.

Does anybody have experience blocking based on these connection attempts?


Best regards,

~gb




------------------------------



------------------------------

Subject: Digest Footer

_______________________________________________
sshguard-users mailing list
ssh...@li...
https://lists.sourceforge.net/lists/listinfo/sshguard-users


------------------------------

End of sshguard-users Digest, Vol 117, Issue 3
**********************************************