From: Amir 'C. C. <ce...@3p...> - 2012-08-26 18:04:24
|
Hi, Is there any way for me to see who has performed a _successful_ SMTP auth with saslauthd? I'm running CentOS 5.8, using sendmail and saslauthd for SMTP auth. Auth is required for any sending of outside mail... while looking at my SMTP logs, it appears that a user account may have been compromised, as I see entries that look like the following: Aug 26 13:41:58 kismet sm-scanner[12936]: q7NJUPhL023672: to=<xxx>, delay=2+22:11:32, xdelay=00:00:01, mailer=esmtp, pri=25320000, relay=xxx. [xxx], dsn=4.1.1, stat=Deferred: 450 4.1.1 <xxx>: Recipient address rejected: unverified address: unknown user: "xxx" I don't have any open relaying enabled, SMTP AUTH is required, so this suggests that a user account has been compromised. However... I can't figure out how to check WHICH user account! /var/log/secure contains error messages when a user FAILS to authenticate... but there are no log messages for success. So, I can't figure out which user is the one performing successful auth prior to these clear spam attempts. Any help would be greatly appreciated... and ASAP since I want to terminate these spam issues immediately. Thanks!! --- Amir |
From: Amir 'C. C. <ce...@3p...> - 2012-08-26 19:12:28
|
Sorry for the multiple sends, as I mentioned, I wasn't sure if the originals got through... apologies for the duplicates! To update: One thing I just realized is that the error messages are from sm-scanner, not sm-acceptingconnections... not sure if that means anything or not, except that sm-scanner doesn't log the connecting IP, it just seems to log the relay IP. For users who use TLS, I can see their authentications in /var/log/maillog, but users who authenticate without TLS don't show up there as far as I can tell. So, basically, I'm trying to find out if there's a way to get sendmail or SASL to log the sending (authenticated) user, not just the recipient. Thanks. --- Amir At 12:04 PM -0600 08/26/2012, Amir 'CG' Caspi wrote: >Hi, > > Is there any way for me to see who has performed a >_successful_ SMTP auth with saslauthd? > > I'm running CentOS 5.8, using sendmail and saslauthd for SMTP >auth. Auth is required for any sending of outside mail... while >looking at my SMTP logs, it appears that a user account may have been >compromised, as I see entries that look like the following: > >Aug 26 13:41:58 kismet sm-scanner[12936]: q7NJUPhL023672: to=<xxx>, >delay=2+22:11:32, xdelay=00:00:01, mailer=esmtp, pri=25320000, >relay=xxx. [xxx], dsn=4.1.1, stat=Deferred: 450 4.1.1 <xxx>: >Recipient address rejected: unverified address: unknown user: "xxx" > >I don't have any open relaying enabled, SMTP AUTH is required, so >this suggests that a user account has been compromised. > > However... I can't figure out how to check WHICH user >account! /var/log/secure contains error messages when a user FAILS >to authenticate... but there are no log messages for success. > So, I can't figure out which user is the one performing >successful auth prior to these clear spam attempts. > >Any help would be greatly appreciated... and ASAP since I want to >terminate these spam issues immediately. > >Thanks!! > --- Amir > >------------------------------------------------------------------------------ >Live Security Virtual Conference >Exclusive live event will cover all the ways today's security and >threat landscape has changed and how IT managers can respond. Discussions >will include endpoint security, mobile security and the latest in malware >threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >_______________________________________________ >Fail2ban-users mailing list >Fai...@li... >https://lists.sourceforge.net/lists/listinfo/fail2ban-users -- --- Amir 3Phase Internet Solutions http://www.3phase.com |
From: Fabian W. <fa...@we...> - 2012-08-26 20:46:22
|
Hello Amir First, I do not understand what this should have to do with Fail2ban? On 26.08.2012 21:12, Amir 'CG' Caspi wrote: > To update: > > One thing I just realized is that the error messages are from > sm-scanner, not sm-acceptingconnections... not sure if that means > anything or not, except that sm-scanner doesn't log the connecting > IP, it just seems to log the relay IP. > For users who use TLS, I can see their authentications in > /var/log/maillog, but users who authenticate without TLS don't show > up there as far as I can tell. > So, basically, I'm trying to find out if there's a way to get > sendmail or SASL to log the sending (authenticated) user, not just > the recipient. I guess this is just a spam bot, which tries to relay e-mails through to your server. As I see it, this has nothing to do with SMTP Auth or an abused account. If your server is listening on any of the known SMTP ports (25, 465 or 587), anybody can try if it is possible to deliver an e-mail there for any other third party. That is is being logged with "unverified address: unknown user" has probably something to do with your specific sendmail configuration. bye Fabian |
From: Amir 'C. C. <ce...@3p...> - 2012-08-26 21:35:08
|
Hi Fabian, Responses below. At 10:46 PM +0200 08/26/2012, Fabian Wenk wrote: >Hello Amir > >First, I do not understand what this should have to do with Fail2ban? It doesn't... I was just hoping that someone on the list would know what to do! I apologize for using the list for "general help" rather than something f2b-specific. >I guess this is just a spam bot, which tries to relay e-mails >through to your server. As I see it, this has nothing to do with >SMTP Auth or an abused account. If your server is listening on >any of the known SMTP ports (25, 465 or 587), anybody can try if >it is possible to deliver an e-mail there for any other third >party. That is is being logged with "unverified address: unknown >user" has probably something to do with your specific sendmail >configuration. No, unfortunately, that's not correct. As I mentioned, I've turned off all external relaying without SMTP AUTH. Whenever a third party tries to relay through the server without auth, a _different_ error is logged: Aug 26 13:09:31 kismet sm-acceptingconnections[12455]: q7QH90ro 012455: ruleset=check_rcpt, arg1=no...@no..., relay=somehost.domain.com [192.168.0.1], reject=550 5.7.1 no...@no...... Relaying denied. Proper authentication required. Note that this is a different error message and also logged from a different daemon (sm-acceptingconnections, instead of sm-scanner). If a spambot tried to relay without auth (as happens all the time), the above error is what would be logged, not what I showed earlier. The limited Google results that DID come up for this error indicate that the "unverified address" message is from the receiving server (which is relay=<host> parameter, specifically it's looking for eforward1.registrar-servers.com), and it would only occur when sendmail is attempting to send a message it thinks is "legitimate." Therefore, it would only occur AFTER a proper, successful SMTP AUTH. This is why I think one of the user accounts has been compromised. I should note that these errors did not appear prior to about a week ago, so this is a recent occurrence, which again makes me suspect an account being compromised. Thanks. --- Amir |
From: Fabian W. <fa...@we...> - 2012-08-27 10:28:58
|
Hello Amir On 26.08.2012 23:34, Amir 'CG' Caspi wrote: > At 10:46 PM +0200 08/26/2012, Fabian Wenk wrote: >>Hello Amir >> >>First, I do not understand what this should have to do with Fail2ban? > > It doesn't... I was just hoping that someone on the list would know > what to do! I apologize for using the list for "general help" rather > than something f2b-specific. Most often the chances are much higher to have some good help on the mailing list for the used software. But sure, this can vary. >>I guess this is just a spam bot, which tries to relay e-mails >>through to your server. As I see it, this has nothing to do with >>SMTP Auth or an abused account. If your server is listening on >>any of the known SMTP ports (25, 465 or 587), anybody can try if >>it is possible to deliver an e-mail there for any other third >>party. That is is being logged with "unverified address: unknown >>user" has probably something to do with your specific sendmail >>configuration. > > No, unfortunately, that's not correct. As I mentioned, I've turned > off all external relaying without SMTP AUTH. Whenever a third party > tries to relay through the server without auth, a _different_ error > is logged: > > Aug 26 13:09:31 kismet sm-acceptingconnections[12455]: q7QH90ro > 012455: ruleset=check_rcpt, arg1=no...@no..., > relay=somehost.domain.com [192.168.0.1], reject=550 5.7.1 > no...@no...... Relaying > denied. Proper authentication required. You are right, this are the entries for relaying which is tried from outside, I also do have them and I created a Fail2ban filter (to stay partly on topic in this mailing list ;) to catch them and block connections from such IP addresses, see [1]. [1] http://www.wenks.ch/fabian/fail2ban/#sendmail > The limited Google results that DID come up for this error indicate > that the "unverified address" message is from the receiving server > (which is relay=<host> parameter, specifically it's looking for > eforward1.registrar-servers.com), and it would only occur when > sendmail is attempting to send a message it thinks is "legitimate." > Therefore, it would only occur AFTER a proper, successful SMTP AUTH. > This is why I think one of the user accounts has been compromised. I am quite confused, if I have a look at your log entry again, it got me thinking: Aug 26 13:41:58 kismet sm-scanner[12936]: q7NJUPhL023672: to=<det...@zo...>, delay=2+22:11:32, xdelay=00:00:01, mailer=esmtp, pri=25320000, relay=eforward1.registrar-servers.com. [69.160.33.82], dsn=4.1.1, stat=Deferred: 450 4.1.1 <det...@zo...>: Recipient address rejected: unverified address: unknown user: "det...@zo..." What else do you see, if you grep for q7NJUPhL023672 in this log file? For me it looks like one of your user tries to send an e-mail to det...@zo..., which on the receiving side does not exist. But the strange part is, that you get a 4xx (soft) error for "user unknown" back from eforward1.registrar-servers.com instead of a 5xx (hard) error, so your server will try do deliver this message later. Maybe this is some "broken" grey listing or else a broken setup on the other side. Could it be, that one of your users got hit with bounces because his e-mail address was abused in a spam run as sender address and he did create some kind of filter in his mail client, which will reject such messages, and then tries to send them again to the invalid e-mail address, which probably caused the bounce at first? > I should note that these errors did not appear prior to about a week > ago, so this is a recent occurrence, which again makes me suspect an > account being compromised. Could also be. Or one of your users computer is being abused to send spam, which will be sent through his configured mail client, and those using his regular credential for SMTP authentication. Try to check from which IP addresses you are getting this e-mails for relaying. Do you also see regular e-mails from one of your real users from the same IP address or not? Cross check also with IMAP or POP3 login from the same IP address. bye Fabian |
From: Amir 'C. C. <ce...@3p...> - 2012-08-27 15:02:22
|
Hi Fabian, At 12:28 PM +0200 08/27/2012, Fabian Wenk wrote: >For me it looks like one of your user tries to send an e-mail to >det...@zo..., which on the receiving side does not >exist. But the strange part is, that you get a 4xx (soft) error >for "user unknown" back from eforward1.registrar-servers.com >instead of a 5xx (hard) error, so your server will try do deliver >this message later. Maybe this is some "broken" grey listing or >else a broken setup on the other side. After a lot of digging last night, it turns out that your suggestion is pretty close to correct. The message in question is being sent by the mail daemon, not by a user, and it is, indeed, spam backscatter. Spam is/was being sent to an email alias that forwards off-site... the sender wasn't listed in RBLs and wasn't rejected by my server immediately. The receiving system uses milters during the SMTP transaction and rejected the spam during SMTP, causing my system to send the bounce message, but to an invalid address (obviously), causing these errors to appear on my system since it was the one trying to forward the bounce. Normally, my system doesn't send backscatter, but because this is an off-site email alias, the SMTP server has already accepted the message before the forwarding software gets it... so when the bounce arrives, my system thinks it's for a legitimate email. It's an unfortunate consequence of the hosting software I use. =( I guess the only way to stop this is to figure out how to stop the forwarding software from reverse-forwarding bounce messages... or to implement anti-spam milters at the SMTP level. (Currently, I have SpamAssassin/MailScanner running post-SMTP, after the message has been accepted; the anti-spam techniques at the SMTP level are primarily RBLs. But, I could implement SpamAssassin via sendmail... downside is that legitimate messages may get blocked without the user ever knowing.) >Try to check from which IP addresses you are getting this e-mails >for relaying. Do you also see regular e-mails from one of your >real users from the same IP address or not? Cross check also with >IMAP or POP3 login from the same IP address. Well, that's what I was trying to check, but like I said, I couldn't see which user was authenticating to send the email. In this case, it's because there was no user... but in general, I can't see successful user authentications, only auth failures. The Cyrus-SASL mailing list provided some instructions on increasing the loglevel, I'll see if that helps... But, at least for now, this problem is "solved" (in that I know why it's happening, though not really how to stop it). Thanks for the help! --- Amir |
From: Fabian W. <fa...@we...> - 2012-08-27 21:05:17
|
Hello Amir On 27.08.2012 17:01, Amir 'CG' Caspi wrote: > Normally, my system doesn't send backscatter, but because > this is an off-site email alias, the SMTP server has already accepted > the message before the forwarding software gets it... so when the > bounce arrives, my system thinks it's for a legitimate email. > It's an unfortunate consequence of the hosting software I use. =( > > I guess the only way to stop this is to figure out how to stop the > forwarding software from reverse-forwarding bounce messages... or to > implement anti-spam milters at the SMTP level. (Currently, I have > SpamAssassin/MailScanner running post-SMTP, after the message has > been accepted; the anti-spam techniques at the SMTP level are > primarily RBLs. But, I could implement SpamAssassin via sendmail... > downside is that legitimate messages may get blocked without the user > ever knowing.) In my setup the spam tagging with SpamAssassin does happen before the e-mails are delivered into the mailboxes. Normally I do not reject any e-mails based on RBLs or anything similar. E-mails which are recognizes as spam are put into a special folder in the users mailbox, and are automatically deleted after 70 days. As my IMAP server is using Sieve, users can create filter there, and even create a forwarding rule after the spam has been put into that special folders. So the chances are not that high any more, that on the forwarding end the mail is being rejected back to me. > But, at least for now, this problem is "solved" (in that I know why > it's happening, though not really how to stop it). Above is an idea, but probably you will find something else which fits better into your setup. > Thanks for the help! You're welcome! bye Fabian |
From: Amir 'C. C. <ce...@3p...> - 2012-08-27 22:25:29
|
Hi Fabian, Yes, SpamAssassin tags the mail before it is _delivered_ to the mailboxes, but not before it is _accepted_ by the SMTP server. There are two separate events: first, sendmail decides whether to accept or reject the mail... if it accepts the mail, it closes the SMTP connection and then (if the mail is accepted) it gets passed to SpamAssassin for processing and ultimate delivery to the user (either to their inbox, their spam folder, or whatever). The benefit of this method is that users can filter their spam however they want (into their inbox, their spam folder, or outright deletion). The downside is that if the "user" is actually an email alias to an off-site mailbox, the spam _cannot_ get filtered, because it gets forwarded off-site before that can happen... my control panel software doesn't allow filtering prior to forwarding for these email aliases, it just forwards everything that comes to the alias. There is a way to set up sendmail so that it calls SpamAssassin directly _during_ the SMTP transaction, so that if SpamAssassin considers the mail to be spam, sendmail will actually reject the mail directly during the transaction. In this case, not only will sendmail provide an error message (e.g. 451) to the sending server, it will also reject the mail outright, i.e. that spam-tagged mail will never get delivered to any mailbox on the server. The benefit of this latter method is that it would prevent this kind of backscatter from spam forwarded to aliases, because it would never forward the spam at all - it would just get blocked. The downside is that this could result in some legitimate mail getting tagged as spam, rejected, and then the user will never see it. In the former method, above (the one you and I currently implement), a regular (non-alias) user has the option to see spam messages, depending on their filtering criteria. I guess I'll have to weigh the pros and cons... Thanks! --- Amir At 11:05 PM +0200 08/27/2012, Fabian Wenk wrote: >Hello Amir > >On 27.08.2012 17:01, Amir 'CG' Caspi wrote: >> Normally, my system doesn't send backscatter, but because >> this is an off-site email alias, the SMTP server has already accepted >> the message before the forwarding software gets it... so when the >> bounce arrives, my system thinks it's for a legitimate email. >> It's an unfortunate consequence of the hosting software I use. =( >> >> I guess the only way to stop this is to figure out how to stop the >> forwarding software from reverse-forwarding bounce messages... or to >> implement anti-spam milters at the SMTP level. (Currently, I have >> SpamAssassin/MailScanner running post-SMTP, after the message has >> been accepted; the anti-spam techniques at the SMTP level are >> primarily RBLs. But, I could implement SpamAssassin via sendmail... >> downside is that legitimate messages may get blocked without the user >> ever knowing.) > >In my setup the spam tagging with SpamAssassin does happen before >the e-mails are delivered into the mailboxes. Normally I do not >reject any e-mails based on RBLs or anything similar. E-mails >which are recognizes as spam are put into a special folder in the >users mailbox, and are automatically deleted after 70 days. As my >IMAP server is using Sieve, users can create filter there, and >even create a forwarding rule after the spam has been put into >that special folders. So the chances are not that high any more, >that on the forwarding end the mail is being rejected back to me. > > >> But, at least for now, this problem is "solved" (in that I know why >> it's happening, though not really how to stop it). > >Above is an idea, but probably you will find something else which >fits better into your setup. > >> Thanks for the help! > >You're welcome! > > >bye >Fabian > |
From: Fabian W. <fa...@we...> - 2012-08-28 14:38:23
|
Hello Amir On 28.08.2012 00:25, Amir 'CG' Caspi wrote: > Yes, SpamAssassin tags the mail before it is _delivered_ to > the mailboxes, but not before it is _accepted_ by the SMTP server. Same here. > There are two separate events: first, sendmail decides whether to > accept or reject the mail... if it accepts the mail, it closes the > SMTP connection and then (if the mail is accepted) it gets passed to > SpamAssassin for processing and ultimate delivery to the user (either > to their inbox, their spam folder, or whatever). Same here, but I do only reject e-mails from unknown sender domains or for recipients which do not exists locally. I stopped using RBLs on the SMTP level to reject e-mails around a decade ago. I just use them in SA for tagging. So in general my mail server is a nice one to the outside world and receives all proper e-mails. > The benefit of this method is that users can filter their > spam however they want (into their inbox, their spam folder, or > outright deletion). The downside is that if the "user" is actually Right. > an email alias to an off-site mailbox, the spam _cannot_ get > filtered, because it gets forwarded off-site before that can > happen... my control panel software doesn't allow filtering prior to > forwarding for these email aliases, it just forwards everything that > comes to the alias. OK, this is not very nice. If you are using an IMAP/POP server like cyrus or dovecot, you could use Sieve [1], where the user could create rules (including mail forward) on the server, which are parsed after SA and before delivering the mail into the mailbox. If you are using the Roundcube [2] web mail client, their is the SieveRules [3] plugin available, so users can manage their own Sieve rules. [1] http://en.wikipedia.org/wiki/Sieve_%28mail_filtering_language%29 [2] http://www.roundcube.net/ [3] http://www.tehinterweb.co.uk/roundcube/#pisieverules I use this to move certain e-mails directly into corresponding folders. So I do not have to setup the filters on any of my mail clients and can just access my e-mails through IMAP from any of my computers (or smartphone) and the filtering is already done. > There is a way to set up sendmail so that it calls > SpamAssassin directly _during_ the SMTP transaction, so that if > SpamAssassin considers the mail to be spam, sendmail will actually > reject the mail directly during the transaction. In this case, not In Sendmail this is called milter. You have more then one possibility here, e.g. use spamass-milter [4] directly for SA or use SA through amavisd-new [5] together with amavisd-milter [6]. Currently I use amavisd-new only for virus filtering with ClamAV. [4] http://savannah.nongnu.org/projects/spamass-milt/ [5] http://www.ijs.si/software/amavisd/ [6] http://amavisd-milter.sourceforge.net/ But if you are using spam filtering (and rejecting) directly in a Sendmail milter setup, then you probably would like to setup a multiple MTA setup, so that regular outgoing e-mails from your users are not going through SA (as there is a high chance, that they are delivering their e-mails from IP addresses which are in any of the RBLs). It should be possible to have several Sendmail instances running on the same system (you need to configure different queues), e.g. one on port 25 to receive e-mails from the outside world and let them through milter (with SA) and then deliver them to your IMAP/POP server. And then you need to have an other Sendmail instance which is listening on port 465 (SSL) and 587 with SMTP Auth to receive and relay e-mails from your users. If your users are still using port 25 to deliver e-mails, it is recommended that they adjust their clients. With the multiple MTA setup, you could also use amavisd-new without milter and integrate it into the mail flow as an other MTA (amavisd-new can also receive and forward e-mails on TCP ports, check out [7]). [7] http://www.ijs.si/software/amavisd/README.sendmail-dual This multiple MTA setup is something on my to-do list, but I guess it is also something which I should first test on a test system. ;) > only will sendmail provide an error message (e.g. 451) to the sending > server, it will also reject the mail outright, i.e. that spam-tagged > mail will never get delivered to any mailbox on the server. > The benefit of this latter method is that it would prevent > this kind of backscatter from spam forwarded to aliases, because it > would never forward the spam at all - it would just get blocked. The Yes, or you use my idea from above, and do the forwarding with Sieve on the IMAP/POP server (if possible). > downside is that this could result in some legitimate mail getting > tagged as spam, rejected, and then the user will never see it. In This is why I do receive all the e-mails and deliver them to the mailbox, so the user could check in the spam folder. > the former method, above (the one you and I currently implement), a > regular (non-alias) user has the option to see spam messages, > depending on their filtering criteria. > > I guess I'll have to weigh the pros and cons... Good luck with that, you now should have some more ideas how this could be done. PS: No need to use "reply all", reply only to the list is perfect, as I do filter e-mails based on the "List-Id" header line. bye Fabian |