From: Charles B. <br...@br...> - 2013-09-21 23:38:59
|
Hello Simon Why are you collecting bad IP data? I have lots, and other data of interest to somebody working to combat the bad guys. Can I trust you with it if I start sending it automatically? Call me paranoid, it could immediately identify me! whois 417.ch sieht sehr gut aus, aber...(looks OK, but...) Regards Charles Bradshaw On Sat, 2013-09-21 at 18:05 +0200, mr51m0n wrote: > Hi > > I have written a tutorial on how to graph fail2ban "bans" with > badips.com. Maybe it's worth to put it on the HowTo section of > fail2ban.org? > > Here's the link: http://www.netmess.org/examine-your-attackers/ > > Please let me know if you think this is the right place to ask and if > the quality is ok. > > Thank you, Simon > > > ------------------------------------------------------------------------------ > LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! > 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint > 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes > Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/22/13. > http://pubads.g.doubleclick.net/gampad/clk?id=64545871&iu=/4140/ostg.clktrk > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users |
From: <am...@ba...> - 2013-09-23 09:35:29
|
Hi Charles I'm sorry for my late reply! Due to some incidents over the weekend, I had to cut the strings. > Why are you collecting bad IP data? I have two main goals: 1. Provide a blacklist to overcome the botnet problem with fail2ban. This means that if someone tries to login 6 times from each of the, say, 10000 members of a botnet, one can still try 60000 passwords. 2. Gather statistics to better understand attacks in general and make them public (website / API) (3. because it's fun) > > Can I trust you with it if I start sending it automatically? What if I just say yes? Of course, you can not trust me as I can not trust you. > > Call me paranoid, it could immediately identify me! Yes, it could. I promise I'll do my best to protect reporters IP addresses. They are not accessible over the website nor the API and it will never be. I plan to publish data based on the country the reporters come from (kind of a heatmap) but not something unique like an IP. The bigger problem I guess is how do I protect my identity? Since I'm the guy who really collects and redistributes the data. If you like, I'm happy to welcome you as a reporter to badips.com. If you have data you cannot deliver to badips' API at the moment but still like to publish it on badips.com, please let me know. R, Amy |
From: Charles B. <br...@br...> - 2013-09-23 19:08:22
|
Hello Amy Please don't misunderstand my previous post. I started to write a very much longer response to 'mr51m0n' and then realised I had no idea who I might be talking too. I have a contribution to make: I have the logs of many thousands of offending IPs. The most interesting fact to emerge so far is that blocking bots does NOT appear to reduce the number of hits! It's early days yet, but since the beginning of August this year fail2ban has reported 4500 blocks! I installed f2b because of a sudden upsurge in the number of (target unspecified) hits. These are not attempts at SSH, for which there are very much better strategies for combating brute force password attacks. I'm also seeing a huge and probably related amount email containing what are pretty obviously bot infections. These should be investigated. I'm now too old and tired to engage directly. ;-) Combating botnets is a very serious business. If your definition of "fun" can be translated as an intellectual challenge then I agree, but fun, per se, it definitely is not. Right now, we're loosing the WAR, for that's what it is. In the sense that we might use them for blocking, I don't think published bad IP addresses serve any useful purpose for everyday sysadmins. To the contrary we could be telling bot controllers which machines to abandon quietly! On the other hand, to some research project the data could be very useful indeed. To the extent that if it becomes a successful weapon in the WAR then we are very likely to become casualties if we under estimate to enemy. On reflection, I think publishing a script to log IP is well motivated, but naive. (or an attempt!!) Without a clear disposition of the goals of the project. More importantly, a clear definition of the protection mechanism of the sources and distributions of the, undoubtedly 'very useful' data, is a prerequisite. "What if I just say yes?" Then prove it? Simple, propose a trust mechanism. If I can see that I am contributing to a meaningful project I will, in advance, provide you with a solid means of identifying me. Provided that I can initially acquire some degree of trust in such a project. My original post was directed to 'mr51m0n', perhaps for some perfectly good honest reason, he has yet not responded! I wrote some more stuff here which I decided not to post. I've said enough. Just because I quote Martin Luther does not make me a good guy! Charles Bradshaw "We must learn to live together as brothers or perish together as fools." - Martin Luther King, Jr. On Mon, 2013-09-23 at 09:35 +0000, am...@ba... wrote: > Hi Charles > > I'm sorry for my late reply! > > Due to some incidents over the weekend, I had to cut the strings. > > > Why are you collecting bad IP data? > > I have two main goals: > > 1. Provide a blacklist to overcome the botnet problem with fail2ban. This means that if someone tries to login 6 times from each of the, say, 10000 members of a botnet, one can still try 60000 passwords. > > 2. Gather statistics to better understand attacks in general and make them public (website / API) > > (3. because it's fun) > > > > > Can I trust you with it if I start sending it automatically? > > What if I just say yes? > > Of course, you can not trust me as I can not trust you. > > > > > Call me paranoid, it could immediately identify me! > > Yes, it could. I promise I'll do my best to protect reporters IP addresses. They are not accessible over the website nor the API and it will never be. I plan to publish data based on the country the reporters come from (kind of a heatmap) but not something unique like an IP. > > The bigger problem I guess is how do I protect my identity? Since I'm the guy who really collects and redistributes the data. > > > If you like, I'm happy to welcome you as a reporter to badips.com. > > If you have data you cannot deliver to badips' API at the moment but still like to publish it on badips.com, please let me know. > > R, Amy > > ------------------------------------------------------------------------------ > LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! > 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint > 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes > Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. > http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users |
From: <am...@ba...> - 2013-09-25 12:16:39
|
Hey > I have a contribution to make: I have the logs of many thousands of > offending IPs. The most interesting fact to emerge so far is that > blocking bots does NOT appear to reduce the number of hits! It's early > days yet, but since the beginning of August this year fail2ban has > reported 4500 blocks! Yes, generally using fail2ban does not make you less attractive as a target but it reduces the number of passwords one can try per time. badips.com is very interested in your data. I'm working on a f2b integration as proposed by Daniel Black for that matter. > I installed f2b because of a sudden upsurge in the number of (target > unspecified) hits. These are not attempts at SSH, for which there are > very much better strategies for combating brute force password attacks. True, but sometimes you can not use better techniques. But in general I'd recommend Keys for ssh, google auth and so forth. But f2b can also help a bit to keep your logs cleaner. > I'm also seeing a huge and probably related amount email containing what > are pretty obviously bot infections. These should be investigated. I'm > now too old and tired to engage directly. ;-) For this purpose, I'd be happy to get your data. I believe at some point the logfiles of a system would be more helpful than just the IPs, but for now I do not have the infrastructure to parse logs. > Combating botnets is a very serious business. If your definition of > "fun" can be translated as an intellectual challenge then I agree, but > fun, per se, it definitely is not. "fun" for me means that I'm not paid for my work on badips.com. But it doesn't mean that I'm not willing to do it properly. > Right now, we're loosing the WAR, for that's what it is. Yes, and as you said before, I do not believe badips.com can change that. But it can help to understand whats going on. > In the sense that we might use them for blocking, I don't think > published bad IP addresses serve any useful purpose for everyday > sysadmins. To the contrary we could be telling bot controllers which > machines to abandon quietly! This is a risk, still, these machines would be quiet then - for anybody, not just users of badips.com blocklist. > On the other hand, to some research project the data could be very > useful indeed. To the extent that if it becomes a successful weapon in > the WAR then we are very likely to become casualties if we under > estimate to enemy. It's protection and research in one I believe. > On reflection, I think publishing a script to log IP is well motivated, > but naive. (or an attempt!!) Without a clear disposition of the goals > of the project. More importantly, a clear definition of the protection > mechanism of the sources and distributions of the, undoubtedly 'very > useful' data, is a prerequisite. I'll update the website as soon as I have a formulated and clear goal. > Then prove it? Simple, propose a trust mechanism. My trust mechanism is an algorythm that rates "bad" IP addresses and reporters so I can tell the "score" of an IP (how likely it is that a certain IP really is bad). The algorythm is not yet in place and requires some more data to work. I think the trust in badips.com should come over time, since people should notice that my intetions are not evil. Otherwise I can live with people not trusting me, I can understand that. If you have a good and working mechanism, I'll be happy the hear about. > If I can see that I am contributing to a meaningful project I will, in > advance, provide you with a solid means of identifying me. Provided that > I can initially acquire some degree of trust in such a project. Unfortunately I can not promise you that badips.com will ever be meaningful. I can tell you that I'm trying to establish it, but I believe I need you guys for this as well. (tell the people that it exists, write tutorials and so on) > My original post was directed to 'mr51m0n', perhaps for some perfectly > good honest reason, he has yet not responded! > > I wrote some more stuff here which I decided not to post. I've said > enough. If it's not meant for public, you can send it to me directly, use the gpg key in the badips.com about section if you need to encrypt it. R, Amy |
From: <al...@ma...> - 2013-09-25 13:29:27
|
Hi Amy, > From am...@ba... > I have a contribution to make: I have the logs of many thousands of > offending IPs. > "fun" for me means that I'm not paid for my work on badips.com. > But it doesn't mean that I'm not willing to do it properly. You might be interested to know that on 24 November 2011, the Court of Justice of the EU (the "ECJ") confirmed that IP Addresses are Personal Data (Source: http://www.mondaq.com/x/162538/Copyright/ECJ+Confirms+That+IP+Addresses+Are+ Personal+Data ) Few years prior to that judgement relevant EU regulator held the same view (Source: http://www.washingtonpost.com/wp-dyn/content/article/2008/01/21/AR2008012101 340.html ) I think it's very relevant to your activities because - "Under EU law, personal data can only be gathered legally under strict conditions, for a legitimate purpose. Furthermore, persons or organisations which collect and manage your personal information must protect it from misuse and must respect certain rights of the data owners which are guaranteed by EU law." (Source: http://ec.europa.eu/justice/data-protection/ ) You might also want to consider that making a public database of personal data qualified by you as "bad" may lead to contentious issues related to defamation. It might therefore be prudent for you to take legal advice about your planned activities. Best regards, Alex Chudnovsky Managing Director Email : al...@ma... Web : http://www.majestic12.co.uk Majestic-12 Ltd (t/a Majestic-SEO) Faraday Wharf, Holt Street Birmingham Science Park, Aston Birmingham, B7 4BB United Kingdom |
From: <am...@ba...> - 2013-09-25 13:55:18
|
Hi Alex > You might be interested to know that on 24 November 2011, the Court of > Justice of the EU (the "ECJ") confirmed that IP Addresses are Personal Data I'm ideed. This could really be a problem. How do they do this at projecthoneypot.org or any site similar to that? > "Under EU law, personal data can only be gathered legally under strict > conditions, for a legitimate purpose. Furthermore, persons or organisations > which collect and manage your personal information must protect it from > misuse and must respect certain rights of the data owners which are > guaranteed by EU law." (Source: http://ec.europa.eu/justice/data-protection/ > ) I wonder if badips.com has a "legitimate purpose"? Do you think it helps to put the service into another country? Anyone having experience with that? Thanks, Amy |
From: <al...@ma...> - 2013-09-25 14:23:05
|
Hi Amy, > From am...@ba... [mailto:am...@ba...] > How do they do this at projecthoneypot.org or any site similar to that? My guess is that they did not think it through. This sort of thing always happens with vigilante activities which typically focus on trying to punish as many suspects as possible even when it results in collateral damage. > I wonder if badips.com has a "legitimate purpose"? You could argue that, but it would still require to obey full extend of the EU data protection laws, say how would you ensure data is protected from misuse after making complete list public? > Do you think it helps to put the service into another country? I am not a lawyer but I don't think that it would provide the kind of immunity that you expect. Either way defamation laws are present in most countries of the world. Alex ---------------------------------------------------------------------------- - October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk _______________________________________________ Fail2ban-users mailing list Fai...@li... https://lists.sourceforge.net/lists/listinfo/fail2ban-users |
From: Charles B. <br...@br...> - 2013-09-25 20:38:52
|
Hi All, On Wed, 2013-09-25 at 15:22 +0100, al...@ma... wrote: > Hi Amy, > > > From am...@ba... [mailto:am...@ba...] > > How do they do this at projecthoneypot.org or any site similar to that? > > My guess is that they did not think it through. We are thinking about the problem. > > This sort of thing always happens with vigilante activities which typically > focus on trying to punish as many suspects as possible even when it results > in collateral damage. A vigilante is a person taking the law into their own hands. I don't see any vigilantes here. Just people wanting to do something about botnet attacks on their servers. > > > I wonder if badips.com has a "legitimate purpose"? The motivation is legit, it's just that the thinking is way off. > > You could argue that, but it would still require to obey full extend of > the EU data protection laws, say how would you ensure data is protected > from misuse after making complete list public? > > > Do you think it helps to put the service into another country? NO. If you're deemed legally responsible and you come under the jurisdiction of the court it's you I'm going to sue for my loss of business if my IP gets published on badip.com not your web host in the USA. > > I am not a lawyer but I don't think that it would provide the kind of > immunity that you expect. Definitely it would not. I'm not a lawyer either. However, I doubt any jurisdiction in the world would convict me for privately gathering infected IP data for the purpose of designing mechanisms to defend my property from illegal abuse attacks. The key word here is privately. The legality of publishing such data is irrelevant. Actually, it's just plain foolish to do so. And would, in any case, be counter productive in a number of ways, including probable collateral damage. > > Either way defamation laws are present in most countries of the world. Good advice, thanks. Charles Bradshaw > > > ---------------------------------------------------------------------------- > - > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most > from the latest Intel processors and coprocessors. See abstracts and > register > > http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users > > > > ------------------------------------------------------------------------------ > October Webinars: Code for Performance > Free Intel webinars can help you accelerate application performance. > Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from > the latest Intel processors and coprocessors. See abstracts and register > > http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk > _______________________________________________ > Fail2ban-users mailing list > Fai...@li... > https://lists.sourceforge.net/lists/listinfo/fail2ban-users |
From: <al...@ma...> - 2013-09-25 20:54:34
|
> From: Charles Bradshaw [mailto:br...@br...] > I doubt any jurisdiction in the world would convict me for privately gathering infected > IP data for the purpose of designing mechanisms to defend my property from illegal abuse attacks. > The key word here is privately. I'd agree - private collection of IPs accessing your own system is unlikely to be challenged (or known in the first place) since attempts to connect to one's system were made from those IPs in the first place. Selling this list or just giving it away for free might be another matter though, at least in EU. Alex |
From: <am...@ba...> - 2013-09-23 09:47:50
|
> Having the badips.com as a action to fail2ban and properly part of the > repository would be better. glad to hear that. > Unlike what is documented on the site you shouldn't add another ban > command to the iptables-multiport, it should be its own command with > just a actionban specified. The jail used should specify multiple > actions rather than merging two command sets that are for different > purposes. The key retrieval doco can exist as a comment in the > action.d/badips.com.conf so that way it doesn't depend on multiple sites > to stay active. I'll adapt my config and let you know as soon as I have something. > If you want to put this together as a github pull request we can get it > incorporated properly. > > If the badips.com is just a way to graph data that's fine but the site > probably needs a little more definition as to what's it goal is. One can also get IP blacklists and inegrate them into their firewall. See http://www.badips.com/apidoc. For example, do this: http://www.badips.com/get/list/ssh/0?format=ipset to receive a ipset formatted list of all IPs currently listed in the badips.com DB. But you are right, the website should define itself better, I'm working on that. > > Thanks for coming to us about fail2ban work that has been done. Its > certainly better than finding bug reports and patches on random blogs > years later. Thanks for you help! R, Amy |