Menu
▾
▴
os-sim-devel — Developers
You can subscribe to this list here.
| 2003 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
(1) |
Sep
(8) |
Oct
|
Nov
|
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2004 |
Jan
|
Feb
|
Mar
|
Apr
(4) |
May
(7) |
Jun
|
Jul
(1) |
Aug
|
Sep
(1) |
Oct
(2) |
Nov
(2) |
Dec
(1) |
| 2005 |
Jan
|
Feb
(3) |
Mar
(2) |
Apr
|
May
|
Jun
|
Jul
|
Aug
(3) |
Sep
(6) |
Oct
(8) |
Nov
|
Dec
|
| 2006 |
Jan
(4) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(3) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2007 |
Jan
(1) |
Feb
|
Mar
|
Apr
(2) |
May
|
Jun
|
Jul
|
Aug
(4) |
Sep
|
Oct
|
Nov
(2) |
Dec
|
| 2008 |
Jan
(4) |
Feb
(2) |
Mar
(3) |
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2010 |
Jan
|
Feb
(1) |
Mar
|
Apr
(3) |
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Jan-Oliver W. <ja...@in...> - 2005-02-17 14:31:43
|
Hi David, [ I am sending this to the devel list now since I do not want to bother you alone with my reports ] only today I found the time to give OSSIM a new try. [14:05] I am stating with a fresh and clean Debian Sarge as of today. On Tue, Feb 01, 2005 at 05:31:38PM +0100, David Gil wrote: > El mié, 26-01-2005 a las 12:49 +0100, Jan-Oliver Wagner escribió: > > On Tue, Jan 25, 2005 at 03:30:24PM +0100, David Gil wrote: > > > Please, don't use that manual, it's deprecated. Please use this > > > (http://www.ossim.net/docs/INSTALL.Debian.quick.txt) instead. > > > > hm. Would be good to take the other stuff offline then. > > Yes, we pretend to update the doc, but in the meantime may be better to > notice that... > > > > With this manual I hope you can install OSSIM in less than 1 hour. > > > > OK, lets see [12:00] ... ;-) > > > > - here also is missing "apt-get update" > > I think it's obvious.. Well, people not used to use Debian stumble across this. I observed this multiple times. I am leaving out the Performance section. > > - while doing "apt-get install ossim-mysql" I am asked: > > > > | Create the database structure now, using the following commands: > > | cd /usr/share/doc/ossim-mysql/contrib/ > > | zcat create_mysql.sql.gz ossim_*.sql.gz | mysql ossim -p > > | zcat create_snort_tbls_mysql.sql.gz \ > > | create_acid_tbls_mysql.sql.gz | mysql snort -p > > | Use -u and -h mysql options if you need to specify a non-default user > > | and host. > > | After you created the database structure, press 'ok' to continue. > > > > Unfortunately, /usr/share/doc/ossim-mysql/contrib/ does not exist! > > (nor does /usr/share/doc/ossim-mysql). > > Mmmm, it's seems that /usr/share/doc/ossim-mysql is created after > debconf execute... I need to change de debconf template. the problem is still there. > Type: > dpkg -L ossim-mysql polynoe:~# dpkg -L ossim-mysql /. /usr /usr/share /usr/share/doc /usr/share/doc/ossim-mysql /usr/share/doc/ossim-mysql/contrib /usr/share/doc/ossim-mysql/contrib/create_mysql.sql.gz /usr/share/doc/ossim-mysql/contrib/create_pgsql.sql.gz /usr/share/doc/ossim-mysql/contrib/ossim_config.sql.gz /usr/share/doc/ossim-mysql/contrib/ossim_data.sql.gz /usr/share/doc/ossim-mysql/contrib/realsecure.sql.gz /usr/share/doc/ossim-mysql/contrib/snort_nessus.sql.gz /usr/share/doc/ossim-mysql/contrib/create_snort_tbls_mysql.sql.gz /usr/share/doc/ossim-mysql/contrib/096-to-097.sql.gz /usr/share/doc/ossim-mysql/contrib/097-to-098.sql.gz /usr/share/doc/ossim-mysql/contrib/create_acid_tbls_mysql.sql.gz /usr/share/doc/ossim-mysql/changelog.gz /usr/share/doc/ossim-mysql/INSTALL.gz /usr/share/doc/ossim-mysql/copyright /usr/share/doc/ossim-mysql/changelog.Debian.gz I do not understand the item "Edit /etc/mysql/my.cnf and modify the "bind-address" entry if you want MySQL will listen on port TCP-3306 after restart." so I did not change the file. apt-get install ossim-server There is still the wrong text in one dialog which says "enter database" but actually a username must be entered. apt-get install ossim-agent prompting 127.0.0.1 and simply saying not to use it is a bit vague. Better make proposals or explain the situation in more detail. apt-get install ossim-framework There is a dialog saying: NOTE: Manual configuration required You will need to go to http://localhost/acidlab first to force the database modifications for ACIDlab. It is also advised that you run this either over HTTPS or with some form of access control on the webserver. We do not attempt to install using either technique. Your installation description should say whether this is important for ossim or not. However, the command lynx http://localhost/acidlab does not work anyway. There seems to be no server listening at this point of time. oops, and again a dialog asks for database but should for a username. the guide says "- Edit the phpgacl configuration by hand at /etc/ossim/framework/ossim.conf. Debconf management is incoming.." but to my opinion the file is configured correctly already. > > OK, again I will start ignoring ... :-( > > > > - while doing "apt-get install ossim-framework": > > > > |... > > | Creating config file /etc/apache/modules.conf with new version > > | > > | Setting up ossim-framework (0.9.7+cvs20050125-1) ... > > | Package `apache' is not installed and no info is available. > > | Use dpkg --info (= dpkg-deb --info) to examine archive files, > > | and dpkg --contents (= dpkg-deb --contents) to list their contents. > > | > > | Setting up fontconfig (2.2.3-4) ... > > |... > > It's fixed in 0.9.8rc1. confirmed. > > sounds strange, but I will ignore. > > > > time is [12:22] > > > > - "Go to http://yourhost/phpgacl/setup.php to insert the tables in the > > database." > > > > well: > > telnet localhost 80 > > Trying 127.0.0.1... > > telnet: Unable to connect to remote host: Connection refused > > > > oh, it is https ... > > > > > - doing the "lets get started" (I guess thats what I should do?): > > Once phpgacl is configured, go to http://yourhost/ossim/. You do not > need to enter to phpgacl admin page.. OK. Time is now 14:38. Have to stop for some minutes. 15:02 - proceeding. I entered http://yourhost/ossim/ and first was asked to do something about phpGACL. I did so and it seemed to be success. However, the resulting web page says "*IMPORTANT* Please make sure you create the <phpGACL root>/admin/templates_c directory, and give it write permissions for the user your web server runs as." But that is already OK, so the hint is superfluous. > > Warning: file(../CREDITS): failed to open stream: No such file or > > directory in /usr/share/phpgacl/admin/about.php on line 73 > > > > Warning: implode(): Bad arguments. in > > /usr/share/phpgacl/admin/about.php on line 73 > > phpGACL > > > New phpgacl improvements have been introduced today, can you test with > the new phpgacl-3.3.4 package? confirmed to not appear any more. [15:15] Great, I have now a management interface in my browser running. Some error messages here and there, but I will look into this later. > Sorry my poor english. You english is very good. Thanks for your answers. Best Jan -- Jan-Oliver Wagner http://intevation.de/~jan/ Intevation GmbH http://intevation.de/ FreeGIS http://freegis.org/ |
|
From: Michael B. <mic...@gm...> - 2004-12-23 11:15:07
|
Hello and merry x-mas everyone, For those who wonder when a new version of SIM ISO will be, it's "real soon now". The reason why it is taking such a long time is because the next version will be based on CentOS 3, a RHEL 3 clone. Fedora is moving too fast to do proper testing, and they discontinue the updates almost as soon as the new version is out, while RHEL 3 (and therefore CentOS 3) is supported until 2008. This means that there will be fewer components to check between each release. Of course, changing base distribution isn't exactly painless - the original build script are very Fedora centric (even if FC is a RH sponsored RH clone, it is not just plug-and-play) and it takes some time to re-write and double-check all the processes. The missing component right now is the core OSSIM packages, and that is the real reason why I write this email: I need to get hold of the .src.rpm (or the RPM build script) so I can re-compile OSSIM and it's dependencies against CentOS 3. I would greatly appreciate if anyone could upload or email me the .src.rpm and/or build script so I can feed my auto builder I just finished building ;) Best regards Michael Boman |
|
From: michael p. <dad...@ya...> - 2004-11-02 13:26:32
|
Hi all, Firstly, thank you for this excellent project. I do not wish to step on anybodies toes but I have been using Inprotect (http://inprotect.no1.com.au/), and thought that it would be good for you guys to consider incorporating it, (it using a mysql backend). Thanks again!! --------------------------------- ALL-NEW Yahoo! Messenger - all new features - even more fun! |
|
From: Dominique K. <dk...@os...> - 2004-11-02 11:51:56
|
Hi Francisco, thank you very much, we'll try the script as soon as we get the hands=20 on a suse. The author you are referring too is Ken Gregoire and you can=20= find his ossim related docs here:=20 http://www.mybizguard.com/linux/ossim/. Yup, the inclusion of the db configuration in 0.9.7 wasn't appropriate=20= after the two release candidates but since we're going to stick to that=20= form of configuration we wanted to introduce it as soon as possible. My=20= apologies for any inconvenience. And last but not least, the mailing list. He creado os-...@li... para discusi=F3n en=20 espa=F1ol de cualquier asunto relacionado con ossim. Estar=E1 activa = dentro=20 de 24 horas aprox. Greetings, Dominique Am 30.10.2004 um 15:56 schrieb Francisco Pereira: > Hi list. > > First, I want to thanks the ossim team for their ideas and their work. > > I attached a script that does basic setup for os-sim (0.9.7rc2 &=20 > 0.9.7) in suse 9.1. The goal it to make the installation of ossim more=20= > automatically and less complex. > The script is based on the doc ossim7fc2.txt (I lost the url and the=20= > author of the doc, and i need it to give the proper credits in the=20 > script) > Please let me know your opinions. The script needs modifications to=20 > work with other distributons (this version is a concept-proof, and=20 > only works for suse 9.1) and better regexps. > I have also made some ossim rpms for suse 9.1 that works for me, but=20= > I'm not sure if its a good idea to release them, because they need=20 > more testing. > > I have some patches for ossim to have a better suse 9.1 integration (I=20= > prefer ossim to be distribution neutral, or at least more distribution=20= > "configurability"). > I have to review them before posting because there are changes between=20= > 0.9.7rc2 and 0.9.7. I didn't understand why so many changes and new=20 > features between a release candidate and a final version. I think that=20= > is not good and productive to manage the releases in that way. I would=20= > expect that there are few changes (only bugfixes) from a rc and a=20 > final version. > > Regards, > Francisco. > > PD: =BFNo hay listas de ossim en espa=F1ol? Obviamente prefiero = espa=F1ol ;-) > <setup_scripts_suse91.tar.gz>= |
|
From: Francisco P. <fpe...@lo...> - 2004-10-30 13:55:29
|
Hi list. First, I want to thanks the ossim team for their ideas and their work. I attached a script that does basic setup for os-sim (0.9.7rc2 & 0.9.7)=20 in suse 9.1. The goal it to make the installation of ossim more=20 automatically and less complex. The script is based on the doc ossim7fc2.txt (I lost the url and the=20 author of the doc, and i need it to give the proper credits in the script= ) Please let me know your opinions. The script needs modifications to work=20 with other distributons (this version is a concept-proof, and only works=20 for suse 9.1) and better regexps. I have also made some ossim rpms for suse 9.1 that works for me, but I'm=20 not sure if its a good idea to release them, because they need more testi= ng. I have some patches for ossim to have a better suse 9.1 integration (I=20 prefer ossim to be distribution neutral, or at least more distribution=20 "configurability"). I have to review them before posting because there are changes between=20 0.9.7rc2 and 0.9.7. I didn't understand why so many changes and new=20 features between a release candidate and a final version. I think that=20 is not good and productive to manage the releases in that way. I would=20 expect that there are few changes (only bugfixes) from a rc and a final=20 version. Regards, Francisco. PD: =BFNo hay listas de ossim en espa=F1ol? Obviamente prefiero espa=F1ol= ;-) |
|
From: Michael B. <mic...@bo...> - 2004-10-09 15:26:16
|
As the DNS server for ossim.net seems to have some problem I've taken the liberty to create an alternative name for www.ossim.net: ossim.dnsalias.net. You can use this name if the DNS for ossim.net is down. It points to the same machine, so it won't help if the machine is down. People with 200 Mb of storage and bandwidth to spare could contact DK to see if we could setup a few mirrors... Best regards Michael Boman -- Michael Boman <mic...@bo...> BOSECO Internet Security Solutions - http://www.boseco.com |
|
From: raycounterman <ray...@co...> - 2004-09-22 00:56:36
|
paul cannon paul-ossim <mailto:ossim%40xserve.flids.com?Subject=%5BOSSIM%5D%20OSSIM%20License&In-Re ply-To=4BCCC97C-E187-11D8-BAA8-000A95D2C684%40mac.com> at nafpik.com provided some useful work on the issue of licensing (Thu Jul 29 15:39:29 EDT 200) I appreciate the effort and am asking if there has been further work on what is licensed, under what terms, and the constraints on development for sale of a product using OSSIM. Regards, Ray Counterman |
|
From: Michael B. <mic...@bo...> - 2004-07-09 07:00:24
|
For you who has been asking if there are any new version of SIM being released anytime soon can now rejoice to the news that SIM ISO 0.9.6-1 has been released and can be downloaded from the download section at http://www.boseco.com. Please report any bugs found to the SIM Development & Bug reports forum. Please visit http://www.boseco.com for more information and download. Best regards Michael Boman -- Michael Boman <mic...@bo...> BOSECO Internet Security Solutions - http://www.boseco.com |
|
From: <ben...@id...> - 2004-05-22 13:02:41
|
Dear Open Source developer I am doing a research project on "Fun and Software Development" in which I kindly invite you to participate. You will find the online survey under http://fasd.ethz.ch/qsf/. The questionnaire consists of 53 questions and you will need about 15 minutes to complete it. With the FASD project (Fun and Software Development) we want to define the motivational significance of fun when software developers decide to engage in Open Source projects. What is special about our research project is that a similar survey is planned with software developers in commercial firms. This procedure allows the immediate comparison between the involved individuals and the conditions of production of these two development models. Thus we hope to obtain substantial new insights to the phenomenon of Open Source Development. With many thanks for your participation, Benno Luthiger PS: The results of the survey will be published under http://www.isu.unizh.ch/fuehrung/blprojects/FASD/. We have set up the mailing list fa...@we... for this study. Please see http://fasd.ethz.ch/qsf/mailinglist_en.html for registration to this mailing list. _______________________________________________________________________ Benno Luthiger Swiss Federal Institute of Technology Zurich 8092 Zurich Mail: benno.luthiger(at)id.ethz.ch _______________________________________________________________________ |
|
From: Michael B. <mi...@ay...> - 2004-05-10 05:42:57
|
I am proud to announce that there is now a FTP and HTTP mirror of the ISO. It will take a little while more before a new version is released due to other commitments. Please report the bugs using the forums. All this is at http://www.boseco.com Downloads and Forums are accessible using the menu to the left. --=20 Michael Boman |
|
From: Yoann V. <yo...@pr...> - 2004-05-05 17:51:37
|
On Mon, 2004-05-03 at 21:09 +0200, DK wrote: [...] Hi, > The main problems I see for an easy integration are: > > - Monitor querys > Querying various ntop/opennms aspects is very important for us and > the correlation directives since they provide very useful input. Theses application could easily send alert to a prelude-manager, throught a script or by modifying the application directly, which don't allow you to query theses applications directly, but at least, you get the ability ti get the latest reported status of theses applications. > - Remote process status > We are able to start/stop, enable/disable plugins from within the > web-ui. The request goes through the server and the agent monitor > processes, stops and starts them and so on. Same as above, plus you should soon have the ability to control a Prelude sensors throught the administrative console (check: http://article.gmane.org/gmane.comp.security.ids.prelude.devel/65/match=administrative) > - P0f & arpwatch integration > It seems prelude p0f integration is a little bit outdated and I'm not > sure how we could get the ip-mac listing. (Prelude NIDS could provide > this but I think arpwatch is a little bit more, err, "lightweight"). I'm not sure what you mean with "ip-mac listing". If it's simply accessing it, it is in the database. > - DB querys > Our agent is able to query various things from remote DBs such as > risk levels, availability status, etc... libpreludedb implement that too... > - Missing agents > There are a couple of agents that would need to be ported to prelude > format because we rely heavily on them too (rrd anomaly, rrd > threshold). > > --- > > Server side problems. > > This is the greatest hurdle I see. Our server already implements a > strong DB integration writing both into ossim's own and acid/snort's > tables. We use a modified version of acid that provides risk, priority, > reliability and asset data and I think that will continue to be our > main event viewing platform. The server is able to log almost every > normalized data input into acid so we use it to show fw-1, iptables, > IIS, realsecure, p0f, etc... alerts too, each one with it's own > reliability, asset, etc. info. > Prelude also logs it's event into DB so, what could be done here ? This > is proving to be the most difficult thought. I'm drawing some visio > like diagrams in order to make it easier to discuss the matter. The > graphics on our site are outdated as is the roadmap (1.0 & 2.0 are > mixed alltogether and we haven't even reached 1.0). Implementing the database querying part shouldn't be too hard, thanks to libpreludedb. Basically the main problem would just be to enable ossim to have 2 differents database backends, and to arrange for every data available in an IDMEF message to be shown by ossim. -- Yoann Vandoorselaere <yo...@pr...> |
|
From: DK <dk...@os...> - 2004-05-03 19:10:57
|
Hi Gene, After reading the replies I think pros and cons is little misleading. As I delve deeper into prelude I like it more and more and it ratifies my first impression: ossim can learn a lot from it. So rather than pros and cons it should say "possible problems / hurdles such and integration would mean", mainly modifications that would need to be done to both. Ah, most of my impressions are based on the Prelude Architecture Guide. From ossim's point of view, I think the main benefits would be: - IDMEF communication. The protocol we use for internal communication (sensor-server, server-webui) is similar, and now I regret not implementing IDMEF right from the start. We were too lazy... - SSL. Until today we needed to tunnel everything through an external ciphered layer or run all the components on a trusted network. I think libprelude solves this in a good way. - Failover No effort has been made yet by ossim to provide agent/server failover. I like the pseudo-routing done with relay managers. - Timeout and retransmission. Sync & Async. That's something we already have implemented but again, I think libpreludes approach looks robuster. - Native syslog server. If I did understand it right prelude-lml implements a pseudo syslog server (only processing logs, not storing them). That's a feature we needed too, i.e. for fw-1 input. - HIDS. At this time we don't accept input from HIDS and urgently need to change this. - New data input Prelude's sensor list is impressive. If we could correlate all that input... ;-) --- The main problems I see for an easy integration are: - Monitor querys Querying various ntop/opennms aspects is very important for us and the correlation directives since they provide very useful input. - Remote process status We are able to start/stop, enable/disable plugins from within the web-ui. The request goes through the server and the agent monitor processes, stops and starts them and so on. - P0f & arpwatch integration It seems prelude p0f integration is a little bit outdated and I'm not sure how we could get the ip-mac listing. (Prelude NIDS could provide this but I think arpwatch is a little bit more, err, "lightweight"). - DB querys Our agent is able to query various things from remote DBs such as risk levels, availability status, etc... - Missing agents There are a couple of agents that would need to be ported to prelude format because we rely heavily on them too (rrd anomaly, rrd threshold). --- Server side problems. This is the greatest hurdle I see. Our server already implements a strong DB integration writing both into ossim's own and acid/snort's tables. We use a modified version of acid that provides risk, priority, reliability and asset data and I think that will continue to be our main event viewing platform. The server is able to log almost every normalized data input into acid so we use it to show fw-1, iptables, IIS, realsecure, p0f, etc... alerts too, each one with it's own reliability, asset, etc. info. Prelude also logs it's event into DB so, what could be done here ? This is proving to be the most difficult thought. I'm drawing some visio like diagrams in order to make it easier to discuss the matter. The graphics on our site are outdated as is the roadmap (1.0 & 2.0 are mixed alltogether and we haven't even reached 1.0). Did you have similar thoughts? Any useful input ? Hopefully I get some time to finish the diagrams and we can talk more on this as soon as possible. Greetings, Dominique PD: Jascha, thanks for the papers, I will have a look at them tonight. The University of Leon, Spain, has some very interesting working code involving neural networks applied to IDSs. They were interested on building something on top of ossim's correlation engine but sadly we didn't have much contact these last months... Am 03.05.2004 um 18:56 schrieb Gene R Gomez: > Hi Dominique, > I'm glad to see that you're evaluating the pros and cons of > integration. > One thing that I wanted to mention before you get too heavy into it is > that my initial plan was to base any integration off of the 0-9 Prelude > branch (currently in the late stages of development), rather than 0-8. > Two reasons: > 1. While Prelude has always been oriented towards IDMEF, 0-9 is fully > IDMEF-compliant. IDMEF will become more and more important after its > final version is ratified and made an official standard for IDS data > exchange. > 2. 0-9 implements libpreludedb, a database abstraction API that makes > integration of other tools much easier. All we'd need to do is write > up > libpreludedb calls and then the API would take care of doing the > neccessary SQL manipulations so that you wouldn't have to worry about > what > backend was being used or how the schema was designed. In fact, I > think > it's likely that a libpreludedb plugin could be written for the current > OSSIM database so that future data exchange would be easier (a series > of > SELECTs from one database and INSERTs to the other made via the API > would > be easy to pull off). > Anyway, let us know if you have any additional questions, or if there > is > some way we can help with the development of your pro/con list. > > Thanks! > Gene R Gomez > >> -----Original Message----- >> From: pre...@pr... >> [mailto:pre...@pr...]On Behalf Of DK >> Sent: Monday, May 03, 2004 4:54 AM >> To: pre...@pr... >> Cc: os-...@li... >> Subject: [prelude-devel] prelude/ossim thoughts >> >> >> Hi, >> >> first of all I want to say hello since this is the first time I post >> to >> the prelude-devel list. I'm CCing os-sim-devel too. >> >> A couple of weeks ago we (at ossim.net) were approached by Gener R. >> Gomez and Krzysztof Zaraska regarding the benefits of a possible >> prelude/ossim integration. We've been quite busy in the last weeks but >> finally got some time and had a look into it. >> >> My first impression is that, as Gene stated earlier, both projects >> could >> benefit from such an integration / collaboration. Prelude >> provides a strong sensor-manager architecture, much better collection >> mecanisms than ossim and a lot of interesting features that ossim >> lacks. Ossim's focus on the other hand is the integration of tools, >> their interoperability and the presentation layer that glues >> everything >> together and, of course, the correlation stuff we're heavily working >> on. I think prelude could benefit from them too. >> >> I'm writing down all the pros and cons (and possible problems) of such >> an integration and will send it to this list as soon as possible. If >> this isn't the right place to discuss such matter please tell me >> where / >> whom to write. >> >> BTW, reading the lists archives I saw a mention to CALM. CALM, as used >> in ossim has nothing to do with http://www.kung-foo.tv/calmapi.php. We >> didn't know of the existence of the calm correlation api (but it's >> interesting read...). Ossim's CALM is a simple event accumulation >> algorithm that tries to come up with a realtime measurement of a >> hosts/nets/global risk. >> >> Greetings, >> >> Dominique >> >> _______________________________________________ >> Prelude-devel site list >> Pre...@pr... >> http://www.prelude-ids.org/mailman/listinfo/prelude-devel >> >> _______________________________________________ >> Prelude-devel site list >> Pre...@pr... >> http://www.prelude-ids.org/mailman/listinfo/prelude-devel > > |
|
From: Gene R G. <ge...@go...> - 2004-05-03 16:57:25
|
Hi Dominique, I'm glad to see that you're evaluating the pros and cons of integration. One thing that I wanted to mention before you get too heavy into it is that my initial plan was to base any integration off of the 0-9 Prelude branch (currently in the late stages of development), rather than 0-8. Two reasons: 1. While Prelude has always been oriented towards IDMEF, 0-9 is fully IDMEF-compliant. IDMEF will become more and more important after its final version is ratified and made an official standard for IDS data exchange. 2. 0-9 implements libpreludedb, a database abstraction API that makes integration of other tools much easier. All we'd need to do is write up libpreludedb calls and then the API would take care of doing the neccessary SQL manipulations so that you wouldn't have to worry about what backend was being used or how the schema was designed. In fact, I think it's likely that a libpreludedb plugin could be written for the current OSSIM database so that future data exchange would be easier (a series of SELECTs from one database and INSERTs to the other made via the API would be easy to pull off). Anyway, let us know if you have any additional questions, or if there is some way we can help with the development of your pro/con list. Thanks! Gene R Gomez > -----Original Message----- > From: pre...@pr... > [mailto:pre...@pr...]On Behalf Of DK > Sent: Monday, May 03, 2004 4:54 AM > To: pre...@pr... > Cc: os-...@li... > Subject: [prelude-devel] prelude/ossim thoughts > > > Hi, > > first of all I want to say hello since this is the first time I post to > the prelude-devel list. I'm CCing os-sim-devel too. > > A couple of weeks ago we (at ossim.net) were approached by Gener R. > Gomez and Krzysztof Zaraska regarding the benefits of a possible > prelude/ossim integration. We've been quite busy in the last weeks but > finally got some time and had a look into it. > > My first impression is that, as Gene stated earlier, both projects could > benefit from such an integration / collaboration. Prelude > provides a strong sensor-manager architecture, much better collection > mecanisms than ossim and a lot of interesting features that ossim > lacks. Ossim's focus on the other hand is the integration of tools, > their interoperability and the presentation layer that glues everything > together and, of course, the correlation stuff we're heavily working > on. I think prelude could benefit from them too. > > I'm writing down all the pros and cons (and possible problems) of such > an integration and will send it to this list as soon as possible. If > this isn't the right place to discuss such matter please tell me where / > whom to write. > > BTW, reading the lists archives I saw a mention to CALM. CALM, as used > in ossim has nothing to do with http://www.kung-foo.tv/calmapi.php. We > didn't know of the existence of the calm correlation api (but it's > interesting read...). Ossim's CALM is a simple event accumulation > algorithm that tries to come up with a realtime measurement of a > hosts/nets/global risk. > > Greetings, > > Dominique > > _______________________________________________ > Prelude-devel site list > Pre...@pr... > http://www.prelude-ids.org/mailman/listinfo/prelude-devel > > _______________________________________________ > Prelude-devel site list > Pre...@pr... > http://www.prelude-ids.org/mailman/listinfo/prelude-devel |
|
From: Jascha <ja...@lo...> - 2004-05-03 15:15:52
|
I believe you may be refering to my post regarding CALM (http://www.kung-foo.tv/calmapi.php) :) I had misunderstood and thought you all were working on something I have been trying to put together for some time now. So I jumped the gun from excitement. What I have been trying to do is use neural networks to train an IDS ('expert system'). It is more a theory than something I actually have working. But I have been very interested in computer immunology and its relation to IDSs. I have looked at many things such as cfengine (http://www.iu.hio.no/cfengine/) and its relation to such a persuit. One of the papers I have drawn a lot of the concepts from is: "Probabilistic anomaly detection in distributed computer networks" http://www.iu.hio.no/~mark/papers/anomaly.pdf As well as others: http://www.iu.hio.no/cfengine/papers.html http://www.iu.hio.no/~mark/research/immune/ http://www.cs.unm.edu/~immsec/papers.htm Just to clarify. I seem to have gotten my 'CALMs' confused. ;) I have done a lot of work with neural networks in the past in nrelation to financial forcasting (stocks etc) but am still working on getting something trainable in terms of IDSs. I look forward to your list of Pros and Cons as well. Plus I would be interested of anyones imput on what I have mentioned. Regards, Jascha jascha[at]localareasecurity.com http://localareasecurity.com -----Original Message----- From: pre...@pr... [mailto:pre...@pr...]On Behalf Of DK Sent: Monday, May 03, 2004 4:54 AM To: pre...@pr... Cc: os-...@li... Subject: [prelude-devel] prelude/ossim thoughts Hi, first of all I want to say hello since this is the first time I post to the prelude-devel list. I'm CCing os-sim-devel too. A couple of weeks ago we (at ossim.net) were approached by Gener R. Gomez and Krzysztof Zaraska regarding the benefits of a possible prelude/ossim integration. We've been quite busy in the last weeks but finally got some time and had a look into it. My first impression is that, as Gene stated earlier, both projects could benefit from such an integration / collaboration. Prelude provides a strong sensor-manager architecture, much better collection mecanisms than ossim and a lot of interesting features that ossim lacks. Ossim's focus on the other hand is the integration of tools, their interoperability and the presentation layer that glues everything together and, of course, the correlation stuff we're heavily working on. I think prelude could benefit from them too. I'm writing down all the pros and cons (and possible problems) of such an integration and will send it to this list as soon as possible. If this isn't the right place to discuss such matter please tell me where / whom to write. BTW, reading the lists archives I saw a mention to CALM. CALM, as used in ossim has nothing to do with http://www.kung-foo.tv/calmapi.php. We didn't know of the existence of the calm correlation api (but it's interesting read...). Ossim's CALM is a simple event accumulation algorithm that tries to come up with a realtime measurement of a hosts/nets/global risk. Greetings, Dominique _______________________________________________ Prelude-devel site list Pre...@pr... http://www.prelude-ids.org/mailman/listinfo/prelude-devel |
|
From: DK <dk...@os...> - 2004-05-03 09:54:33
|
Hi, first of all I want to say hello since this is the first time I post to the prelude-devel list. I'm CCing os-sim-devel too. A couple of weeks ago we (at ossim.net) were approached by Gener R. Gomez and Krzysztof Zaraska regarding the benefits of a possible prelude/ossim integration. We've been quite busy in the last weeks but finally got some time and had a look into it. My first impression is that, as Gene stated earlier, both projects could benefit from such an integration / collaboration. Prelude provides a strong sensor-manager architecture, much better collection mecanisms than ossim and a lot of interesting features that ossim lacks. Ossim's focus on the other hand is the integration of tools, their interoperability and the presentation layer that glues everything together and, of course, the correlation stuff we're heavily working on. I think prelude could benefit from them too. I'm writing down all the pros and cons (and possible problems) of such an integration and will send it to this list as soon as possible. If this isn't the right place to discuss such matter please tell me where / whom to write. BTW, reading the lists archives I saw a mention to CALM. CALM, as used in ossim has nothing to do with http://www.kung-foo.tv/calmapi.php. We didn't know of the existence of the calm correlation api (but it's interesting read...). Ossim's CALM is a simple event accumulation algorithm that tries to come up with a realtime measurement of a hosts/nets/global risk. Greetings, Dominique |
|
From: Michael B. <mi...@ay...> - 2004-04-28 09:43:00
|
Hi all, I'd just like to draw your attention to the two polls I have running at http://www.boseco.com/index.php?name=3DPNphpBB2&file=3Dindex&c=3D2 (In the development thread). Also, I would very much like feedback from those who have downloaded and tested the ISO. What is your experience with it? What, if anything, went wrong and what should be done to make it better? Finally, those of you that haven't downloaded and tested it.. May I ask "why not"? If there is any technical issues with the distribution system etc (basically anything but the "I like to roll my own" reason) I would like to know about it. --=20 Michael Boman |
|
From: Michael B. <mi...@ay...> - 2004-04-27 02:08:32
|
I am proud to announce the availability of SIM ISO 2004-04-26 snapshot. Please note that this is a beta release and as such does not have any official support, and it's main purpose is to facilitate testing of the software (installation CD as well as OS-SIM itself).=20 Please report any bugs found to the SIM Development forum.=20 -- Michael Boman=20 Full announcement at: http://www.boseco.com/index.php?name=3DPNphpBB2&file=3Dviewtopic&t=3D3 Extract: New features / bug fixes in this release:=20 - Bugfix: Default configuration for OS-SIM (/etc/ossim/*) is created at installation=20 - Bugfix: p0f / arpwatch now starts automatically with the correct parameters=20 - Update: OS-SIM is updated to latest CVS version (snapshot taken today, 2004-04-26)=20 - New: Auto-updating of Snort signatures / Nessus plugins every 6 hours=20 - New: Auto-generating random passwords for database access=20 Known bugs:=20 - If your graphics card can't be detected properly and you end up in console (text) mode, the default root password is "password".=20 - Snort fails to start if network (eth0) isn't configured (or fail to start for some reason). - "firstboot" resets the default firewall rules. - The URL to nTop is not correct --=20 Michael Boman |
|
From: DK <dk...@os...> - 2004-04-01 10:10:20
|
Thanks Tyler,
we noticed that too with the latest nessus plugins. Fixed in 0.9.4,
thanks for the hint.
Greetings,
Dominique
Am 01.04.2004 um 05:51 schrieb Tyler:
> In trying to get the nessus integration running I ran across an error
> in
> the update_nessus_ids.pl script. The error was in the
> "12033:LeifWright's blog.cgi command execution" entry. So what I did
> was add the following to line 59:
>
> $plugin_rel_hash{$key}=~tr/'//d;
>
>
> Thanks,
> Tyler
>
>
>
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IBM Linux Tutorials
> Free Linux tutorial presented by Daniel Robbins, President and CEO of
> GenToo technologies. Learn everything from fundamentals to system
> administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
> _______________________________________________
> Os-sim-devel mailing list
> Os-...@li...
> https://lists.sourceforge.net/lists/listinfo/os-sim-devel
>
|
|
From: Tyler <ty...@sc...> - 2004-04-01 03:51:44
|
In trying to get the nessus integration running I ran across an error in
the update_nessus_ids.pl script. The error was in the
"12033:LeifWright's blog.cgi command execution" entry. So what I did
was add the following to line 59:
$plugin_rel_hash{$key}=~tr/'//d;
Thanks,
Tyler
|
|
From: <dk...@ip...> - 2003-09-30 11:06:04
|
Hi, sorry for the late reply. On Sun, Sep 28, 2003 at 06:48:34PM -0400, Jose Vicente Nunez Zuleta wrote= : > Hi, >=20 > Je je, this is getting lengthy but lets talk :) >=20 > On Sun, 2003-09-28 at 10:03, DK wrote: > > Hi, > >=20 > > El 9/27/03 16:46, "Jose Vicente Nunez Zuleta" <jo...@ne...> > > escribi=F3: > >=20 > > Both issues are going to be solved when we release the roadmap (I don= 't know > > when, hopefully next week) so we can provide the following: > >=20 > > - In depth system architecture. > > - What to expect from updates until 1.0 is released. What we want to = do > > until 2.0. When to expect all of this. > > - Tasks for ourselves as well as for anyone wishing to help with the > > project. > >=20 >=20 >=20 > That sounds great, this will help a lot of people to cooperate with the > project. >=20 >=20 > > Interesting indeed. Why do you think it would be useful to integrate = jabber > > ? >=20 > Jabber is platform neutral, OpenSource and can be used for: >=20 > 1) Monitoring in real time. Just send the messages to Jabber and with > any client you can have them displayed in real time. Instead of getting > hundred of nasty emails address (that you will delete after wards) you > can keep watching the messages on a window.=20 > 2) You don have to reinvent the wheel for a messaging platform, it is > already there. > 3) Everybody has a IM client and a web browser. This will cut developin= g > costs. I think it's and interesting addon but far away from our main focus. As o= f today we have enough work to maintain the web interface ;) >=20 > > Besides, our intention is to move as many things we can away from sys= log > > because syslog (or syslog-ng, as you want) slows things down. We have= to > > speed everything up, speed it a lot up so that's our main focus as of= today. > >=20 >=20 > What do you mean with Syslog slowing things down? Syslog can become th= e > main entry point of messages to this system and because is already ther= e > it means that you don have to deploy a custom event forwarder on every > machine. just think about this: >=20 > 1) IPTables has the option to send the reject or discard messages to > Syslog. Is easier to configure than the ulog module (wich can send > messages to a MySQL database for example). >=20 > 2) Snort can send messages to Syslog instead of emails, database > connections or even SNMP traps (SNMP takes a lot of work to get working > by the way). >=20 > 3) Syslog is available =B4as is=B4 on every Cisco, Nokia Firewall, Sola= ris > machine, Linux machine, etc. You only need to add the following to the > /etc/syslog.conf to make it happen (i mean on a Unix box): >=20 > *.* @loghost >=20 > I agree that Syslog is not secure and can be tampered but i also believ= e > than is an error to ignore it as a data source of events. On a > enterprise, you will have many data sources and Syslog is the standard > on Unix. We don't ignore it and we'll always use it but if it's possible we want t= o get the data from other sources. >=20 >=20 >=20 > > I have to draw a new architecture map, hopefully this week I can get = two > > hours to redraw it. > >=20 >=20 > Cool. I know is not easy as it sounds :) Yeah, and when you have to do your daily work too it gets even harder. >=20 > > >=20 > > > Ok. Again, i think PostgreSQL is better suited for this task (this = can be > > > discussed in detail). > >=20 > > We have to decide this issue soon. I think it is going to be easier t= o > > rewrite our C code to make use of postgresql rather than move OpenNMS= to > > MySQL. No decision taken as of today. > >=20 >=20 > ok. >=20 > > >=20 > > >> - Of course nessus-opennms integration would be done after talking= with > > >> Opennms's creators. > > >=20 > > > Great. > > >=20 > > >> - We have to write a data consolidator which accepts input from ma= ny > > >> more > > >> devices. > > >=20 > > > Syslog could be a first option, and then more could be added as 'pl= ugins'? > > >=20 > >=20 > > As stated before, we want to move away from syslog for speed reasons,= but of > > course not everything because some products are better integrated wit= hin > > syslog. And we want to correlate system events too (both unix & windo= ws). > >=20 >=20 > I=B4m not a big fan of Syslog, but hey they even have a version on Wind= ows > :) >=20 > > =20 > > > You could still use a language like Python to do the scripting part= ; Is easier > > > to extend in C than Perl (you don't need > > > Swig for that). Supports objects better than Perl, etc. Also if you= wanna use > > > Jython instead of Python that makes it easier > > > to glue it together with Java. > > >=20 > > > I would use C only for tasks that require the speed, i think Java i= s better > > > suited as the main languaje of the application. > >=20 > > Personally I don=B9t like Java although this opinion is changing late= ly. The > > scripting part is only a temporary solution because every single comp= onent > > needs to run as fast as possible so C is going to be used for the mai= n core. > > Around that we can build over every language that suits our needs. Ja= va is > > going to be an alternative, that for sure, but we will also use PHP, = Perl > > and why not, python is also an alternative (I love python). > >=20 >=20 > I think Java is a strong languaje on the server side. Is portable (so > the system can be run even on Windows, for the benefit of the poor > Windoooze server out there :)). The (lack of) speed of Java is a myth > that can give us a lot to talk for days, but here are some things for > you to think about it: >=20 > 1) Java offers more facilities for rapid development. You have to admit > than working on C/C++ will require a lot of third party libraries that > already come with Java. Database connectivity (JDBC), XML management > 2) This application will be doing SQL queries and showing reports to th= e > user, rigth? The data capture is left to sensors like Snort, the networ= k > discovery to tools like OpenNMS (written almost entirely in Java) so i > don see how writing the whole app in C will give you more speed while > the database engine and probably PHP (or JSP) will be showing the data > back to the users. >=20 > I think other aspects should be considered besides speed, like how fast > you can develop the application, reusable components, scalability, etc. >=20 > Let the flame war begins :D. Nah, no flame war. Your opinions make sense and our development is tendin= g to a similar architecture: - C core. - Php (or anything else) for the presentation layer. - Few misc scripts written in whatever (python, sh, perl, etc...) >=20 >=20 > > > But again, what sense it makes to have two applications discovering= nodes at > > > the same time? OpenNMS discovery capabilitites > > > can be extended easily using Java plugins and the Assest database i= s fairly > > > complete; I think the goal of this project > > > should be focus on how to analize all the data gathered by OpenNMS,= Snort, > > > Syslog instead of replicate > > > the polling and discovery functionality. > >=20 > > Perhaps my lack of knowledge of the inner workings of OpenNMS mislead= s me, > > but I think for some small specific functions we should better rely o= n > > specific programs, as with Ntop. > >=20 >=20 > Don=B4t worry, lets go one step at the time. >=20 > > How does opennms's service detection work ? Does it rely on port numb= er or > > does it make some checks to ensure port 53 is DNS indeed (for example= ) ? Is > > this work based on existing applications like amap or is it a complet= e new > > write up in Java ? > >=20 >=20 > OpenNMS has two daemons: capsd and polld. Capsd detects if a host has a > given service running, while polld checks a service previously detected. >=20 > And here is when OpenNMS kicks nmap ass, because OpenNMS can run > synthetic transactions so it can really interact with an application to > see if it really leaves there, is not just a plain portscan (which can > be misleading). Do you know amap ? (I think it's from THC) >=20 > For example, the JDBC poller i wrote for OpenNMS does that: talks with > the database using JDBC and ask for the database metadata, so is a real > query going on there. Also i wrote a custom plugin for my company that > parses an XML and based on that detects is we are running a web service > on a given box. >=20 > So far, the best article i=B4ve seen on OpenNMS is here: >=20 > http://www-106.ibm.com/developerworks/java/library/j-jmx3/ I'll read it. >=20 > Read it and you will master the basics. >=20 >=20 > > What about host and port discovery ? Does it use nmap or scanrand or > > something similar or is it written up from 0 ? > >=20 >=20 > Nope, you tell it what you expect to look on the servers and thats it. > Also, doing a portscan on services that can crash (because they are > poorly written) is not a good idea. The plugin concept of OpenNMS doesn > use nmap nor scanrand at all. >=20 > > If you have tried the program out you'll have noticed that OpenNMS > > integration at this point is minimal. Only one link from the main pag= e but > > not as tight integrated as ntop or rrd for example. > >=20 >=20 > OpenNMS can use rrdtool (just check the SNMP graphics for example). You > can customize it too (check on the OpenNMS page a short tutorial i wrot= e > about how to generate custom graphics on OpenNMS with rrdtool). >=20 >=20 >=20 > > One of the first items in my TODO list is to dissect OpenNMS's code s= o to > > know exactly what can be used where, but as always, time, time, time.= .. > >=20 >=20 > Check the archives of the list or even better, why you don=B4t ask some > questions on the developers list? When I get to 100% OpenNMS integration I surely will do. At this time I can't tell you much more, I haven't done almost anything t= owards getting a decent roadmap because I'm introducing heavy parser modi= fications, I'll release them on friday probably. But I promise to get tha= t damn roadmap written up as soon as possible so we can discuss real deve= lopment issues. Greetings, DK |
|
From: <fo...@ip...> - 2003-09-29 18:54:32
|
Os-sim-devel -- confirmation of subscription -- request 856328 We have received a request from 213.37.34.92 for subscription of your email address, <fo...@ip...>, to the os-...@li... mailing list. To confirm the request, please send a message to os-...@li..., and either: - maintain the subject line as is (the reply's additional "Re:" is ok), - or include the following line - and only the following line - in the message body: confirm 856328 (Simply sending a 'reply' to this message should work from most email interfaces, since that usually leaves the subject line in the right form.) If you do not wish to subscribe to this list, please simply disregard this message. Send questions to os-...@li.... |
|
From: Jose V. N. Z. <jo...@us...> - 2003-09-28 22:49:32
|
Hi, Je je, this is getting lengthy but lets talk :) On Sun, 2003-09-28 at 10:03, DK wrote: > Hi, >=20 > El 9/27/03 16:46, "Jose Vicente Nunez Zuleta" <jo...@ne...> > escribi=F3: >=20 > Both issues are going to be solved when we release the roadmap (I don't= know > when, hopefully next week) so we can provide the following: >=20 > - In depth system architecture. > - What to expect from updates until 1.0 is released. What we want to do > until 2.0. When to expect all of this. > - Tasks for ourselves as well as for anyone wishing to help with the > project. >=20 That sounds great, this will help a lot of people to cooperate with the project. > Interesting indeed. Why do you think it would be useful to integrate ja= bber > ? Jabber is platform neutral, OpenSource and can be used for: 1) Monitoring in real time. Just send the messages to Jabber and with any client you can have them displayed in real time. Instead of getting hundred of nasty emails address (that you will delete after wards) you can keep watching the messages on a window.=20 2) You don have to reinvent the wheel for a messaging platform, it is already there. 3) Everybody has a IM client and a web browser. This will cut developing costs. > Besides, our intention is to move as many things we can away from syslo= g > because syslog (or syslog-ng, as you want) slows things down. We have t= o > speed everything up, speed it a lot up so that's our main focus as of t= oday. >=20 What do you mean with Syslog slowing things down? Syslog can become the main entry point of messages to this system and because is already there it means that you don have to deploy a custom event forwarder on every machine. just think about this: 1) IPTables has the option to send the reject or discard messages to Syslog. Is easier to configure than the ulog module (wich can send messages to a MySQL database for example). 2) Snort can send messages to Syslog instead of emails, database connections or even SNMP traps (SNMP takes a lot of work to get working by the way). 3) Syslog is available =B4as is=B4 on every Cisco, Nokia Firewall, Solari= s machine, Linux machine, etc. You only need to add the following to the /etc/syslog.conf to make it happen (i mean on a Unix box): *.* @loghost I agree that Syslog is not secure and can be tampered but i also believe than is an error to ignore it as a data source of events. On a enterprise, you will have many data sources and Syslog is the standard on Unix. > I have to draw a new architecture map, hopefully this week I can get tw= o > hours to redraw it. >=20 Cool. I know is not easy as it sounds :) > >=20 > > Ok. Again, i think PostgreSQL is better suited for this task (this ca= n be > > discussed in detail). >=20 > We have to decide this issue soon. I think it is going to be easier to > rewrite our C code to make use of postgresql rather than move OpenNMS t= o > MySQL. No decision taken as of today. >=20 ok. > >=20 > >> - Of course nessus-opennms integration would be done after talking w= ith > >> Opennms's creators. > >=20 > > Great. > >=20 > >> - We have to write a data consolidator which accepts input from many > >> more > >> devices. > >=20 > > Syslog could be a first option, and then more could be added as 'plug= ins'? > >=20 >=20 > As stated before, we want to move away from syslog for speed reasons, b= ut of > course not everything because some products are better integrated withi= n > syslog. And we want to correlate system events too (both unix & windows= ). >=20 I=B4m not a big fan of Syslog, but hey they even have a version on Window= s :) > =20 > > You could still use a language like Python to do the scripting part; = Is easier > > to extend in C than Perl (you don't need > > Swig for that). Supports objects better than Perl, etc. Also if you w= anna use > > Jython instead of Python that makes it easier > > to glue it together with Java. > >=20 > > I would use C only for tasks that require the speed, i think Java is = better > > suited as the main languaje of the application. >=20 > Personally I don=B9t like Java although this opinion is changing lately= . The > scripting part is only a temporary solution because every single compon= ent > needs to run as fast as possible so C is going to be used for the main = core. > Around that we can build over every language that suits our needs. Java= is > going to be an alternative, that for sure, but we will also use PHP, Pe= rl > and why not, python is also an alternative (I love python). >=20 I think Java is a strong languaje on the server side. Is portable (so the system can be run even on Windows, for the benefit of the poor Windoooze server out there :)). The (lack of) speed of Java is a myth that can give us a lot to talk for days, but here are some things for you to think about it: 1) Java offers more facilities for rapid development. You have to admit than working on C/C++ will require a lot of third party libraries that already come with Java. Database connectivity (JDBC), XML management 2) This application will be doing SQL queries and showing reports to the user, rigth? The data capture is left to sensors like Snort, the network discovery to tools like OpenNMS (written almost entirely in Java) so i don see how writing the whole app in C will give you more speed while the database engine and probably PHP (or JSP) will be showing the data back to the users. I think other aspects should be considered besides speed, like how fast you can develop the application, reusable components, scalability, etc. Let the flame war begins :D. > > But again, what sense it makes to have two applications discovering n= odes at > > the same time? OpenNMS discovery capabilitites > > can be extended easily using Java plugins and the Assest database is = fairly > > complete; I think the goal of this project > > should be focus on how to analize all the data gathered by OpenNMS, S= nort, > > Syslog instead of replicate > > the polling and discovery functionality. >=20 > Perhaps my lack of knowledge of the inner workings of OpenNMS misleads = me, > but I think for some small specific functions we should better rely on > specific programs, as with Ntop. >=20 Don=B4t worry, lets go one step at the time. > How does opennms's service detection work ? Does it rely on port number= or > does it make some checks to ensure port 53 is DNS indeed (for example) = ? Is > this work based on existing applications like amap or is it a complete = new > write up in Java ? >=20 OpenNMS has two daemons: capsd and polld. Capsd detects if a host has a given service running, while polld checks a service previously detected. And here is when OpenNMS kicks nmap ass, because OpenNMS can run synthetic transactions so it can really interact with an application to see if it really leaves there, is not just a plain portscan (which can be misleading). For example, the JDBC poller i wrote for OpenNMS does that: talks with the database using JDBC and ask for the database metadata, so is a real query going on there. Also i wrote a custom plugin for my company that parses an XML and based on that detects is we are running a web service on a given box. So far, the best article i=B4ve seen on OpenNMS is here: http://www-106.ibm.com/developerworks/java/library/j-jmx3/ Read it and you will master the basics. > What about host and port discovery ? Does it use nmap or scanrand or > something similar or is it written up from 0 ? >=20 Nope, you tell it what you expect to look on the servers and thats it. Also, doing a portscan on services that can crash (because they are poorly written) is not a good idea. The plugin concept of OpenNMS doesn use nmap nor scanrand at all. > If you have tried the program out you'll have noticed that OpenNMS > integration at this point is minimal. Only one link from the main page = but > not as tight integrated as ntop or rrd for example. >=20 OpenNMS can use rrdtool (just check the SNMP graphics for example). You can customize it too (check on the OpenNMS page a short tutorial i wrote about how to generate custom graphics on OpenNMS with rrdtool). > One of the first items in my TODO list is to dissect OpenNMS's code so = to > know exactly what can be used where, but as always, time, time, time... >=20 Check the archives of the list or even better, why you don=B4t ask some questions on the developers list? Hope to hear some news soon. JV. > >>=20 > >> Again thank you very much. > >=20 > > Let me known if you're looking for developers. >=20 > As soon as our roadmap is complete. >=20 > Thanks for the input. >=20 > DK --=20 Jose Vicente Nunez Zuleta <jo...@us...> |
|
From: DK <dk...@ip...> - 2003-09-28 14:59:45
|
Hi, El 9/27/03 16:46, "Jose Vicente Nunez Zuleta" <jo...@ne...> escribi=F3: > Hi to all, my answers are below > >> Os-sim is growing very fast and at this point I agree with you, we have >> to >> be careful about what direction we want take. This is why we stopped >> development for a few days so we can decide what, when and how we want >> to >> integrate tools, what DB / languages to use and how to glue everything >> together. >=20 > I think an update on the architecture diagram is a must. Also it is impor= tant > to have a list of deliverables, > so people interested in the project knows what to expect when (maybe this= is > probably too much). >=20 Both issues are going to be solved when we release the roadmap (I don't kno= w when, hopefully next week) so we can provide the following: - In depth system architecture. - What to expect from updates until 1.0 is released. What we want to do until 2.0. When to expect all of this. - Tasks for ourselves as well as for anyone wishing to help with the project. >>=20 >> Again I have to ask you to excuse us for the lack of documentation. I >> hope >> this will change soon, we are working on it so we are able to transmit >> our >> thoughts and ideas about how everything should work. >>=20 >=20 > I was thinking than maybe a less loosely coupled architecture is the best > here; In that way the product could be > integrated not only with OpenNMS but other NMS out there as well. >=20 > I'm sending a deployment diagram i draw, maybe you will find it interesti= ng :) >=20 Interesting indeed. Why do you think it would be useful to integrate jabber ? Besides, our intention is to move as many things we can away from syslog because syslog (or syslog-ng, as you want) slows things down. We have to speed everything up, speed it a lot up so that's our main focus as of today= . I have to draw a new architecture map, hopefully this week I can get two hours to redraw it. >>=20 >>=20 >> Then next four questions (except PHP vs. Java) should be answered soon. >> Please have a look at my diary >> (http://sourceforge.net/developer/diary.php?diary_id=3D14122&diary_user=3D82= 5580 >> ) where I posted some thoughts today. I didn't want to show them on the >> main >> site because they are personal thoughts and I would like to discuss >> them >> whit the rest of the team so we can come up with some good >> documentation / >> todo list / roadmap. >>=20 >> Resuming what I wrote related to your questions: >>=20 >> - We want to get rid of MRTG. We don't need it. >=20 > ok >=20 >>=20 >> - We have to decide between mysql & postgresql. Both have its pro's and >> con's. >>=20 >=20 > Ok. Again, i think PostgreSQL is better suited for this task (this can be > discussed in detail). We have to decide this issue soon. I think it is going to be easier to rewrite our C code to make use of postgresql rather than move OpenNMS to MySQL. No decision taken as of today. >=20 >> - Of course nessus-opennms integration would be done after talking with >> Opennms's creators. >=20 > Great. >=20 >> - We have to write a data consolidator which accepts input from many >> more >> devices. >=20 > Syslog could be a first option, and then more could be added as 'plugins'= ? >=20 As stated before, we want to move away from syslog for speed reasons, but o= f course not everything because some products are better integrated within syslog. And we want to correlate system events too (both unix & windows). >> - C is needed. Perl isn't but can solve temporary problems. At the end, >> the >> whole core should be written in C. >=20 > You could still use a language like Python to do the scripting part; Is e= asier > to extend in C than Perl (you don't need > Swig for that). Supports objects better than Perl, etc. Also if you wanna= use > Jython instead of Python that makes it easier > to glue it together with Java. >=20 > I would use C only for tasks that require the speed, i think Java is bett= er > suited as the main languaje of the application. Personally I don=B9t like Java although this opinion is changing lately. The scripting part is only a temporary solution because every single component needs to run as fast as possible so C is going to be used for the main core= . Around that we can build over every language that suits our needs. Java is going to be an alternative, that for sure, but we will also use PHP, Perl and why not, python is also an alternative (I love python). =20 >>=20 >> We want to make use a lot of opennms's many good features but we think >> some >> of them are better kept separated because there are already very good >> programs out there that do their specific job. >>=20 >> I don't think it is such a good idea to rewrite ntop entirely only to >> get a >> tighter integration with opennms. >>=20 > I agree. Actually i would leave out NTOP and would rely on OpenNMS for th= e > network monitoring and analysis. Agreed. >>=20 >> As of network discovery, at this moment we use nmap only for a couple >> of >> things. It's planned to add service discovery but I think its better to >> compare and complement NMS data with nmaps input rather than to rely >> entirely on its own network discovery, mainly because many times we >> don't >> want the full set of opennms's discovery and poller related features. >>=20 >=20 > But again, what sense it makes to have two applications discovering nodes= at > the same time? OpenNMS discovery capabilitites > can be extended easily using Java plugins and the Assest database is fair= ly > complete; I think the goal of this project > should be focus on how to analize all the data gathered by OpenNMS, Snort= , > Syslog instead of replicate > the polling and discovery functionality. Perhaps my lack of knowledge of the inner workings of OpenNMS misleads me, but I think for some small specific functions we should better rely on specific programs, as with Ntop. How does opennms's service detection work ? Does it rely on port number or does it make some checks to ensure port 53 is DNS indeed (for example) ? Is this work based on existing applications like amap or is it a complete new write up in Java ? What about host and port discovery ? Does it use nmap or scanrand or something similar or is it written up from 0 ? If you have tried the program out you'll have noticed that OpenNMS integration at this point is minimal. Only one link from the main page but not as tight integrated as ntop or rrd for example. One of the first items in my TODO list is to dissect OpenNMS's code so to know exactly what can be used where, but as always, time, time, time... >>=20 >> Again thank you very much. >=20 > Let me known if you're looking for developers. As soon as our roadmap is complete. Thanks for the input. DK |
|
From: Jose V. N. Z. <jo...@ne...> - 2003-09-27 15:33:29
|
Hi to all, my answers are below > Hi JV, >=20 > First of all thanks for the input. >=20 > I'll answer to your questions / appointments below. >=20 > El 9/26/03 16:27, "Jose Vicente Nunez Zuleta" > <jo...@us...> escribi=F3: >=20 > > Greetings, > >=20 > > I saw the project and i think is definitely worth the effort; If you > > have a medium to large network under you control you will notice=20 > pretty > > soon than without the appropiate tools you will not be able to=20 > correlate > > and investigate event the most basic events. >=20 > Os-sim is growing very fast and at this point I agree with you, we have= =20 > to > be careful about what direction we want take. This is why we stopped > development for a few days so we can decide what, when and how we want=20 > to > integrate tools, what DB / languages to use and how to glue everything > together. I think an update on the architecture diagram is a must. Also it is impor= tant to have a list of deliverables, so people interested in the project knows what to expect when (maybe this= is probably too much).=20 >=20 >=20 > >=20 > > The idea of using readily available tools like Snort, OpenNMS, NMAP=20 > etc > > is very attractive because a large number of sites are already using > > them so a 'glue' that ties all that information together is more than > > welcome. > >=20 > > Still, there are some points related with the arquitecture of the > > project that doesn't seem to fit appropriately. > >=20 > > All my comments are based on the information contained on the > > arquitecture diagram published by the os-sim developers > > (http://os-sim.sourceforge.net/docs/ossim3.jpg) and the tools=20 > mentioned > > on their web site (http://os-sim.sourceforge.net/home.html) >=20 > Again I have to ask you to excuse us for the lack of documentation. I=20 > hope > this will change soon, we are working on it so we are able to transmit=20 > our > thoughts and ideas about how everything should work. >=20 I was thinking than maybe a less loosely coupled architecture is the best= here; In that way the product could be=20 integrated not only with OpenNMS but other NMS out there as well. I'm sending a deployment diagram i draw, maybe you will find it interesti= ng :) >=20 >=20 > Then next four questions (except PHP vs. Java) should be answered soon. > Please have a look at my diary > (http://sourceforge.net/developer/diary.php?diary_id=3D14122&diary_user= =3D825580 > ) where I posted some thoughts today. I didn't want to show them on the= =20 > main > site because they are personal thoughts and I would like to discuss=20 > them > whit the rest of the team so we can come up with some good=20 > documentation / > todo list / roadmap. >=20 > Resuming what I wrote related to your questions: >=20 > - We want to get rid of MRTG. We don't need it. ok >=20 > - We have to decide between mysql & postgresql. Both have its pro's and > con's. >=20 Ok. Again, i think PostgreSQL is better suited for this task (this can be= discussed in detail). > - Of course nessus-opennms integration would be done after talking with > Opennms's creators. Great. > - We have to write a data consolidator which accepts input from many=20 > more > devices. Syslog could be a first option, and then more could be added as 'plugins'= ? > - C is needed. Perl isn't but can solve temporary problems. At the end,= =20 > the > whole core should be written in C. You could still use a language like Python to do the scripting part; Is e= asier to extend in C than Perl (you don't need Swig for that). Supports objects better than Perl, etc. Also if you wanna= use Jython instead of Python that makes it easier to glue it together with Java. I would use C only for tasks that require the speed, i think Java is bett= er suited as the main languaje of the application. >=20 > We want to make use a lot of opennms's many good features but we think=20 > some > of them are better kept separated because there are already very good > programs out there that do their specific job. >=20 > I don't think it is such a good idea to rewrite ntop entirely only to=20 > get a > tighter integration with opennms. >=20 I agree. Actually i would leave out NTOP and would rely on OpenNMS for th= e network monitoring and analysis. >=20 > As of network discovery, at this moment we use nmap only for a couple=20 > of > things. It's planned to add service discovery but I think its better to > compare and complement NMS data with nmaps input rather than to rely > entirely on its own network discovery, mainly because many times we=20 > don't > want the full set of opennms's discovery and poller related features. >=20 But again, what sense it makes to have two applications discovering nodes= at the same time? OpenNMS discovery capabilitites can be extended easily using Java plugins and the Assest database is fair= ly complete; I think the goal of this project should be focus on how to analize all the data gathered by OpenNMS, Snort= , Syslog instead of replicate the polling and discovery functionality. >=20 > Again thank you very much. Let me known if you're looking for developers. JV. >=20 > Greetings, >=20 > DK >=20 > >=20 > > 1) Too many OpenSource tools, some overlapping: For example NTOP=20 > could > > be replaced with custom OpenNMS graphs and custom polling of SNMP=20 > OIDS > > on the target machines; MGRT could be replaced directly with RRDD=20 > tool > > and again OpenNMS can show the custom graphs too; You mention NMAP=20 > for > > network discovery, but OpenNMS already does that job. > >=20 > > Two databases? PostgreSQL can do the job of MySQL (and OpenNMS=20 > already > > uses it). Snort can log events directly to PostgreSQL. MySQL probably= =20 > is > > too simple for an enterprise solution like this one. > >=20 > > Sortova consulting was doing some works to integrate Nessus with=20 > OpenNMS > > (currently nobody is working on that tough). Maybe is better to > > contribute code to integrate them than reinventing the wheel here. > >=20 > > 2) Too many languajes and platforms to maintain: The arquitecture=20 > itself > > is composed of at least two databases (PostgreSQL and MySQL), at=20 > least > > two web presentation layers (PHP, JSP), several programming languajes > > (Perl, PHP, Java, C). This is a mainteinance disaster waiting to=20 > happen > > unless a hughe team of developers with a large skillset is ready to > > maintain all the pieces that tie together the application. > >=20 > > The team should reconsider to rewrite at least some of the analisys > > components. Also only one database shold be used. > >=20 > > 3) What about other devices like switches, firewalls that doesn't=20 > even > > speak SNMP but Syslog?. What about machine that run only Syslog=20 > deamons? > > A custom Syslog listener could be a very good glue to gather=20 > information > > from several places like Snort sensors, IPTables firewalls, Switches. > > One problem with SNMP traps is than they are not very secure and once > > you know the password you can tamper the contents or even flood the=20 > SNMP > > trap daemon. > >=20 > > 4) PHP and JSP, which one is better? My Opinion is that PHP is not > > prepared for the enterprise. PHP doesn't have the optimizations than > > Java do on the server side and also ties the presentation too close=20 > with > > the logic. It's true than Java takes more time to master, but is also > > true than is much more flexlible and standarized than PHP (PHP=20 > doesn't > > have a common database layer AFIK). The team should try to replace=20 > tools > > like ACID (wich source code is freely available) and maybe thing=20 > about > > integrating those changes with OpenNMS (as a custom web app) or > > developing the tool using Java as a separate web app. > >=20 > > In my opinion, a tigher integration with OpenNMS will solve many of > > these issues; > >=20 > > What do you think about this? > >=20 > > JV. --=20 Jose Vicente Nunez Zuleta <jo...@ne...> Newbreak LLC |
|
From: DK <dk...@ip...> - 2003-09-26 15:17:50
|
Hi JV, First of all thanks for the input. I'll answer to your questions / appointments below. El 9/26/03 16:27, "Jose Vicente Nunez Zuleta" <jo...@us...> escribi=F3: > Greetings, >=20 > I saw the project and i think is definitely worth the effort; If you > have a medium to large network under you control you will notice pretty > soon than without the appropiate tools you will not be able to correlate > and investigate event the most basic events. Os-sim is growing very fast and at this point I agree with you, we have to be careful about what direction we want take. This is why we stopped development for a few days so we can decide what, when and how we want to integrate tools, what DB / languages to use and how to glue everything together. >=20 > The idea of using readily available tools like Snort, OpenNMS, NMAP etc > is very attractive because a large number of sites are already using > them so a 'glue' that ties all that information together is more than > welcome. >=20 > Still, there are some points related with the arquitecture of the > project that doesn't seem to fit appropriately. >=20 > All my comments are based on the information contained on the > arquitecture diagram published by the os-sim developers > (http://os-sim.sourceforge.net/docs/ossim3.jpg) and the tools mentioned > on their web site (http://os-sim.sourceforge.net/home.html) Again I have to ask you to excuse us for the lack of documentation. I hope this will change soon, we are working on it so we are able to transmit our thoughts and ideas about how everything should work. Then next four questions (except PHP vs. Java) should be answered soon. Please have a look at my diary (http://sourceforge.net/developer/diary.php?diary_id=3D14122&diary_user=3D82558= 0 ) where I posted some thoughts today. I didn't want to show them on the mai= n site because they are personal thoughts and I would like to discuss them whit the rest of the team so we can come up with some good documentation / todo list / roadmap. Resuming what I wrote related to your questions: - We want to get rid of MRTG. We don't need it. - We have to decide between mysql & postgresql. Both have its pro's and con's. - Of course nessus-opennms integration would be done after talking with Opennms's creators. - We have to write a data consolidator which accepts input from many more devices. - C is needed. Perl isn't but can solve temporary problems. At the end, the whole core should be written in C. We want to make use a lot of opennms's many good features but we think some of them are better kept separated because there are already very good programs out there that do their specific job. I don't think it is such a good idea to rewrite ntop entirely only to get a tighter integration with opennms. As of network discovery, at this moment we use nmap only for a couple of things. It's planned to add service discovery but I think its better to compare and complement NMS data with nmaps input rather than to rely entirely on its own network discovery, mainly because many times we don't want the full set of opennms's discovery and poller related features. Again thank you very much. Greetings, DK >=20 > 1) Too many OpenSource tools, some overlapping: For example NTOP could > be replaced with custom OpenNMS graphs and custom polling of SNMP OIDS > on the target machines; MGRT could be replaced directly with RRDD tool > and again OpenNMS can show the custom graphs too; You mention NMAP for > network discovery, but OpenNMS already does that job. >=20 > Two databases? PostgreSQL can do the job of MySQL (and OpenNMS already > uses it). Snort can log events directly to PostgreSQL. MySQL probably is > too simple for an enterprise solution like this one. >=20 > Sortova consulting was doing some works to integrate Nessus with OpenNMS > (currently nobody is working on that tough). Maybe is better to > contribute code to integrate them than reinventing the wheel here. >=20 > 2) Too many languajes and platforms to maintain: The arquitecture itself > is composed of at least two databases (PostgreSQL and MySQL), at least > two web presentation layers (PHP, JSP), several programming languajes > (Perl, PHP, Java, C). This is a mainteinance disaster waiting to happen > unless a hughe team of developers with a large skillset is ready to > maintain all the pieces that tie together the application. >=20 > The team should reconsider to rewrite at least some of the analisys > components. Also only one database shold be used. >=20 > 3) What about other devices like switches, firewalls that doesn't even > speak SNMP but Syslog?. What about machine that run only Syslog deamons? > A custom Syslog listener could be a very good glue to gather information > from several places like Snort sensors, IPTables firewalls, Switches. > One problem with SNMP traps is than they are not very secure and once > you know the password you can tamper the contents or even flood the SNMP > trap daemon. >=20 > 4) PHP and JSP, which one is better? My Opinion is that PHP is not > prepared for the enterprise. PHP doesn't have the optimizations than > Java do on the server side and also ties the presentation too close with > the logic. It's true than Java takes more time to master, but is also > true than is much more flexlible and standarized than PHP (PHP doesn't > have a common database layer AFIK). The team should try to replace tools > like ACID (wich source code is freely available) and maybe thing about > integrating those changes with OpenNMS (as a custom web app) or > developing the tool using Java as a separate web app. >=20 > In my opinion, a tigher integration with OpenNMS will solve many of > these issues; >=20 > What do you think about this? >=20 > JV. |
1 message has been excluded from this view by a project administrator.
){kind=link}