Re: [Os-sim-devel] OSIM arquitecture: too many components?

[Jose V. N. Z. <jo...@ne...>]

Hi to all, my answers are below


> Hi JV,
>=20
> First of all thanks for the input.
>=20
> I'll answer to your questions / appointments below.
>=20
> El 9/26/03 16:27, "Jose Vicente Nunez Zuleta"
> <jo...@us...> escribi=F3:
>=20
> > Greetings,
> >=20
> > I saw the project and i think is definitely worth the effort; If you
> > have a medium to large network under you control you will notice=20
> pretty
> > soon than without the appropiate tools you will not be able to=20
> correlate
> > and investigate event the most basic events.
>=20
> Os-sim is growing very fast and at this point I agree with you, we have=
=20
> to
> be careful about what direction we want take. This is why we stopped
> development for a few days so we can decide what, when and how we want=20
> to
> integrate tools, what DB / languages to use and how to glue everything
> together.

I think an update on the architecture diagram is a must. Also it is impor=
tant to have a list of deliverables,
so people interested in the project knows what to expect when (maybe this=
 is probably too much).=20

>=20
>=20
> >=20
> > The idea of using readily available tools like Snort, OpenNMS, NMAP=20
> etc
> > is very attractive because a large number of sites are already using
> > them so a 'glue' that ties all that information together is more than
> > welcome.
> >=20
> > Still, there are some points related with the arquitecture of the
> > project that doesn't seem to fit appropriately.
> >=20
> > All my comments are based on the information contained on the
> > arquitecture diagram published by the os-sim developers
> > (http://os-sim.sourceforge.net/docs/ossim3.jpg) and the tools=20
> mentioned
> > on their web site (http://os-sim.sourceforge.net/home.html)
>=20
> Again I have to ask you to excuse us for the lack of documentation. I=20
> hope
> this will change soon, we are working on it so we are able to transmit=20
> our
> thoughts and ideas about how everything should work.
>=20

I was thinking than maybe a less loosely coupled architecture is the best=
 here; In that way the product could be=20
integrated not only with OpenNMS but other NMS out there as well.

I'm sending a deployment diagram i draw, maybe you will find it interesti=
ng :)

>=20
>=20
> Then next four questions (except PHP vs. Java) should be answered soon.
> Please have a look at my diary
> (http://sourceforge.net/developer/diary.php?diary_id=3D14122&diary_user=
=3D825580
> ) where I posted some thoughts today. I didn't want to show them on the=
=20
> main
> site because they are personal thoughts and I would like to discuss=20
> them
> whit the rest of the team so we can come up with some good=20
> documentation /
> todo list / roadmap.
>=20
> Resuming what I wrote related to your questions:
>=20
> - We want to get rid of MRTG. We don't need it.

ok

>=20
> - We have to decide between mysql & postgresql. Both have its pro's and
> con's.
>=20

Ok. Again, i think PostgreSQL is better suited for this task (this can be=
 discussed in detail).

> - Of course nessus-opennms integration would be done after talking with
> Opennms's creators.

Great.

> - We have to write a data consolidator which accepts input from many=20
> more
> devices.

Syslog could be a first option, and then more could be added as 'plugins'=
?

> - C is needed. Perl isn't but can solve temporary problems. At the end,=
=20
> the
> whole core should be written in C.

You could still use a language like Python to do the scripting part; Is e=
asier to extend in C than Perl (you don't need
Swig for that). Supports objects better than Perl, etc. Also if you wanna=
 use Jython instead of Python that makes it easier
to glue it together with Java.

I would use C only for tasks that require the speed, i think Java is bett=
er suited as the main languaje of the application.

>=20
> We want to make use a lot of opennms's many good features but we think=20
> some
> of them are better kept separated because there are already very good
> programs out there that do their specific job.
>=20
> I don't think it is such a good idea to rewrite ntop entirely only to=20
> get a
> tighter integration with opennms.
>=20
I agree. Actually i would leave out NTOP and would rely on OpenNMS for th=
e network monitoring and analysis.
>=20
> As of network discovery, at this moment we use nmap only for a couple=20
> of
> things. It's planned to add service discovery but I think its better to
> compare and complement NMS data with nmaps input rather than to rely
> entirely on its own network discovery, mainly because many times we=20
> don't
> want the full set of opennms's discovery and poller related features.
>=20

But again, what sense it makes to have two applications discovering nodes=
 at the same time? OpenNMS discovery capabilitites
can be extended easily using Java plugins and the Assest database is fair=
ly complete; I think the goal of this project
should be focus on how to analize all the data gathered by OpenNMS, Snort=
, Syslog instead of replicate
the polling and discovery functionality.
>=20
> Again thank you very much.

Let me known if you're looking for developers.

JV.

>=20
> Greetings,
>=20
> DK
>=20
> >=20
> > 1) Too many OpenSource tools, some overlapping: For example NTOP=20
> could
> > be replaced with custom OpenNMS graphs and custom polling of SNMP=20
> OIDS
> > on the target machines; MGRT could be replaced directly with RRDD=20
> tool
> > and again OpenNMS can show the custom graphs too; You mention NMAP=20
> for
> > network discovery, but OpenNMS already does that job.
> >=20
> > Two databases? PostgreSQL can do the job of MySQL (and OpenNMS=20
> already
> > uses it). Snort can log events directly to PostgreSQL. MySQL probably=
=20
> is
> > too simple for an enterprise solution like this one.
> >=20
> > Sortova consulting was doing some works to integrate Nessus with=20
> OpenNMS
> > (currently nobody is working on that tough). Maybe is better to
> > contribute code to integrate them than reinventing the wheel here.
> >=20
> > 2) Too many languajes and platforms to maintain: The arquitecture=20
> itself
> > is composed of at least two databases (PostgreSQL and MySQL), at=20
> least
> > two web presentation layers (PHP, JSP), several programming languajes
> > (Perl, PHP, Java, C). This is a mainteinance disaster waiting to=20
> happen
> > unless a hughe team of developers with a large skillset is ready to
> > maintain all the pieces that tie together the application.
> >=20
> > The team should reconsider to rewrite at least some of the analisys
> > components. Also only one database shold be used.
> >=20
> > 3) What about other devices like switches, firewalls that doesn't=20
> even
> > speak SNMP but Syslog?. What about machine that run only Syslog=20
> deamons?
> > A custom Syslog listener could be a very good glue to gather=20
> information
> > from several places like Snort sensors, IPTables firewalls, Switches.
> > One problem with SNMP traps is than they are not very secure and once
> > you know the password you can tamper the contents or even flood the=20
> SNMP
> > trap daemon.
> >=20
> > 4) PHP and JSP, which one is better? My Opinion is that PHP is not
> > prepared for the enterprise. PHP doesn't have the optimizations than
> > Java do on the server side and also ties the presentation too close=20
> with
> > the logic. It's true than Java takes more time to master, but is also
> > true than is much more flexlible and standarized than PHP (PHP=20
> doesn't
> > have a common database layer AFIK). The team should try to replace=20
> tools
> > like ACID (wich source code is freely available) and maybe thing=20
> about
> > integrating those changes with OpenNMS (as a custom web app) or
> > developing the tool using Java as a separate web app.
> >=20
> > In my opinion, a tigher integration with OpenNMS will solve many of
> > these issues;
> >=20
> > What do you think about this?
> >=20
> > JV.
--=20
Jose Vicente Nunez Zuleta <jo...@ne...>
Newbreak LLC