Hope this helps:
Here are my ideas, tho a little incoherent and incomplete.

Decision Time

Programming Style
IPC/Thread
Authentication
File Singatures/Attack templates
Locking Method
Heuristics

One thing to assume:  The users will run only one agent on a system (or multiple agents with different config files, kinda like apache..)

Now,
Problem 1: Programming Style

not sure if this is what you want, but here it goes:

prototypes:
return_type func_name(type varname, type varname); /* purpose, usage, etc. */

These will be in a header file with the same name as the c file (ie, net.c and net.h) and included in the c file.  comment after the prototype on what it does, how to use it etc.  if more than one line, put a block of comments above the prototype.. like:
/*
  comments...
  more comments...
*/
proto......

function bodies:
return_type func_name(type varname, type varname)
{
  /* tab width = 2 spaces */
  /* always tab after a '{' and untab before '}' */

  /* all a functions variable should go at the top with comments on what it is , like this */
  int a; /* sentinel value for main loop */
  /* don't do initialization here, instead, declare all variables first, then skip a line and do all initialization. */
  int a; /* blah */
  int b; /* blah */

  a = 0;
  b = 0;
/* skip another line, then put function's body */

/* function code here */
  for(a=0; a<++b; a++)
  {
    /* only use '{' and '}' if loop or if statement's body is more than one line..  otherwise drop them */
  }
 
  /* skip aline */
  return 1;
} 

obviously, we want all closing brackets on the same tab as the openers.  this makes the code easier to read.  on comments:  comment on what each line is/should be doing.  this will make debuging much easier.
#ifdef's etc. should not be tabbed, instead, put them at the very left of the page...

That's about all i have on that, i hope you all add lots to this!

Problem 2: IPC / Threads
For now, we will be using a multi-threaded style.  each thread will be made, unless somebody knows a better more efficient way, using a fork() call in a seperate function.  Please tell me you know a better way.

I dont exactly know how to implement IPC in this style, but i was thinking that we should develop a sort of API.  for example, each module should have a start up function which will do all the module's initializations etc., then call the modules' main function (which will fork()) to start the thread.  then, that thread will communicate with other threads by using either global variables (pluch!), calls to other modules' functions, or something better that you could think up.  Let me know if you want an example.

Problem 3: Authentication
Possible solutions:
Hard Coded Keyed Hash,
Certs,
Self-Signed Certs,
Public Key Crypto,
Symmetric Key Crypto,
    "       "     "  using a keyed hash..
or some other combination.

- I'd suggest not using symmetric key crpyto because the key could conceivably be reversed from the compiled code.
- A keyed hash could be possible and would keep the key from going over the wire (that is good!).
- Certs is probably not a possibility since they are expensive (to my knowledge!).  We want something that will be hardcore and cheap.
- Public Key crypto would be good, but a little wierd in this context... (think more on this, plz)
- Symmetric Crypto with a Keyed Hash could be used, but a reversed hash could be 'injected' into a skeletal node to 'spoof' its authenticity and screw things up.  not good. 
Any other ideas.
For now, i say we go with a keyed hash with a hash built at compile time into a hash.h file.  use a random bit shift to give us a little more security than just plainly storing the hash.

Problem 4: File signatures/ attack templates.
Not my specialty, but i'd suggest doing something low-level to check the status of files in important locations (/etc, /root, /bin, etc.).  Somebody proposed using the inodes i beleive, and if there is a way to do this, it would limit this to the ext2 FS, but would be good for it. 
Also, the Host-Based IDS should be able track users (compile time option) and report any suspiciuous activity (ie., root commands coming over eth0...  hmmm).
Another thing to consider, pretty unrelated to the current issue, is having a /etc/agent.conf file that could be used to change run time variables via a HUP signal or something to that effect.  Please, Please... add to this section somebody who knows.

Problem 5: Network Heuristics
First of all, we want to try and respect the privacy of the users of this program.  The net module will have heuristics scanning, and this means (as far as i can tell) sniffing all traffic.  If they dont want this, they should have the option (at compile time) to turn this feature off.

Questions:
Should we check the entire packet we have or just it's headers?

Look at e-watch (i believe a link is in the dev forum).  That would be a good palce to start with this sort of thing..

End

That is all i have right now... Please, post your comments. I will be sending out emails to get the attention of ALL the development staff, so please read and post.

red0x

davis:

re the net module:

Encryption:
That all depends, do we want the inter agent communications to be faster or more secure.  I'm thinking that people wont be trying to use agents across the internet, so bandwidth shouldn't be a concern.  However, we want as much security for speed as we can get (does that make sense?).  Maybe what we should do is implement both options and benchmark them.  Then we will decide for final.  sound good?

Protocols to support/watch:
I think these should be good:
IP/ICMP/UDP/TCP/DHCP/ARP/RARP/IPSec  (any others you suggest?)

for now, the firewalls to block off hosts will be linux 4.2.0's implementation of IPTABLES  (whatever version is currently out.)  Then, we can later add a 'module' or something to talk to off host firewalls.  How's that sound (that should make things a little simpler).

Merry Christmas and Happy New Year.  Welcome Back.

EOM - red0x

Ok here are a few answers from my point of
view and experience... sorry to all that I'm
not too involved in the project ATM. I'm on
vacation for another 10-14 days and have crappy
internet access. After being back to work I will
do more on the project.

I. multithreaded is a form of IPC :) - I think
we should use pthreads (if we have to), threads ar e a pain in the back because they make debugging
very hard. We should implement threads in a way we can deactivate them (e.g. with #defines, so we can
build a non-threaded very slow debug version...). At least that's my experience on the topic. If you
have concrete problems with pthreads, please mail me.

II. I haven't seen enough source/txt to understand
how you actually want to implement modules? shared libraries?! Anyone care to explain?

III. mutexes, if we use the easiest way, pthreads

NIDS

1) i know the problem I'm having exactly the same
with another application. the problem is that OpenSSL extremely sucks, especially it is cannot
work with IPC/threaded applications, and has many
hard to debug errors. I am DESPERATELY searching for an implementation of an asymmetrical cipher
(rsa/dh) that I can put in libmix!!
The problem is that they always come in their own
libraries which are either incomplete, or incompatible or so big and ugly that I can't extract the algorithm without missing underlying
functions that it needs :( If anyone has any idea
on this point, it is greatly appreciated, as it will help me in a number of other projects!
(Perhaps I should start a separate sourceforge project for libmix with asymmetrical crypto?)

III. I just know IPSec and IPv6 is horribly complex, especially you don't want to know how
SDNS works.. if we ever implement complex protocols like these we should add them later after a stable version... on arp, rarp etc.
you can study LibNet and Nemesis to see how they
work in a protocol stack. But anyways, I vote
against bothering with them at least for now.

IV. RTP/VPIM/SIP are obsolete or very unpopular,
and as for DHCP its on top of tcp. We could just
log a connection attempt to that (optionally of course) with a tcp rule. Integrating all application protocols seems to much for me, as
IDS's work mainly on a pattern matching basis.
Emulation of application protocols to "understand"
what exactly is happening (e.g. tracing a ftp
session) seems like an interesting but very complex task and is nothing we should think about
in the beginning/conceptual design phase...

5) I can help on that but we will need very detailed technical specs..

6) I dunno, are there any open-source protocols
for firewall communication, actually?? Also,
every protocol is a form of access points and a potential vulnerability which means we have to
audit that part of the code very good.

  Hey people,

     So from what I read, here's what I gathered... I've already went ahead
  and started putting in some threading code.

  Problem 1 : Alrighty...
  Problem 2 :

     1)  pthreads is a go.  I will check in skeleton code for this sometime tonite.
          Mind that it doesn't work, something funky going on which I'm trying to
          debug.  But it compiles and runs :-P  Mixter, I am not sure how your
          threading and forking idea works.  I have worked on a project porting
          IPC to threads, and its a lot more than just #defines...  I'm currently
          using a simple locking queue for inter-thread comm.

     2)  I'm using the pthread_lock and pthread_mutex for my locking mechanism.

     3)  IPC is inter-process communications, threading technically is not an IPC
         mechanism since everything stays in one process.  Yeah yeah technicalities.
         Well at least not pthreads, kthreads on the other hand...             

  Problem 3 :

    Okay, so we have nice discussions here.  Problem with keyed hash is still the
  problem of key. Note on how secure netcom protocols are
  designed (IPSec, SSL),  they both use asym crypto to exchange nonces
  as sym keys.  After further thought, I think compiling in a hash key for use
  during asym xchange and using sym. Extension of current proposal... For crying
  out loud, we have a host-IDS to prevent the reverse-engineering right ?

    Mixter, I believe you can rip out dh from FreeSwan pretty easily.  They have it
  implemented as a individual module if memory serves me correctly.  Ultimately
  we might not have a choice but to use openssl.  It won't need to support
  reentrancy in anycase since only one thread will be doing network comms.
  Crossing my fingers...

  Problem 4 :

     Attack signatures...hmmm... we need to find someone for this.  Or we can go
  with my original proposal and implement templates for things we know are
  allowable for the network :-).  I really want to see if that would work out.

  Problem 5 : Network Heuristics ? Huh...
   
  Protocol Support :

    IP/ICMP/UDP/TCP/DHCP/ARP/RARP/IPSec 
    I think if we want to really distinguish this product/project, we need to support
  more than this.  How about SIP/RTP and the very least ?  They are easier to
  implement than IPSec (hehe...I am definitely NOT volunteering for that one).
  The boogey man IKE is coming your way :-) 
  
  - davis
 
 
  

Authentication
File Singatures/Attack templates
Heuristics

Authentication:
Serious IDS use Pubkeys.
From what I remember, each IDS device creates a key pair. The pubkeys have to manually be placed
on each host. This is not good for management.
I'm not sure but could something be done with LDAP, Kerboros or CEP?

File Singatures/Attack templates:
Can Tripwire partly help us with file checks?

Heuristics:
Can Snort or Portsentry partly help us with network attack checks?
What about signature updates?

Something to think/talk about:
What about making the agent part very small and not too intrusive on the host.
The agent will be very bare bones generic.
Each agent communicates with a central management device where by tasks are assigned to them.
When tasks are completed, results are fed back to management device, who then checks logs ...
Agent's should have 2 side, 1, perform routine checks like break-in ... , 2, check for realtime attacks.

Hello All,

  Just stopping by to say hi. Hi. Hehe...

-davis_alumni