sguil-devel Mailing List for Sguil (Page 24)
Status: Beta
Brought to you by:
bamm
You can subscribe to this list here.
2003 |
Jan
|
Feb
(36) |
Mar
(16) |
Apr
(4) |
May
(15) |
Jun
|
Jul
(1) |
Aug
(13) |
Sep
(3) |
Oct
(62) |
Nov
(22) |
Dec
(13) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2004 |
Jan
(10) |
Feb
|
Mar
(15) |
Apr
(7) |
May
(9) |
Jun
(4) |
Jul
(3) |
Aug
(3) |
Sep
(4) |
Oct
(1) |
Nov
(4) |
Dec
(2) |
2005 |
Jan
(5) |
Feb
(16) |
Mar
(7) |
Apr
|
May
(16) |
Jun
(13) |
Jul
(3) |
Aug
(1) |
Sep
(1) |
Oct
|
Nov
(9) |
Dec
(11) |
2006 |
Jan
(7) |
Feb
(1) |
Mar
(55) |
Apr
(1) |
May
|
Jun
(2) |
Jul
|
Aug
(7) |
Sep
(5) |
Oct
(4) |
Nov
(2) |
Dec
(2) |
2007 |
Jan
(2) |
Feb
(10) |
Mar
(4) |
Apr
(15) |
May
(2) |
Jun
|
Jul
(1) |
Aug
|
Sep
|
Oct
(8) |
Nov
|
Dec
(1) |
2008 |
Jan
|
Feb
(4) |
Mar
(10) |
Apr
(35) |
May
(15) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(4) |
Nov
(4) |
Dec
(2) |
2009 |
Jan
(1) |
Feb
(1) |
Mar
|
Apr
(3) |
May
|
Jun
(7) |
Jul
(2) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2010 |
Jan
(1) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
2011 |
Jan
(5) |
Feb
(16) |
Mar
|
Apr
|
May
(5) |
Jun
(13) |
Jul
|
Aug
(1) |
Sep
|
Oct
|
Nov
|
Dec
|
2012 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(3) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(16) |
2013 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(2) |
Oct
|
Nov
|
Dec
|
2014 |
Jan
(2) |
Feb
|
Mar
|
Apr
(3) |
May
|
Jun
|
Jul
(1) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Michael B. <mic...@se...> - 2003-02-20 17:04:22
|
I am planning to use sensor id 0 for default configuration, something that should be enabled on ALL sensors. Any objections? /Mike --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Michael B. <mic...@se...> - 2003-02-20 16:21:29
|
On Thu, Feb 20, 2003 at 10:04:53AM -0600, Steve Halligan wrote: > >> We need a feature freeze and release a new version once the=20 > >WorkInProgress > >> stuff are finished. What I can see as outstanding issues are: > >>=20 > >> - Sguil.tk rule managment > >> - sguilagent in C > >> - feature freeze in DB scheme > >> - header/tail, default Cc and Subject of email >=20 >=20 > I can do the email stuff today. I will use ex...@ex... for the > default CC and comment the line out in sguil.conf. > Any suggestions about a default Subject? > Any Suggestions about a default header/tail? >=20 > My initial thoughts: > Subject: Incident Report (possibly pop a date/time in here) > Header: "The following intrusion attempt was detected from your > network. All times are UTC and are accurate." > Tail: "For additional information please reply to this email address." >=20 > Maybe we don't need to put anything in these by default, but I think it > would be good to put something in as a suggestion to the end users. For > example, many incident Reports get flat-out ignored by abuse contacts > due to simply omitting that bit about Timezone and Time accuracy. >=20 >=20 > Thought? >=20 > -steve Sounds good for a start. Ours looks like this (took a random event from the= DB): ---8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<-- Subject: Potential Attack Coming From Your Network Dear Hostmaster We recently experienced a possible attack from an IP address that originates on your network. We would be most grateful if you would investigate this issue and take the nessasary action. Included are the full details of the attack. Please see below. (All timestamps are in UTC and are accurate) ------------------------------------------------------------------------ Count:9 Event#2.127297 2003-02-20 15:59:07 ATTACK RESPONSES 403 Forbidden 192.168.1.2 -> 203.124.0.246 IPVer=3D4 hlen=3D5 tos=3D0 dlen=3D386 ID=3D56703 flags=3D2 offset=3D0 ttl= =3D128 chksum=3D0 Protocol: 6 sport=3D80 -> dport=3D1124 Seq=3D3775002546 Ack=3D3664071746 Off=3D8 Res=3D0 Flags=3D***AP*** Win=3D63= 911 urp=3D48096 chksum=3D0 Warmest Regards SecureCiRT Team SecureCiRT Pte Ltd Blk 750C Chai Chee Road #04-01 Technopark@ChaiChee Singapore 469003 Tel: 6243 6800 DID: 6243 6802 Fax: 6441 5119 ---8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<-----8<-- Anyway, as soon as sguilagent is done (or even earlier) I will get going on the abuse_queue enum('Y','N'), abuse_sent enum('Y','N'), fields in database to batch the abuse reports. Want spend less and less time reporting stuff and more and more time analysing stuff. So there will be a daemon that wakes up every so often, look in the DB what needs to be reported, and send of a standard email (to the address listed in the abuse_email table) with the details. This way the IA engineer only need to tell the system that he wants to report it, and the system does the rest without human interaction. /Mike --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Bamm V. <ba...@sa...> - 2003-02-20 16:20:31
|
Not sure if this made it out to everyone the first time. I am having problems with my ISP, so bear with me. Bammkkkk On Thu, Feb 20, 2003 at 09:42:50AM -0600, Bamm Visscher wrote: > On Thu, Feb 20, 2003 at 11:04:34PM +0800, Michael Boman wrote: > > [snip] > > You must have broke into my house and looked at my notes ;) Yes, we do need a feature freeze. For me, the next step is getting rule/sensor management working, and then to simplify the installation steps (Makefiles and an install.sh would be nice). > > > We need a feature freeze and release a new version once the WorkInProgress > > stuff are finished. What I can see as outstanding issues are: > > > > - Sguil.tk rule managment > > - sguilagent in C > > - feature freeze in DB scheme > > - header/tail, default Cc and Subject of email > > > > Let's wrap something we can release (CVS is too fluid at the moment), > > branch off a stable_<version> branch so we can start hacking for our > > hearts content in _HEAD without worry about producing a non-working CVS > > with work-in-progress files. > > > > I too have been mulling over the current directory structure and came up with similar ideas. I think the snort/barnyard_mods should fall under ./sensor. I sent a note to cmg asking if he can incorporate the changes to stream4 into the main snort distro, but haven't recieved a reply. I also would like to submit a "portscan3" to him but I have some other mods I want to include in that. Andrew has offered to incorporate op_sguil into baryand 2.0 (yeah!), and I am waiting for him to stableize it before I port things to it. Seems to be a lot of changes in by2.0. Anyway, lets hope bith "mods" directories are gone sooner than later. > > > > It would also be nice if we could re-arrange the directory structure a > > bit. Suggestion: > > > > ./sql - contains all the SQL creation scripts > > > > ./snort_mods and > > ./barnyard_mods are just fine as they are now > > > > ./sensor - contains everything that can or should be placed on > > sensor. SguilAgent is a good start > > > > ./server - contains everything that can or should be placed on a > > server. This includes sguild and xscriptd > > > > ./console - sguil.tk, the helper scripts (IMHO awhois.sh should be a > > part of the archive) > > > > ./doc - all documentation > > > > I could cook up some Make files so ppl can do: > > > > cd <path-to-squil>; make sensor > > > > which installs all the sensor stuff in their correct places. What do you guys think? > > > > /Mike > > > > Yeah, yeah, yeah. Stupid sourceforge lists should modify the Reply-To: :) > > > PS > > Bamm, don't forget to CC sguil-devel > > DS > > > > -- > > Michael Boman > > Security Architect, SecureCiRT Pte Ltd > > http://www.securecirt.com > > > Bammkkkk > |
From: Bamm V. <ba...@sa...> - 2003-02-20 16:12:24
|
Looks good to me. Bammkkkk > I can do the email stuff today. I will use ex...@ex... for the > default CC and comment the line out in sguil.conf. > Any suggestions about a default Subject? > Any Suggestions about a default header/tail? > > My initial thoughts: > Subject: Incident Report (possibly pop a date/time in here) > Header: "The following intrusion attempt was detected from your > network. All times are UTC and are accurate." > Tail: "For additional information please reply to this email address." > > Maybe we don't need to put anything in these by default, but I think it > would be good to put something in as a suggestion to the end users. For > example, many incident Reports get flat-out ignored by abuse contacts > due to simply omitting that bit about Timezone and Time accuracy. > > > Thought? > > -steve |
From: Michael B. <mic...@se...> - 2003-02-20 16:10:36
|
On Thu, Feb 20, 2003 at 09:16:47AM -0600, Bamm Visscher wrote: > On Thu, Feb 20, 2003 at 11:10:29PM +0800, Michael Boman wrote: > > On Thu, Feb 20, 2003 at 08:03:48AM -0600, Bamm Visscher wrote: > > > Yes it could, but I was trying to refrain from putting those libs on > > > the sensor. Matter of fact, I have been mulling over having BY send RT > > > events to sensor_agent, who would forward them up to sguild for inser= tion > > > into the DB. This way the mysql libs could be removed from the sensor= . I > > > like the idea of having a single comm tunnel from the sensor the cent= ral > > > server. > > >=20 > > > Bammkkkk > >=20 > > Let me get this straight. You are against putting libmysqlclient.so on > > the sensor, but are willing to stick TCL there? I don't get it. TCL is > > tens of times bigger then libmysqlclient... (libmysqlclient.so is 248k > > on this machine, while /usr/lib/tcl8.3 is 2.2 Mb) > >=20 > > /Mike > >=20 > > It's not a "size" thing. I have never really liked the idea of sensors > doing INSERTs, etc. I'd rather have central loading facility. If all > comms were sent over a single comms channel it becomes easier employ > encryption (tcl has an SSL extension for sockets). You and I might have > dedicated VPNs for getting data securely back to the NOC, but I think an > "internal" solution for accomplishing this is a good thing. >=20 > Bammkkkk For me the sensor is a very much a space issue (but that's just me). I also don't see the problem with letting barnyard & other things have direct access to database (as MySQL ACL are pretty good on keeping the sensor from doing 'bad' stuff). The secret of having a secure system is to follow the KISS rule: don't put stuff which you don't use and dont let the application do more then it needs to. I belive that mysqlclient is much more secure and stable then TCL and our own code that has been around for a couple of months at most (versus MySQLs years). I also belive it's the engineer that deploys the sensor responsibility to make sure the communication between sensor and server are meeting the requirements of CIA (Confidentiality, Integrity, Availibility).. Also, I place much more trust in MySQL then I do in any middle-ware daemon... I find sguil socket protocol as a nice way to get real-time events (rather then poll the DB every so often) but I wouldn't paint myself in a corner with this proparity protocol, in which case it needs to be extended forever because it needs to support more things.. (double coding, first extend sguil socket protocol and then add code in your app to use it). It also means that I would need to write parses in god know how many languages that understands sguil protocol instead of using any of the existing standards with their APIs. /Mike --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Steve H. <sha...@33...> - 2003-02-20 16:05:04
|
>> We need a feature freeze and release a new version once the >WorkInProgress >> stuff are finished. What I can see as outstanding issues are: >> >> - Sguil.tk rule managment >> - sguilagent in C >> - feature freeze in DB scheme >> - header/tail, default Cc and Subject of email I can do the email stuff today. I will use ex...@ex... for the default CC and comment the line out in sguil.conf. Any suggestions about a default Subject? Any Suggestions about a default header/tail? My initial thoughts: Subject: Incident Report (possibly pop a date/time in here) Header: "The following intrusion attempt was detected from your network. All times are UTC and are accurate." Tail: "For additional information please reply to this email address." Maybe we don't need to put anything in these by default, but I think it would be good to put something in as a suggestion to the end users. For example, many incident Reports get flat-out ignored by abuse contacts due to simply omitting that bit about Timezone and Time accuracy. Thought? -steve |
From: Bamm V. <ba...@sa...> - 2003-02-20 15:55:19
|
On Thu, Feb 20, 2003 at 11:04:34PM +0800, Michael Boman wrote: [snip] You must have broke into my house and looked at my notes ;) Yes, we do need a feature freeze. For me, the next step is getting rule/sensor management working, and then to simplify the installation steps (Makefiles and an install.sh would be nice). > We need a feature freeze and release a new version once the WorkInProgress > stuff are finished. What I can see as outstanding issues are: > > - Sguil.tk rule managment > - sguilagent in C > - feature freeze in DB scheme > - header/tail, default Cc and Subject of email > > Let's wrap something we can release (CVS is too fluid at the moment), > branch off a stable_<version> branch so we can start hacking for our > hearts content in _HEAD without worry about producing a non-working CVS > with work-in-progress files. > I too have been mulling over the current directory structure and came up with similar ideas. I think the snort/barnyard_mods should fall under ./sensor. I sent a note to cmg asking if he can incorporate the changes to stream4 into the main snort distro, but haven't recieved a reply. I also would like to submit a "portscan3" to him but I have some other mods I want to include in that. Andrew has offered to incorporate op_sguil into baryand 2.0 (yeah!), and I am waiting for him to stableize it before I port things to it. Seems to be a lot of changes in by2.0. Anyway, lets hope bith "mods" directories are gone sooner than later. > It would also be nice if we could re-arrange the directory structure a > bit. Suggestion: > > ./sql - contains all the SQL creation scripts > > ./snort_mods and > ./barnyard_mods are just fine as they are now > > ./sensor - contains everything that can or should be placed on > sensor. SguilAgent is a good start > > ./server - contains everything that can or should be placed on a > server. This includes sguild and xscriptd > > ./console - sguil.tk, the helper scripts (IMHO awhois.sh should be a > part of the archive) > > ./doc - all documentation > > I could cook up some Make files so ppl can do: > > cd <path-to-squil>; make sensor > > which installs all the sensor stuff in their correct places. What do you guys think? > > /Mike > Yeah, yeah, yeah. Stupid sourceforge lists should modify the Reply-To: :) > PS > Bamm, don't forget to CC sguil-devel > DS > > -- > Michael Boman > Security Architect, SecureCiRT Pte Ltd > http://www.securecirt.com Bammkkkk |
From: Bamm V. <ba...@sa...> - 2003-02-20 15:29:18
|
It's not a "size" thing. I have never really liked the idea of sensors doing INSERTs, etc. I'd rather have central loading facility. If all comms were sent over a single comms channel it becomes easier employ encryption (tcl has an SSL extension for sockets). You and I might have dedicated VPNs for getting data securely back to the NOC, but I think an "internal" solution for accomplishing this is a good thing. Bammkkkk On Thu, Feb 20, 2003 at 11:10:29PM +0800, Michael Boman wrote: > On Thu, Feb 20, 2003 at 08:03:48AM -0600, Bamm Visscher wrote: > > Yes it could, but I was trying to refrain from putting those libs on > > the sensor. Matter of fact, I have been mulling over having BY send RT > > events to sensor_agent, who would forward them up to sguild for insertion > > into the DB. This way the mysql libs could be removed from the sensor. I > > like the idea of having a single comm tunnel from the sensor the central > > server. > > > > Bammkkkk > > Let me get this straight. You are against putting libmysqlclient.so on > the sensor, but are willing to stick TCL there? I don't get it. TCL is > tens of times bigger then libmysqlclient... (libmysqlclient.so is 248k > on this machine, while /usr/lib/tcl8.3 is 2.2 Mb) > > /Mike > |
From: Michael B. <mic...@se...> - 2003-02-20 15:11:05
|
On Thu, Feb 20, 2003 at 08:03:48AM -0600, Bamm Visscher wrote: > Yes it could, but I was trying to refrain from putting those libs on > the sensor. Matter of fact, I have been mulling over having BY send RT > events to sensor_agent, who would forward them up to sguild for insertion > into the DB. This way the mysql libs could be removed from the sensor. I > like the idea of having a single comm tunnel from the sensor the central > server. >=20 > Bammkkkk Let me get this straight. You are against putting libmysqlclient.so on the sensor, but are willing to stick TCL there? I don't get it. TCL is tens of times bigger then libmysqlclient... (libmysqlclient.so is 248k on this machine, while /usr/lib/tcl8.3 is 2.2 Mb) /Mike > On Thu, Feb 20, 2003 at 08:25:30AM +0800, Michael Boman wrote: > > On Wed, Feb 19, 2003 at 05:04:11PM -0600, Bamm Visscher wrote: > > > Actually, sguild loads the portscan data into the db. If you then > > > highlight a portscan event in sguil.tk, the packet information will be > > > replaced by a portscan display. Sguild queries the db for the data ba= sed > > > on the src_ip and date of the scan. > >=20 > > But it doesn't do any RT events with it, right? In which case the > > snortagent can upload the portscan data straight into DB instead of go > > via sguild, right? > >=20 > > /Mike --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Michael B. <mic...@se...> - 2003-02-20 15:05:11
|
On Thu, Feb 20, 2003 at 07:55:54AM -0600, Bamm Visscher wrote: > Yeah, that is basically what I was thinking too. I hadn't wanted > xscriptd to access the DB but I can't think of a better way to do it. Did > you ever get Jeffrey added as a developer?=20 Jeffrey is added, but has no access to anything.. > My ISP's mailserver went > bonkers yesterday and I just got a whole bunch of mail. It's odd too, > since I recieved replies to msgs you sent yesterday before I got the > originals today. >=20 > Do you have a quick description of how the rule management stuff works > in ./rulemanager? It works something like this: =3D=3D=3D=3D P O P U L A T E D B =3D=3D=3D=3D 1) First you create the DB scheme (hold on for two hours as I have added stuff I haven't commited yet) 2) Download the rules from snort.org or use oinkmaster or what-ever.. 3) Edit ~/rulemanager/server/loadrules.pl, put in your DB details 4) Run ~/rulemanager/server/loadrules.pl <path-to-your-ruleset> This populates the DB with the rules 5) When you have new updates of rules just repeat step 4. =3D=3D=3D=3D P O P U L A T E S E N S O R =3D=3D=3D=3D 1) Copy ~/rulemanager/sensor/extractrules to sensor 2) Run ./extractrules -S <sensor id> -d </path/to/snort/config/dir> \ -u <db user> -p <db password> -h <db host> -n sguildb This will download sguil.vars and sguil.rules in </path/to/snort/config/dir>. sguil.vars contains all variables while sguil.rules contains all rules. The code in sguilagent (C version) does more then that, like updating sid-msg.map, bpf filter file, classification file etc.. So basicly you can try the rule managment stuff with extractrules, but don't deploy (unless you want to re-deploy) until sguilagent (C version) comes out this weekend. =3D=3D=3D M A N A G E R U L E S =3D=3D=3D 1) Copy ~/rulemanager/wwwgui/* to a directory on your webserver. Make sure webserver has mod_php with MySQL support. See http://rman.sf.net for more details =3D=3D=3D D B D E T A I L S =3D=3D=3D Just a brief description here: rman_rules contains the rules rman_rgroup contains rule groups rman_rrgid is a glue table that connects rules to rulegroups. A single rule can be part of multiple groups and a group can contain multiple rules. I will use this to create "sensor groups" later (ie: a group per sensor so you have in-depth control what rules goes where). rman_senrgrp is another glue table. It keeps track of what rulegroups are assosiated with each sensor. Again it is many groups on one sensor. This is the second part I will exploit for sensor groups. rman_vars contains variable names. rman_varvals contains variable values. With this you can have different definition per sensor. I think all this will make more sense once you can try out the gui. Please take note that sguilagent will be able to do much more then updating rules and variables, so the scheme is not entirely static yet. Which brings me to another thing.=20 We need a feature freeze and release a new version once the WorkInProgress stuff are finished. What I can see as outstanding issues are: - Sguil.tk rule managment - sguilagent in C - feature freeze in DB scheme - header/tail, default Cc and Subject of email Let's wrap something we can release (CVS is too fluid at the moment), branch off a stable_<version> branch so we can start hacking for our hearts content in _HEAD without worry about producing a non-working CVS with work-in-progress files. It would also be nice if we could re-arrange the directory structure a bit. Suggestion: =2E/sql - contains all the SQL creation scripts =2E/snort_mods and=20 =2E/barnyard_mods are just fine as they are now =2E/sensor - contains everything that can or should be placed on sensor. SguilAgent is a good start =2E/server - contains everything that can or should be placed on a server. This includes sguild and xscriptd =20 =2E/console - sguil.tk, the helper scripts (IMHO awhois.sh should be a part of the archive) =2E/doc - all documentation I could cook up some Make files so ppl can do:=20 cd <path-to-squil>; make sensor which installs all the sensor stuff in their correct places. What do you gu= ys think? /Mike PS Bamm, don't forget to CC sguil-devel DS --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Bamm V. <ba...@sa...> - 2003-02-20 14:18:43
|
Yes it could, but I was trying to refrain from putting those libs on the sensor. Matter of fact, I have been mulling over having BY send RT events to sensor_agent, who would forward them up to sguild for insertion into the DB. This way the mysql libs could be removed from the sensor. I like the idea of having a single comm tunnel from the sensor the central server. Bammkkkk On Thu, Feb 20, 2003 at 08:25:30AM +0800, Michael Boman wrote: > On Wed, Feb 19, 2003 at 05:04:11PM -0600, Bamm Visscher wrote: > > Actually, sguild loads the portscan data into the db. If you then > > highlight a portscan event in sguil.tk, the packet information will be > > replaced by a portscan display. Sguild queries the db for the data based > > on the src_ip and date of the scan. > > But it doesn't do any RT events with it, right? In which case the > snortagent can upload the portscan data straight into DB instead of go > via sguild, right? > > /Mike > > -- > Michael Boman > Security Architect, SecureCiRT Pte Ltd > http://www.securecirt.com |
From: Michael B. <mic...@se...> - 2003-02-20 13:37:24
|
On Thu, Feb 20, 2003 at 09:13:35PM +0800, Michael Boman wrote: > On Wed, Feb 19, 2003 at 09:14:03PM -0600, Bamm Visscher wrote: > > Well, it's not broken. I did it that way on purpose. I haven't come > > up with a good plan for handling the other components that need get data > > from the sensor (like xscriptd). > >=20 > > Bammkkkk >=20 > Ok.. so what about adding even more columns to the sensor table, one that > specifies where logs are and one that specifies where configuration files > (and rules) are kept? That way xscriptd can ask the DB (either directly > or through sguild) where they keep their stuff.. And ofcourse the rule > managment software can also consult this table for the information, > including start-up parameters... Something like this I had in mind... CREATE TABLE sensor ( sid INT UNSIGNED NOT NULL AUTO_INCREMENT, hostname VARCHAR(255) NOT NULL, interface VARCHAR(255), description TEXT, bpf_filter TEXT, updated TIMESTAMP(14) NOT NULL, active ENUM('Y','N') DEFAULT 'Y', ip VARCHAR(15) DEFAULT NULL, public_key VARCHAR(255) DEFAULT NULL, + config_dir VARCHAR(255) DEFAULT '/etc/snort', + log_dir VARCHAR(255) DEFAULT '/var/log/snort', + snort_bin VARCHAR(255) DEFAULT '/usr/bin/snort', + snort_args VARCHAR(255), PRIMARY KEY (sid), INDEX hostname_idx (hostname) ); /Mike --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Michael B. <mic...@se...> - 2003-02-20 13:13:46
|
On Wed, Feb 19, 2003 at 09:14:03PM -0600, Bamm Visscher wrote: > Well, it's not broken. I did it that way on purpose. I haven't come > up with a good plan for handling the other components that need get data > from the sensor (like xscriptd). >=20 > Bammkkkk Ok.. so what about adding even more columns to the sensor table, one that specifies where logs are and one that specifies where configuration files (and rules) are kept? That way xscriptd can ask the DB (either directly or through sguild) where they keep their stuff.. And ofcourse the rule managment software can also consult this table for the information, including start-up parameters... Does it sounds ok? /Mike > On Thu, Feb 20, 2003 at 07:22:55AM +0800, Michael Boman wrote: > > Sguild can't handle multiple snort instances on a single sensor (ie: > > both eth0 and eth1 are sniffing). Trying to hack the code now... > >=20 > > /Mike > >=20 > > --=20 > > Michael Boman > > Security Architect, SecureCiRT Pte Ltd > > http://www.securecirt.com >=20 --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Jonathan G. <jon...@se...> - 2003-02-20 01:10:38
|
=2D----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday 20 February 2003 2:42 am, Steve Halligan wrote: > >Mysql w/MyISAM is going to cause problems in the long run and > >I may need to rethink part of the event table schema. > > What would be involved in switching to MySQL w/InnoDB instead of MyISAM? > It is transaction-safe and permits row-level locking IIRC. Yes InnoDB does support row level locking. As to whats involved... in the= =20 table create script use TYPE =3D InnoDB will create the tables. There are= =20 also some stuff that needs setting in the my.cfg / my.ini as well though. Easiest way for those that use the system already is to dump the data, drop= =20 the table, recreate the table with InnoDB support and then re-import. All t= he=20 field settings should be the same (however I have had a few minor issues wi= th=20 full text keys on text fields) ttfn Jonathan =2D --=20 Jonathan Gill =20 SecureCiRT Pte Ltd http://www.securecirt.com/ PGP : 315C 314D CD36 CBFF 728E F167 FCD8 15B7 0287 =2D----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQE+VCsc/Nj+exW3AocRAs8/AKCZDc/+2D/DWiIk78Raaa7x1sx9VACeIZSi 0mfsljBMId5RX7y7H4EpW6U=3D =3DqrrR =2D----END PGP SIGNATURE----- |
From: Michael B. <mic...@se...> - 2003-02-20 00:33:10
|
On Wed, Feb 19, 2003 at 05:04:11PM -0600, Bamm Visscher wrote: > Actually, sguild loads the portscan data into the db. If you then > highlight a portscan event in sguil.tk, the packet information will be > replaced by a portscan display. Sguild queries the db for the data based > on the src_ip and date of the scan. But it doesn't do any RT events with it, right? In which case the snortagent can upload the portscan data straight into DB instead of go via sguild, right? /Mike --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Michael B. <mic...@se...> - 2003-02-19 23:32:22
|
On Thu, Feb 20, 2003 at 07:22:55AM +0800, Michael Boman wrote: > Sguild can't handle multiple snort instances on a single sensor (ie: > both eth0 and eth1 are sniffing). Trying to hack the code now... >=20 > /Mike >=20 Quickly gave up. I have renamed the sensors for now but can this please be fixed? /Mike --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Michael B. <mic...@se...> - 2003-02-19 23:23:01
|
Sguild can't handle multiple snort instances on a single sensor (ie: both eth0 and eth1 are sniffing). Trying to hack the code now... /Mike --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Bamm V. <ba...@sa...> - 2003-02-19 23:20:45
|
Actually, sguild loads the portscan data into the db. If you then highlight a portscan event in sguil.tk, the packet information will be replaced by a portscan display. Sguild queries the db for the data based on the src_ip and date of the scan. An older screen shot of this can be seen here: http://www.satexas.com/~bamf/sguil/images/sguil_ps_win.png Bammkkkk On Thu, Feb 20, 2003 at 03:05:34AM +0800, Jeffrey wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Thursday 20 February 2003 02:35, Michael Boman wrote: > > > > PS > > Does sguild do anything more with portscan data except populating the > > database server? > > DS > > at this point I think it doesnt - although i'm not too sure about plans for > the future. > > The agent sends the portscan over, and then sguild attempts to upload the data > into the db, failing which it will leave a copy of the portscan on the hd. > > - -jf |
From: Bamm V. <ba...@sa...> - 2003-02-19 20:37:05
|
----- Forwarded message from Bamm Visscher <ba...@sa...> ----- Date: Wed, 19 Feb 2003 14:19:10 -0600 From: Bamm Visscher <ba...@sa...> To: Jeffrey Lim <jef...@se...> Subject: Re: [Sguil-devel] RcvSsnFile () behaviour Reply-To: ba...@sa... User-Agent: Mutt/1.2.5.1i In-Reply-To: <200...@se...>; from jef...@se... on Thu, Feb 20, 2003 at 03:34:21AM +0800 I really need to start adding comments to my code now that (other) people are actually looking at it ;) The idea here was to take the stats that stream4 produces and load them into the database along with a unique ID (xid) and Sensor ID (sid). Stream4 (snort) can provide the xid (we use time in milliseconds) but it has no idea what the sid is. I thought about providing the sid as an argument to stream4, but the sid is generated _after_ BY is ran for the first time on a specific sensor, thus we could run into problems as new sensors are added. Since sguild loads the session data, and has access to the DB I went with the idea to prepend the sensor id to session data prior to sguild loading it into the DB. Having sguild write to a file first, then create another (tmp) file, probably isn't the most efficient way to accomplish my goal, but I have a habit of doing things the way I know it will work first and then making the process more efficient second. I expect these functions will change some in the future. Here is the bit of code you wanted clarified. I added the comments to the original code too.: set inFileID [open $DB_OUTFILE r] set outFileID [open $DB_OUTFILE.tmp w] # Use i to keep track of how many lines we are loading into the database for DEBUG. set i 0 # Load the entire file into memory (read $inFileID), then create a list # delimited by \n. Finally loop through each 'line', prepend the sensorID (sid) # to it, and append the new line to the tmp file. foreach line [split [read $inFileID] \n] { if {$line != ""} {puts $outFileID "$sensorID|$line"; incr i} } close $inFileID close $outFileID Bammkkkk On Thu, Feb 20, 2003 at 03:34:21AM +0800, Jeffrey Lim wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > I've been looking at this function, and it beats me why it does what it does, > and was hoping somebody could clarify this: > > basically, the input session file is read, and then lines are written to a > tempfile, consisting of lines "$sensorID|i", with 'i' incrementing. Finally, > this temp file is then loaded to the database. > > Given this file format, and the database schema, it would appear that things > do not fit. Have a look at the sessions (the table to which the tempfile is > loaded to) schema (from scripts/create_sguildb.sql) > > CREATE TABLE sessions ( > sid INT UNSIGNED NOT NULL, > xid BIGINT UNSIGNED NOT NULL, > start_time datetime NOT NULL, > end_time datetime NOT NULL, > src_ip INT UNSIGNED NOT NULL, > dst_ip INT UNSIGNED NOT NULL, > src_port INT UNSIGNED NOT NULL, > dst_port INT UNSIGNED NOT NULL, > src_pckts BIGINT UNSIGNED NOT NULL, > dst_pckts BIGINT UNSIGNED NOT NULL, > src_bytes BIGINT UNSIGNED NOT NULL, > dst_bytes BIGINT UNSIGNED NOT NULL, > PRIMARY KEY (sid,xid), > INDEX begin (start_time), > INDEX end (end_time), > INDEX server (src_ip), > INDEX client (dst_ip), > INDEX sport (src_port), > INDEX cport (dst_port)); > > > Incidentally i also seem to have caught a typo error in the line > -e \"LOAD DATA LOCAL INFILE '$PS_OUTFILE'.tmp INTO ... > is '$PS_OUTFILE'.tmp intended? > > > > still grappling with tcl, > - -jf > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.2.1 (GNU/Linux) > > iD8DBQE+U9w9THq81lr912QRAu9UAJ9SUjgJ5OfDoDoHFcbT4I+RpCC1UgCfXdVn > 5to6o83OqyI+rj1MOLMK5o8= > =C0Ws > -----END PGP SIGNATURE----- > > > > ------------------------------------------------------- > This SF.net email is sponsored by: SlickEdit Inc. Develop an edge. > The most comprehensive and flexible code editor you can use. > Code faster. C/C++, C#, Java, HTML, XML, many more. FREE 30-Day Trial. > www.slickedit.com/sourceforge > _______________________________________________ > Sguil-devel mailing list > Sgu...@li... > https://lists.sourceforge.net/lists/listinfo/sguil-devel ----- End forwarded message ----- |
From: Jeffrey L. <jef...@se...> - 2003-02-19 19:34:53
|
=2D----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've been looking at this function, and it beats me why it does what it doe= s,=20 and was hoping somebody could clarify this: basically, the input session file is read, and then lines are written to a= =20 tempfile, consisting of lines "$sensorID|i", with 'i' incrementing. Finally= ,=20 this temp file is then loaded to the database. Given this file format, and the database schema, it would appear that thing= s=20 do not fit. Have a look at the sessions (the table to which the tempfile is= =20 loaded to) schema (from scripts/create_sguildb.sql) CREATE TABLE sessions ( sid INT UNSIGNED NOT NULL, xid BIGINT UNSIGNED NOT NULL, start_time datetime NOT NULL, end_time datetime NOT NULL, src_ip INT UNSIGNED NOT NULL, dst_ip INT UNSIGNED NOT NULL, src_port INT UNSIGNED NOT NULL, dst_port INT UNSIGNED NOT NULL, src_pckts BIGINT UNSIGNED NOT NULL, dst_pckts BIGINT UNSIGNED NOT NULL, src_bytes BIGINT UNSIGNED NOT NULL, dst_bytes BIGINT UNSIGNED NOT NULL, PRIMARY KEY (sid,xid), INDEX begin (start_time), INDEX end (end_time), INDEX server (src_ip), INDEX client (dst_ip), INDEX sport (src_port), INDEX cport (dst_port)); Incidentally i also seem to have caught a typo error in the line -e \"LOAD DATA LOCAL INFILE '$PS_OUTFILE'.tmp INTO ... is '$PS_OUTFILE'.tmp intended? still grappling with tcl, =2D -jf =2D----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQE+U9w9THq81lr912QRAu9UAJ9SUjgJ5OfDoDoHFcbT4I+RpCC1UgCfXdVn 5to6o83OqyI+rj1MOLMK5o8=3D =3DC0Ws =2D----END PGP SIGNATURE----- |
From: Bamm V. <ba...@sa...> - 2003-02-19 19:28:41
|
I spent about two seconds contimplating moving event.status and event.last_modified into their own table when I realized that we'd have to join this new table w/event everytime we ran a query anyway. So we are back too changing the DB if we expect to see a significant preformance increase when using threads. If we change the DB type then individual query times will slow down, but we would be able to use threads to help speed up queries and updates as a whole. You guys have any comments on this? Bammkkkk On Thu, Feb 20, 2003 at 02:54:39AM +0800, Michael Boman wrote: > On Wed, Feb 19, 2003 at 12:42:11PM -0600, Steve Halligan wrote: > > >Mysql w/MyISAM is going to cause problems in the long run and > > >I may need to rethink part of the event table schema. > > > > What would be involved in switching to MySQL w/InnoDB instead of MyISAM? > > It is transaction-safe and permits row-level locking IIRC. > > > > -steve > > Speed. MySQL with InnoDB is as slow as PostgreSQL basicly, well slightly > faster but not by much. MySQL 4 is supposed to have made it faster, > but I haven't done any benchmarks on it as of yet. > > /Mike > > -- > Michael Boman > Security Architect, SecureCiRT Pte Ltd > http://www.securecirt.com |
From: Jeffrey <jef...@se...> - 2003-02-19 19:06:08
|
=2D----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thursday 20 February 2003 02:35, Michael Boman wrote: > > PS > Does sguild do anything more with portscan data except populating the > database server? > DS at this point I think it doesnt - although i'm not too sure about plans for= =20 the future. The agent sends the portscan over, and then sguild attempts to upload the d= ata=20 into the db, failing which it will leave a copy of the portscan on the hd. =2D -jf =2D----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQE+U9V+THq81lr912QRArjHAJ4t1TpxOzluNvXt6ydOipapPNM0ugCePy7N PaBkkvfQz5nLodK1OxEkBNc=3D =3D+rW0 =2D----END PGP SIGNATURE----- |
From: Michael B. <mic...@se...> - 2003-02-19 18:54:54
|
On Wed, Feb 19, 2003 at 12:42:11PM -0600, Steve Halligan wrote: > >Mysql w/MyISAM is going to cause problems in the long run and=20 > >I may need to rethink part of the event table schema.=20 >=20 > What would be involved in switching to MySQL w/InnoDB instead of MyISAM? > It is transaction-safe and permits row-level locking IIRC. >=20 > -steve Speed. MySQL with InnoDB is as slow as PostgreSQL basicly, well slightly faster but not by much. MySQL 4 is supposed to have made it faster, but I haven't done any benchmarks on it as of yet. /Mike --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |
From: Steve H. <sha...@33...> - 2003-02-19 18:42:23
|
>Mysql w/MyISAM is going to cause problems in the long run and >I may need to rethink part of the event table schema. What would be involved in switching to MySQL w/InnoDB instead of MyISAM? It is transaction-safe and permits row-level locking IIRC. -steve |
From: Michael B. <mic...@se...> - 2003-02-19 18:35:43
|
I was trying to add Jeffrey as a developer last night, but I did not have the access to do that. Can you please give me access so I can add developers to the project? Jeffrey (and any other developer I add) is working under my supervision and I will be resposible for what-ever they break (I will however, of course, hunt down the real culpit - but that's beside the point). Alternative, add jfdevelop_cirt as a developer of the project. Best regards Michael Boman PS Does sguild do anything more with portscan data except populating the database server? DS --=20 Michael Boman Security Architect, SecureCiRT Pte Ltd http://www.securecirt.com |