Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

#43 Ignored x duplicate alert(s)

BASE
closed-fixed
Tim Rupp
Interface (166)
7
2006-08-02
2005-03-15
Christian
No

Hi

Thought it was about time we made this a official bug
since Joel was able to reproduce it. I have tested but
cant reproduce it so its abit hard for me to test.... never
the less it needs fixing.

/Christian

snipp from the Forum.

By: hanji - hanji866
Ignored x duplicate alert(s)
2005-03-03 08:24
Hello

Currently I have two instances of BASE running, one is
to monitor incoming snort data, and the other to view
archived snort data (within archive_snort db). When I find
something I want to archive off of the 'real-time'
database, I select them and move them to the archive
db. This works fine, but occasionally it will only move
partial.. or no alerts to my archive, giving me the
message "Ignored x duplicate alert(s)", x being the
number of alerts it could not move.

I've updated the alert cache and rebuilt the alert cache
on both databases, I've also did the same to IP cache.
Both databases have bee 'repaired' as well.

Does anyone have a way to fix this??

thanks much
hanji

By: hanji - hanji866
RE: Ignored x duplicate alert(s)
2005-03-03 08:27
Here is a complete message when problem occurs:

Added 0 alert(s) to the Alert cache
Ignored 27 duplicate alert(s)
Successful Archive alert(s) (move) - 19 alert(s)

This was 46 alerts, all with same alert and dst/src IPs.

Thanks again
hanji

By: Christian - passreality
RE: Ignored x duplicate alert(s)
2005-03-07 10:30
Hanji i have only seen this when the alerts you
copy/move over already are in the archive db then you
will get that msg and that is normal.

Then it will move/copy over all your selected alerts but it
will ignore the duplicates since they alread are in the
archive db.

/Christian

By: hanji - hanji866
RE: Ignored x duplicate alert(s)
2005-03-08 07:55
Hello

Thanks for the response. I can verify that I have gotten
this message on completely new alerts. Also, for
archivability, how could they be duplicate, unless they
occured at that same time with the same alert? I do this
procedure every day, trying to move new alerts
from 'working' BASE to 'archive' BASE. When I'm done
my 'working' BASE is completely empty, which I return
to the next day, moving 'suspect' alerts to the archive
and deleting the rest.

When this first happened I rebuilt the Alert cache on
the 'archive' BASE.. and the problem went away for
several months. Now it has returned, and rebuilding the
alert cache does not fix the problem.

The example error message I included in the above post,
was all from a single IP with same alert(Cyberkit alert)
destined to the same IP, yet it moved less than half?

Thanks!
hanji

By: Kevin Johnson - secureideas
RE: Ignored x duplicate alert(s)
2005-03-08 11:50
What version of BASE are you running? We currently
can not reproduce the issue you are describing. Any
help is appreciated.

Thanks
Kevin

By: hanji - hanji866
RE: Ignored x duplicate alert(s)
2005-03-08 13:40
Hello

Thanks for the response Kevin.

BASE version is:BASE 1.0.2 (racquel)
Snort version is: snort-2.3.0
MySQL version is: mysql-4.0.22-r2

Currently.. this is what I have in my archive DB for
records:

Unique Alerts: 90
Categories: 13
Total Number of Alerts: 38567
Src IP addrs: 770
Dest. IP addrs: 2002
Unique IP links 28639

I just rebuilt the IP cache again today. Still experiencing
the same problem.

Thanks again
hanji

By: Joel Esler - joelesler
RE: Ignored x duplicate alert(s)
2005-03-14 13:58
I have this problem too. I never figured out what it was.

Discussion

  • Joel Esler
    Joel Esler
    2005-03-30

    • priority: 5 --> 2
     
  • Joel Esler
    Joel Esler
    2005-03-30

    • assigned_to: nobody --> caphrim007
     
  • m. vernimmen
    m. vernimmen
    2005-05-20

    Logged In: YES
    user_id=292264

    The cause for the problem seems to be thus:
    http://www.networksecurityarchive.org/html/Snort-Users/2005-04/msg00277.html

    in short:
    snort keeps track of where it was by looking at the highest
    cid in de database, and starts counting from there. If
    entries get moved to the archive and snort gets restarted
    any time after that, it starts at the then highest. If all
    records were moved, it starts at 0 again.
    This means that for all cid's that are already in the
    archive, a duplicate is found (which isn't a duplicate at all).

    so the fix would be:
    If stuff gets moved from to the archive, make sure there is
    an entry in the realtime database to keep snort on the right
    track in case in dies/restarts.

    To fix by hand: update the realtime database by setting the
    cid's to a higher value than those in the archive.

     
  • Joel Esler
    Joel Esler
    2005-05-30

    • priority: 2 --> 7
     
  • Joel Esler
    Joel Esler
    2005-05-30

    Logged In: YES
    user_id=853584

    We need to discuss this one with the developers at Snort.
    See if this is a viable alternative. If so, we can do an
    insert statement, but would that mess up Snort? Can
    Sourcefire figure out a better db alternative.

     
  • Nerveup
    Nerveup
    2006-03-09

    Logged In: YES
    user_id=1429350

    Quoting from:
    http://www.snort.org/reg-bin/forums.cgi?forum_id=4&topic_id=1934

    "If you are working with unix systems then you may take a
    look at FLoP. Here the last_cid is used
    in the correct way..."

     
  • Logged In: NO

    Just curious if there has been any headway on this bug.
    I've done quite a bit of searching on this and many people
    had work arounds or ideas, but none were actual fixes.

    The reason I ask is because I just spent quite a bit of time
    evaluating a number of free/OSS and commercial IDS consoles,
    and, surprisingly, BASE seemed to be the most featureful and
    usable. There are many nice commercial solutions out there,
    but most have some major downfalls or limitations, not the
    least of which is price...

    Anyway, thanks for anything you can do.

    jhart@spoofed.org

     
  • Logged In: NO

    I also had the same problem with the BASE 1.2.4 version.
    At the begin, i thought that was a problem with my DBs,
    then i drop then and did a new instalation with two DB
    (snort and snort_bkp) and the problem reapers after i
    delete some records in the snort db (via BASE).

    cestevam@fox.com.br

     
  • cestevam
    cestevam
    2006-06-15

    Logged In: YES
    user_id=1540124

    Hi
    Sorry by my poor english, i'm a brazilian.

    I had a problem like that and post a comment (like nobody).
    After that, i try to solve the problem turning the debug on
    and i could see the message:

    "Added 0 alert(s) to the Alert cache

    Archive error:Database ERROR:Entrada '21-1' duplicada para
    a chave 1

    INSERT INTO sig_reference (sig_id, ref_seq, ref_id) VALUES
    (21,1,839)
    Ignored 1 duplicate alert(s)
    No alerts were selected or the Archive alert(s) (move) was
    not successful"

    Then i open the snort DB and note that the
    table "sig_reference" was using a primary key with the
    fields "sig_id" and "ref_seq" only.
    Then, i include the field "ref_id" to the key and i had no
    problem with moving alerts anymore.

    I hope this help you too.

     
  • Kevin Johnson
    Kevin Johnson
    2006-07-19

    • status: open --> pending-fixed
     
  • Kevin Johnson
    Kevin Johnson
    2006-07-19

    Logged In: YES
    user_id=836228

    I believe that a fix that was put into CVS and will be part of 1.2.6 will fix this.

    Kevin

     
  • Logged In: YES
    user_id=1312539

    This Tracker item was closed automatically by the system. It was
    previously set to a Pending status, and the original submitter
    did not respond within 14 days (the time period specified by
    the administrator of this Tracker).

     
    • status: pending-fixed --> closed-fixed