Recent changes to FAQ

FAQ modified by Nikolay Klendar

Nikolay Klendar — Mon, 23 Nov 2015 21:23:17 -0000

--- v22
+++ v23
@@ -2,7 +2,7 @@
 CorReactive is a correlation engine which is based on [Esper library](http://www.espertech.com/esper/) to perform complex event processing. CorReactive is a correlation engine, it does not parse events, it expects events represented in JSON format.
 # How to install CorReactive?
 It is required that JAVA 1.8 must be installed
-Simply extract distributive to any directory and launch engine with bin/launcher.bat or bin/launcher.sh script
+Simply extract distributive to any directory and launch engine with bin/run.bat. Linux launcher is in progress (you can modify run.but to sh version it is not hard).
 Currently only windows version was tested. If you have any problem running it in Linux please write me.
 # How to feed events to CorReactive?
 CorReactive currently supports two inputs: 
@@ -18,8 +18,8 @@
                "port": 6379,
                "db": 0,
                "queue":"events",
-               "batch":1000,
-               "reconnect_timeout":600
+               "batch_count":1000,
+               "reconnect_timeout":60
            }
        }
    ]
@@ -128,7 +128,7 @@
                "queue":"alerts",
                "port": 6379,
                "db": 0,
-                "batch_count":100,
+                "batch_count":1,
                "reconnect_timeout":600
            }
        }
@@ -212,8 +212,8 @@
             "port": 6379,
             "db": 0,
             "queue":"events",
-            "batch":500,
-            "reconnect_timeout":600
+            "batch_count":500,
+            "reconnect_timeout":60
         }
     }
 ]
@@ -231,7 +231,8 @@
             "queue":"alerts",
             "port": 6379,
             "db": 0,
-            "reconnect_timeout":600
+            "reconnect_timeout":60,
+            "batch_count":1
         }
     }
 ]

FAQ modified by Nikolay Klendar

Nikolay Klendar — Sun, 22 Nov 2015 19:43:29 -0000

--- v21
+++ v22
@@ -114,7 +114,7 @@
 * *type* is type of enrichment (window or cmd)
 * *param* is a parameter string which will be passed to enrichment plugin. To refference field from alert you can use %{field_name} syntax which will be converted to corresponding filed value.

-# How to get alerts from CorReactive
+# How to get alerts from CorReactive?
 All statements which prepended with @Alert annotation could genereate alerts. If an alert is genereated it is placed to output queue. Particular output is selected based on a outID parameter of @Alert annotation. Currently supported only redis outptut, which you can configure like this in conf/engine.json

FAQ modified by Nikolay Klendar

Nikolay Klendar — Sun, 22 Nov 2015 19:41:57 -0000

--- v20
+++ v21
@@ -1,5 +1,9 @@
 # What is a CorReactive?
 CorReactive is a correlation engine which is based on [Esper library](http://www.espertech.com/esper/) to perform complex event processing. CorReactive is a correlation engine, it does not parse events, it expects events represented in JSON format.
+# How to install CorReactive?
+It is required that JAVA 1.8 must be installed
+Simply extract distributive to any directory and launch engine with bin/launcher.bat or bin/launcher.sh script
+Currently only windows version was tested. If you have any problem running it in Linux please write me.
 # How to feed events to CorReactive?
 CorReactive currently supports two inputs:

@@ -181,7 +185,7 @@
 ~~~~

 # How to integrate CorReactive with Logstash?
-1) Config Logstash output to feed events to CorReactive. Please read [redis output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-redis.html) manual. In high load environment do not forget to use batch_events parameter!
+1) Configure Logstash output to feed events to CorReactive. Please read [redis output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-redis.html) manual. In high load environment do not forget to use batch_events parameter!

 ~~~~
 output {  
@@ -197,5 +201,41 @@
 }
 ~~~~

-2) asd
-
+2) Configure CorReactive to read events from Redis. You can read events from multiple Redis sources
+
+~~~~
+"inputs":[
+    {
+        "type": "redis",
+        "config":{ 
+            "host": "localhost",
+            "port": 6379,
+            "db": 0,
+            "queue":"events",
+            "batch":500,
+            "reconnect_timeout":600
+        }
+    }
+]
+~~~~
+
+3) Configure CorReactive to place alert in Redis. Next grab them with Logstash redis input.
+
+~~~~
+"outputs":[
+    {
+        "type":"redis",
+        "id":1,
+        "config":{
+            "host": "localhost",
+            "queue":"alerts",
+            "port": 6379,
+            "db": 0,
+            "reconnect_timeout":600
+        }
+    }
+]
+~~~~
+
+4) Configure esper event types in conf/types directory
+5) Configutre esper modules in conf/modules

FAQ modified by Nikolay Klendar

Nikolay Klendar — Sun, 22 Nov 2015 19:30:42 -0000

--- v19
+++ v20
@@ -181,8 +181,7 @@
 ~~~~

 # How to integrate CorReactive with Logstash?
-1. Config Logstash output to feed events to CorReactive. Please read [redis output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-redis.html) manual. In high load environment do not forget to use batch_events parameter!
-
+1) Config Logstash output to feed events to CorReactive. Please read [redis output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-redis.html) manual. In high load environment do not forget to use batch_events parameter!

 ~~~~
 output {  
@@ -198,6 +197,5 @@
 }
 ~~~~

-
-2. sdf
-
+2) asd
+

FAQ modified by Nikolay Klendar

Nikolay Klendar — Sun, 22 Nov 2015 19:30:01 -0000

--- v18
+++ v19
@@ -183,6 +183,7 @@
 # How to integrate CorReactive with Logstash?
 1. Config Logstash output to feed events to CorReactive. Please read [redis output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-redis.html) manual. In high load environment do not forget to use batch_events parameter!

+
 ~~~~
 output {  
     redis {
@@ -197,5 +198,6 @@
 }
 ~~~~

+
 2. sdf

FAQ modified by Nikolay Klendar

Nikolay Klendar — Sun, 22 Nov 2015 19:29:33 -0000

--- v17
+++ v18
@@ -191,7 +191,7 @@
         data_type => "list"
         batch => true
         batch_events=>500
-        key => "events”
+        key => "events"
         codec => json  
     }
 }

FAQ modified by Nikolay Klendar

Nikolay Klendar — Sun, 22 Nov 2015 19:28:48 -0000

--- v16
+++ v17
@@ -41,9 +41,9 @@
 You can define as much event types as you want, simply add new json file to conf/types/ directory and it will be converted to Esper type during loading.
 When you feed any json to CorReactive it analyzes @cr_type field and tryes to convert an event to corresponding type which must be defined in conf/types/@cr_type.json file. If @cr_type is not present then engine will treat the event as type of "event". 
 # How to write correlation rules?
-CorReactive is based on Esper library. The best way to understood Esper syntax is to read official [documentation](http://www.espertech.com/esper/documentation.php). Lots of ready to use examples are available [here](http://www.espertech.com/esper/solution_patterns.php).
+CorReactive is based on Esper library. The best way to understand Esper is to read official [documentation](http://www.espertech.com/esper/documentation.php). Lots of ready to use examples are available [here](http://www.espertech.com/esper/solution_patterns.php).
 Correlation rules are placed in files located in conf/modules directory. Each file is treated like [EPL module](http://www.espertech.com/esper/release-5.3.0/esper-reference/html/devlifecycle.html#devlifecycle-eplmodule) tha's why it is possible to place more than one EPL query or statement in one file, please read manual.
-If you want that defined statement fires alert, than special @Alert annotation must be placed before statement definition lit this:
+If you want to fire an alert, than special @Alert annotation must be placed before statement definition like this:

 ~~~~
 @Alert(name='NetworkScan',outID=1)
@@ -59,6 +59,10 @@

 * name parameter is a name of alert, it will be placed in special "alert" field of alert.
 * outID is id of outptut to which alert will be redirected
+
+# How to try Esper rules or debug them
+Please use official [Esper app](http://esper-epl-tryout.appspot.com/epltryout/mainform.html) from Esper developers.
+
 #What does @Persist annotation?
 It is special annotation which was prepaired by author of CorReactive to automatically save data in Esper window every 5 minutes. Saved data is automatically restored during loading stage. You can use it like this:

@@ -75,7 +79,6 @@
     AND login IS NOT NULL AND login!='ANONYMOUS LOGON' 
     AND login NOT LIKE '%$' );
 ~~~~
-

 # What does @Load annotation?
 It is special annotation which was prepaired by author of CorReactive to automatically load data to Esper window every 5 minutes from csv file, which must be located in var/winload directory. You can use it like this:
@@ -130,19 +133,19 @@
 It is possible to do enrichment data wich was got by enrichment.

 #REST API
-Send JSON event to engine
+**Send JSON event to engine**
 `POST /api/events`

-View registered modules
+**View registered modules**
 `GET /api/modules/registered`

-View all registered statements
+**View all registered statements**
 `GET api/modules/statements`

-Manual reload data in window (see @Load annotation)
+**Manual reload data in window (see @Load annotation)**
 `POST /api/window/reload/{moduleName}/{windowName}`

-Do ondemand query
+**Perform on demand query**

 ~~~~
 POST /api/query
@@ -151,7 +154,8 @@
 }
 ~~~~

-Deploy ALL modyles. **Warning**:all modules which will not be present in request will be deleted from disk. This method was created for web based GUI which is in plan, so please use it carefully.
+**Deploy ALL modyles. **
+**Warning**:all modules which will not be present in request will be deleted from disk. This method was created for web based GUI which is in plan, so please use it carefully.

 ~~~~
 POST api/modules/deploy
@@ -166,7 +170,7 @@
 ]
 ~~~~

-Validate module content
+**Validate module content**

 ~~~~
 POST api/modules/validate
@@ -176,3 +180,22 @@
 }
 ~~~~

+# How to integrate CorReactive with Logstash?
+1. Config Logstash output to feed events to CorReactive. Please read [redis output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-redis.html) manual. In high load environment do not forget to use batch_events parameter!
+
+~~~~
+output {  
+    redis {
+        host => "127.0.0.1"
+        db => 0
+        data_type => "list"
+        batch => true
+        batch_events=>500
+        key => "events”
+        codec => json  
+    }
+}
+~~~~
+
+2. sdf
+

FAQ modified by Nikolay Klendar

Nikolay Klendar — Sun, 22 Nov 2015 19:18:53 -0000

--- v15
+++ v16
@@ -3,8 +3,25 @@
 # How to feed events to CorReactive?
 CorReactive currently supports two inputs:

-* [Redis input](http://)
-* [Http POST method](http://)
+* Redis input which must be configured in conf/engine.json 
+
+~~~~
+   "inputs":[
+       {
+           "type": "redis",
+           "config":{ 
+               "host": "localhost",
+               "port": 6379,
+               "db": 0,
+               "queue":"events",
+               "batch":1000,
+               "reconnect_timeout":600
+           }
+       }
+   ]
+~~~~
+
+* Http POST method. Simple post event JSON to http://engine_host:httpPort/api/events with tool like curl or RESTClient plugin for firefox. By default 2015 port is used but you can change it in conf/engine.json

 The simpliest way to feed events to CorReactuve is to use [Logstash](https://www.elastic.co/products/logstash) with [redis output](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-redis.html).
 # How to describe event schema
@@ -110,4 +127,52 @@
        }
    ]
 ~~~~
+It is possible to do enrichment data wich was got by enrichment.

+#REST API
+Send JSON event to engine
+`POST /api/events`
+
+View registered modules
+`GET /api/modules/registered`
+
+View all registered statements
+`GET api/modules/statements`
+
+Manual reload data in window (see @Load annotation)
+`POST /api/window/reload/{moduleName}/{windowName}`
+
+Do ondemand query
+
+~~~~
+POST /api/query
+{
+    'query':'select * from LoginsIP'
+}
+~~~~
+
+Deploy ALL modyles. **Warning**:all modules which will not be present in request will be deleted from disk. This method was created for web based GUI which is in plan, so please use it carefully.
+
+~~~~
+POST api/modules/deploy
+[
+    {
+        'uri':'module_name1',
+        'text':'%Module contents%'
+    },
+    {
+        ...
+    }        
+]
+~~~~
+
+Validate module content
+
+~~~~
+POST api/modules/validate
+{
+    'uri':'test module',
+    'text':'text for validation'
+}
+~~~~
+

FAQ modified by Nikolay Klendar

Nikolay Klendar — Sun, 22 Nov 2015 19:03:48 -0000

--- v14
+++ v15
@@ -90,3 +90,24 @@
 * *type* is type of enrichment (window or cmd)
 * *param* is a parameter string which will be passed to enrichment plugin. To refference field from alert you can use %{field_name} syntax which will be converted to corresponding filed value.

+# How to get alerts from CorReactive
+All statements which prepended with @Alert annotation could genereate alerts. If an alert is genereated it is placed to output queue. Particular output is selected based on a outID parameter of @Alert annotation. Currently supported only redis outptut, which you can configure like this in conf/engine.json
+
+
+~~~~
+"outputs":[
+       {
+           "type":"redis",
+           "id":1,
+           "config":{
+               "host": "localhost",
+               "queue":"alerts",
+               "port": 6379,
+               "db": 0,
+                "batch_count":100,
+               "reconnect_timeout":600
+           }
+       }
+   ]
+~~~~
+

FAQ modified by Nikolay Klendar

Nikolay Klendar — Sun, 22 Nov 2015 18:56:33 -0000

--- v13
+++ v14
@@ -45,6 +45,7 @@
 #What does @Persist annotation?
 It is special annotation which was prepaired by author of CorReactive to automatically save data in Esper window every 5 minutes. Saved data is automatically restored during loading stage. You can use it like this:

+
 ~~~~
 @Persist
 CREATE WINDOW 
@@ -57,6 +58,7 @@
     AND login IS NOT NULL AND login!='ANONYMOUS LOGON' 
     AND login NOT LIKE '%$' );
 ~~~~
+

 # What does @Load annotation?
 It is special annotation which was prepaired by author of CorReactive to automatically load data to Esper window every 5 minutes from csv file, which must be located in var/winload directory. You can use it like this:
@@ -83,7 +85,8 @@
 `@Enrich(dst="nsLookupOut",type="cmd",param="nslookup %{src_ip}")`

 where 
+
 * *dst* is name of field where enrichment data will be placed
-* type is type of enrichment (window or cmd)
-* param is a parameter string which will be passed to enrichment plugin. To refference field from alert you can use %{field_name} syntax which will be converted to corresponding filed value.
+* *type* is type of enrichment (window or cmd)
+* *param* is a parameter string which will be passed to enrichment plugin. To refference field from alert you can use %{field_name} syntax which will be converted to corresponding filed value.