Recent changes to Encryption

Encryption modified by James Edmondson

James Edmondson — Fri, 04 May 2018 17:23:27 -0000

--- v12
+++ v13
@@ -7,13 +7,17 @@
 MADARA allows for encryption/decryption and arbitrary encoding of data per record, per aggregation of records, and per buffers. The most common type of encryption people tend to want is via buffer filters, which will encode or decode an entire buffer of information, usually to/from disk or over the network. In this guide, we'll focus on extending the BufferFilter and using the pre-build AESBufferFilter, which provides 256 bit AES encryption.

 ----
-# BufferFilters
+
+[TOC]
+
+----
+# 1. BufferFilters
 

 BufferFilters are encoders and decoders of character buffers. Individually, these are powerful software constructs that allow you to change how data is represented. You can compress data with BufferFilters, you can encrypt it, or you can remove data entirely so that no one ever sees it. MADARA also allows chaining together BufferFilters so you might compress and then encrypt an image, for instance. In this powerful filter chain concept, MADARA also provides mechanisms to ensure that you reverse the process in the appropriate order (e.g., decrypting and then uncompressing the image from the previous chain, instead of the opposite which wouldn't work at all).

 ----
-# AES 256 Bit Encryption
+# 2. AES 256 Bit Encryption
 

 MADARA includes a 256 Bit AES encryption BufferFilter that can be used for all network transports and for saving/loading knowledge bases from files. The class you are looking for is `madara::filters::AESBufferFilter` which can be used in your code by including `madara/filters/ssl/AESBufferFilter.h`.  You will need to build for ssl support by enabling the ssl feature first. If you are using the GAMS build scripts, you can simply call `$GAMS_ROOT/scripts/linux/base_build.sh madara ssl`.
@@ -23,7 +27,7 @@
 To add AES encryption to save/load checkpoints and save_context, you'll need to use the `madara::knowledge::CheckpointSettings` class. Each [CheckpointSettings](http://madara.sourceforge.net/docs/cpp/d1/d44/structmadara_1_1knowledge_1_1CheckpointSettings.html) contains a data member called buffer_filters. Simply add the BufferFilter to the buffer_filters list with push_back ().

 ----
-# Programming Examples
+# 3. Programming Examples

 **Saving an encrypted knowledge base to disk**
 ~~~~~~ 
@@ -114,6 +118,6 @@

 ~~~~~~ 

-# Advanced Topics
+# 4. Advanced Topics
 In the above examples, key and IV are generated by providing a shared secret to the `generate_key` function, which is a class function in `AESBufferFilter`. However, if you have generated your own key with SSL, e.g. from a command line, you should be able to pass this in to the constructor of `AESBufferFilter`.  Specifically, you'll pass this in via the `AESBufferFilter (unsigned char * key, int key_length)` constructor. You should only pass 32 byte keys into this function, but if you do, the filter will try to handle this as gracefully as possible by only using the first 32 bytes of the key.

Encryption modified by James Edmondson

James Edmondson — Fri, 04 May 2018 17:19:01 -0000

--- v11
+++ v12
@@ -1,5 +1,5 @@
 C++ Guide Series
-[Architecture](MadaraArchitecture) | [Knowledge Base](InteractingWithTheKnowledgeBase) | [Networking](InteractingWithTheTransport) | [Containers](KnowledgeContainers) | [Threads](WorkingWithThreads) | [Optimizations](OptimizingKaRL) | [KaRL](KarlLanguage) | Encryption
+[Architecture](MadaraArchitecture) | [Knowledge Base](InteractingWithTheKnowledgeBase) | [Networking](InteractingWithTheTransport) | [Containers](KnowledgeContainers) | [Threads](WorkingWithThreads) | [Optimizations](OptimizingKaRL) | [KaRL](KarlLanguage) | Encryption | [Checkpointing](Checkpointing)

 ----
 # Overview

Encryption modified by James Edmondson

James Edmondson — Sat, 21 Apr 2018 01:38:45 -0000

--- v10
+++ v11
@@ -113,3 +113,7 @@
 }

 ~~~~~~ 
+
+# Advanced Topics
+In the above examples, key and IV are generated by providing a shared secret to the `generate_key` function, which is a class function in `AESBufferFilter`. However, if you have generated your own key with SSL, e.g. from a command line, you should be able to pass this in to the constructor of `AESBufferFilter`.  Specifically, you'll pass this in via the `AESBufferFilter (unsigned char * key, int key_length)` constructor. You should only pass 32 byte keys into this function, but if you do, the filter will try to handle this as gracefully as possible by only using the first 32 bytes of the key.
+

Encryption modified by James Edmondson

James Edmondson — Sat, 21 Apr 2018 01:32:02 -0000

--- v9
+++ v10
@@ -16,9 +16,9 @@
 # AES 256 Bit Encryption
 

-MADARA includes a 256 Bit AES encryption BufferFilter that can be used for all network transports and for saving/load knowledge bases from files. The class you are looking for is `madara::filters::AESBufferFilter` which can be used in your code by including `madara/filters/ssl/AESBufferFilter.h`. 
+MADARA includes a 256 Bit AES encryption BufferFilter that can be used for all network transports and for saving/loading knowledge bases from files. The class you are looking for is `madara::filters::AESBufferFilter` which can be used in your code by including `madara/filters/ssl/AESBufferFilter.h`.  You will need to build for ssl support by enabling the ssl feature first. If you are using the GAMS build scripts, you can simply call `$GAMS_ROOT/scripts/linux/base_build.sh madara ssl`.

-To add this AES encryption to any network transport, simply call `add_filter` on `madara::transport::QoSTransportSettings`.  For networking, BufferFilters are the absolute last filter that is called during a send. Consequently, encode happens after any AggregateFilters or individual RecordFilters are performed. Decodes happen in the reverse order. This is important for martialling/demartialling, encoding/decoding, and encryption/decryption. Essentially, other filters aren't possible until you reverse the effects of encoding.
+To add this AES encryption to any network transport, simply call `add_filter` on `madara::transport::QoSTransportSettings`.  For networking, BufferFilters are the absolute last filter that is called during a send operation. Consequently, encode happens after any AggregateFilters or individual RecordFilters are performed. Decodes happen in the reverse order. This is important for martialling/demartialling, encoding/decoding, and encryption/decryption. Essentially, other filters aren't possible until you reverse the effects of encoding.

 To add AES encryption to save/load checkpoints and save_context, you'll need to use the `madara::knowledge::CheckpointSettings` class. Each [CheckpointSettings](http://madara.sourceforge.net/docs/cpp/d1/d44/structmadara_1_1knowledge_1_1CheckpointSettings.html) contains a data member called buffer_filters. Simply add the BufferFilter to the buffer_filters list with push_back ().

Encryption modified by James Edmondson

James Edmondson — Sat, 21 Apr 2018 01:19:49 -0000

--- v8
+++ v9
@@ -6,11 +6,13 @@

 MADARA allows for encryption/decryption and arbitrary encoding of data per record, per aggregation of records, and per buffers. The most common type of encryption people tend to want is via buffer filters, which will encode or decode an entire buffer of information, usually to/from disk or over the network. In this guide, we'll focus on extending the BufferFilter and using the pre-build AESBufferFilter, which provides 256 bit AES encryption.

+----
 # BufferFilters
 

 BufferFilters are encoders and decoders of character buffers. Individually, these are powerful software constructs that allow you to change how data is represented. You can compress data with BufferFilters, you can encrypt it, or you can remove data entirely so that no one ever sees it. MADARA also allows chaining together BufferFilters so you might compress and then encrypt an image, for instance. In this powerful filter chain concept, MADARA also provides mechanisms to ensure that you reverse the process in the appropriate order (e.g., decrypting and then uncompressing the image from the previous chain, instead of the opposite which wouldn't work at all).

+----
 # AES 256 Bit Encryption
 

@@ -20,6 +22,7 @@

 To add AES encryption to save/load checkpoints and save_context, you'll need to use the `madara::knowledge::CheckpointSettings` class. Each [CheckpointSettings](http://madara.sourceforge.net/docs/cpp/d1/d44/structmadara_1_1knowledge_1_1CheckpointSettings.html) contains a data member called buffer_filters. Simply add the BufferFilter to the buffer_filters list with push_back ().

+----
 # Programming Examples

 **Saving an encrypted knowledge base to disk**

Encryption modified by James Edmondson

James Edmondson — Sat, 21 Apr 2018 01:12:17 -0000

--- v7
+++ v8
@@ -63,3 +63,50 @@

 **Sending encrypted data over the network**
 ~~~~~~ 
+#include "madara/knowledge/KnowledgeBase.h"
+#include "madara/knowledge/containers/Integer.h"
+#include "madara/knowledge/containers/String.h"
+#include "madara/transport/QoSTransportSettings.h"
+#include "madara/filters/ssl/AESBufferFilter.h"
+#include "madara/utility/Utility.h"
+
+namespace knowledge = madara::knowledge;
+namespace filters = madara::filters;
+namespace transport = madara::transport;
+
+int main (int argc, char ** argv)
+{
+  knowledge::KnowledgeBase context;
+  transport::QoSTransportSettings settings;
+  
+  // setup a couple of containers that point into the context
+  // we can use these to set/get values
+  knowledge::containers::String name ("name", context);
+  knowledge::containers::Integer age ("age", context);
+  
+  // create an AES 256 bit filter and a password
+  filters::AESBufferFilter encryption;
+  encryption.generate_key ("our communal shared secret. so safe.");
+
+  // setup a network to send knowledge over
+  settings.hosts.push_back ("239.255.0.1:4150");
+  settings.type = transport::MULTICAST;
+  settings.add_filter (encryption);
+  
+  // attach the multicast transport with encryption
+  context.attach_transport ("", settings);
+  
+  // change some values so we can aggregate knowledge to send
+  name = "Alfred Collins";
+  age = 42;
+  
+  // send all aggregated knowledge over the transport
+  context.send_modifieds ();
+  
+  // sleep for 5.5 seconds
+  madara::utility::sleep (5.5);
+  
+  return 0;
+}
+
+~~~~~~

Encryption modified by James Edmondson

James Edmondson — Sat, 21 Apr 2018 00:55:17 -0000

--- v6
+++ v7
@@ -20,3 +20,46 @@

 To add AES encryption to save/load checkpoints and save_context, you'll need to use the `madara::knowledge::CheckpointSettings` class. Each [CheckpointSettings](http://madara.sourceforge.net/docs/cpp/d1/d44/structmadara_1_1knowledge_1_1CheckpointSettings.html) contains a data member called buffer_filters. Simply add the BufferFilter to the buffer_filters list with push_back ().

+# Programming Examples
+
+**Saving an encrypted knowledge base to disk**
+~~~~~~ 
+#include "madara/knowledge/KnowledgeBase.h"
+#include "madara/filters/ssl/AESBufferFilter.h"
+
+namespace knowledge = madara::knowledge;
+namespace filters = madara::filters;
+
+int main (int argc, char ** argv)
+{
+  knowledge::KnowledgeBase saver, loader;
+  knowledge::CheckpointSettings settings;
+  
+  // create an AES 256 bit filter and a password
+  filters::AESBufferFilter filter;
+  filter.generate_key ("our communal shared secret. so safe.");
+
+  // create some variables in the 
+  saver.set ("int_var", knowledge::KnowledgeRecord::Integer (15));
+  saver.set ("str_var", std::string ("some string"));
+  saver.set ("double_var", 3.14159);
+
+  // setup a file to save and add the AES filter
+  settings.filename = "encrypted.kb";
+  settings.buffer_filters.push_back (&filter);
+
+  // save the context with the settings (this is all you need to encrypt)
+  saver.save_context (settings);
+
+  // to load from the file, you need to use an AESBufferFilter with the same shared secret!
+  loader.load_context (settings);
+  
+  // print the variables in the loaded configuration
+  loader.print ();
+  
+  return 0;
+}
+~~~~~~ 
+
+**Sending encrypted data over the network**
+~~~~~~

Encryption modified by James Edmondson

James Edmondson — Sat, 21 Apr 2018 00:39:47 -0000

--- v5
+++ v6
@@ -14,3 +14,9 @@
 # AES 256 Bit Encryption
 

+MADARA includes a 256 Bit AES encryption BufferFilter that can be used for all network transports and for saving/load knowledge bases from files. The class you are looking for is `madara::filters::AESBufferFilter` which can be used in your code by including `madara/filters/ssl/AESBufferFilter.h`. 
+
+To add this AES encryption to any network transport, simply call `add_filter` on `madara::transport::QoSTransportSettings`.  For networking, BufferFilters are the absolute last filter that is called during a send. Consequently, encode happens after any AggregateFilters or individual RecordFilters are performed. Decodes happen in the reverse order. This is important for martialling/demartialling, encoding/decoding, and encryption/decryption. Essentially, other filters aren't possible until you reverse the effects of encoding.
+
+To add AES encryption to save/load checkpoints and save_context, you'll need to use the `madara::knowledge::CheckpointSettings` class. Each [CheckpointSettings](http://madara.sourceforge.net/docs/cpp/d1/d44/structmadara_1_1knowledge_1_1CheckpointSettings.html) contains a data member called buffer_filters. Simply add the BufferFilter to the buffer_filters list with push_back ().
+

Encryption modified by James Edmondson

James Edmondson — Fri, 20 Apr 2018 23:56:21 -0000

--- v4
+++ v5
@@ -1,5 +1,5 @@
 C++ Guide Series
-Architecture | [Knowledge Base](InteractingWithTheKnowledgeBase) | [Networking](InteractingWithTheTransport) | [Containers](KnowledgeContainers) | [Threads](WorkingWithThreads) | [Optimizations](OptimizingKaRL) | [KaRL](KarlLanguage) | Encryption
+[Architecture](MadaraArchitecture) | [Knowledge Base](InteractingWithTheKnowledgeBase) | [Networking](InteractingWithTheTransport) | [Containers](KnowledgeContainers) | [Threads](WorkingWithThreads) | [Optimizations](OptimizingKaRL) | [KaRL](KarlLanguage) | Encryption

 ----
 # Overview

Encryption modified by James Edmondson

James Edmondson — Fri, 20 Apr 2018 23:53:37 -0000

--- v3
+++ v4
@@ -7,10 +7,10 @@
 MADARA allows for encryption/decryption and arbitrary encoding of data per record, per aggregation of records, and per buffers. The most common type of encryption people tend to want is via buffer filters, which will encode or decode an entire buffer of information, usually to/from disk or over the network. In this guide, we'll focus on extending the BufferFilter and using the pre-build AESBufferFilter, which provides 256 bit AES encryption.

 # BufferFilters
-![BufferFilter](https://imgur.com/dOaU59p.png)
+

 BufferFilters are encoders and decoders of character buffers. Individually, these are powerful software constructs that allow you to change how data is represented. You can compress data with BufferFilters, you can encrypt it, or you can remove data entirely so that no one ever sees it. MADARA also allows chaining together BufferFilters so you might compress and then encrypt an image, for instance. In this powerful filter chain concept, MADARA also provides mechanisms to ensure that you reverse the process in the appropriate order (e.g., decrypting and then uncompressing the image from the previous chain, instead of the opposite which wouldn't work at all).

 # AES 256 Bit Encryption
-![AESBufferFilter](https://imgur.com/2rnKNzv.png)
+