Recent changes to Keymaster Security Design

Keymaster Security Design modified by James Harris

James Harris — Thu, 23 Apr 2026 10:21:39 -0000

--- v5
+++ v6
@@ -1,7 +1,7 @@
 # Keymaster Security Design

 **Component:** Keymaster (Security Gatekeeper)  
-**Status:** ✅ Early Implementation (Revision 14)  
+**Status:** ✅ In Active Implementation (Revision 14+)  
 **Last Updated:** April 23, 2026  
 **Author:** jstevh

@@ -18,53 +18,54 @@
 External World                  Keymaster (Security Core)                  Sandbox
 ─────────────────────           ────────────────────────────           ─────────────────────
 User drops file                 → Keymaster (Gatekeeper)
-(e.g. PDF)                      → File Type Validator
-(magic bytes + content scan)           → Sandbox Root
-→ Metadata Engine                      /sandbox/document-types/...
-(extract & enforce)                    (organized by metadata)
-→ Folder Organizer
-→ Push Guard (whitelist)
-Only known types (PDF, etc.)
+(e.g. PDF)                      → FileValidator + DocumentType
+(magic bytes check)                    → Sandbox Root
+→ Metadata-driven folder               /sandbox/document-types/...
+→ MObject registration                 (organized by type + date + uuid)
+→ PushGuard (planned)
+Only known DocumentType
 ↓ (optional)
 Remote Node
 text## Key Security Components

-- **Keymaster** – Main gatekeeper with `validateAndIngest(File)` and `pushToNode()`.
-- **FileValidator** – Performs strong type checking using magic bytes (PDF `%PDF-` signature).
-- **MetadataEngine / FolderOrganizer** – Creates metadata-driven paths like `/sandbox/document-types/2026-04-23-uuid123/`.
-- **PushGuard** – (Planned) Enforces whitelist for outgoing files.
+- **Keymaster** – Main gatekeeper with `validateAndIngest(File)`
+- **DocumentType** – Enum defining official supported document types (PDF, DOCX, TXT, ...)
+- **FileValidator** – Performs strong magic-byte validation and returns `DocumentType`
+- **FolderOrganizer** – Creates clean metadata-driven paths
+- **PushGuard** – (Planned) Whitelist enforcement before pushing to nodes

-## Secure File Flow Example (PDF)
+## Secure File Flow (Current Implementation)

-1. User hands a file to Keymaster.
-2. **FileValidator** checks it (magic bytes + content scan) — rejects disguised/malicious files.
-3. File is organized into metadata-driven folder and stored **only** inside the Sandbox.
-4. MObject is created and registered with `sandboxPath`, `fileType`, `sizeBytes`, etc.
-5. When pushing to a node: **PushGuard** will check the type against the whitelist.
+1. User selects file via `addFileToSandboxInteractively()`
+2. **FileValidator.validateDocument()** checks magic bytes and returns `DocumentType`
+3. If valid → create metadata-driven folder (`/sandbox/document-types/YYYY-MM-DD-uuid/pdf/`)
+4. File is copied safely into the Sandbox
+5. MObject is created and registered with rich metadata (`documentType`, `mimeType`, `sandboxPath`, etc.)

 ## Security Guarantees

-| Feature                    | How It Is Implemented                        | Benefit |
-|---------------------------|----------------------------------------------|---------|
-| Sandbox-Only              | All files forced into isolated Sandbox      | Strong containment |
-| Strong Type Validation    | Magic bytes + full content analysis         | Prevents disguised malware |
-| Metadata Folder Structure | Auto-generated paths (`/sandbox/document-types/...`) | Better organization & auditability |
-| Push Whitelist            | Only known `DocumentType` (PDF first)       | Defense-in-depth |
+| Feature                    | Status          | How It Is Implemented                          | Benefit |
+|---------------------------|-----------------|------------------------------------------------|---------|
+| Sandbox-Only              | ✅ Done        | All files forced into isolated Sandbox        | Strong containment |
+| Strong Type Validation    | ✅ Done        | Magic bytes via `FileValidator` + `DocumentType` | Prevents disguised malware |
+| Metadata Folder Structure | ✅ Done        | Auto-generated paths with date + uuid         | Better organization & auditability |
+| Known Document Types      | ✅ Done        | `DocumentType` enum (PDF first)               | Foundation for whitelist |
+| Push Whitelist            | ☐ Planned     | `PushGuard` class (next)                      | Defense-in-depth |

-## Progress & Next Steps (as of Rev 14)
+## Progress & Next Steps (as of April 23, 2026)

-- ✅ **Implemented** `Keymaster.java` + `FileValidator.java`
-- ✅ Strong PDF validation using magic bytes (content-based, not just extension)
-- ✅ Metadata-driven folder organization in Sandbox
-- ✅ `addFileToSandboxInteractively()` now uses the secure `validateAndIngest()` flow
-- ☐ Add initial `DocumentType` enum
+- ✅ `Keymaster.java` updated with secure `validateAndIngest()`
+- ✅ `FileValidator.java` with magic byte checks
+- ✅ `DocumentType` enum added (PDF, DOCX, TXT, UNKNOWN)
+- ✅ Files now stored in well-organized Sandbox folders

 - ☐ Implement `PushGuard` for known-type whitelist before pushing to nodes
 - ☐ Create [[Sandbox Design]] wiki page
+- ☐ Expand validation to more document types

 **Related Pages:**  
 [[Home]]  
-[[Sandbox Design]] (to be created next)
+[[Sandbox Design]] (planned next)

 ---

-*This page is part of the Metadata System design documentation. Committed in Revision 14.*
+*This page is part of the Metadata System design documentation. Updated after Revision 14+ wi

Keymaster Security Design modified by James Harris

James Harris — Thu, 23 Apr 2026 10:08:31 -0000

--- v4
+++ v5
@@ -1,7 +1,7 @@
 # Keymaster Security Design

 **Component:** Keymaster (Security Gatekeeper)  
-**Status:** In Design / Early Implementation  
+**Status:** ✅ Early Implementation (Revision 14)  
 **Last Updated:** April 23, 2026  
 **Author:** jstevh

@@ -11,8 +11,8 @@


 - **Sandbox-Only Storage**: Every file is immediately placed into an isolated Sandbox. No files ever live on the host outside Keymaster control.
 - **Content-Based Validation**: Validation uses magic bytes + content scanning (not just file extensions) to detect disguised or malicious files.
-- **Metadata-Driven Organization**: Files are automatically placed into folders based on extracted metadata (latest design focus).
-- **Known-Type Whitelist for Push**: Only pre-approved document types can be pushed to nodes (starting with document types like PDF).
+- **Metadata-Driven Organization**: Files are automatically placed into folders based on extracted metadata.
+- **Known-Type Whitelist for Push**: Only pre-approved document types can be pushed to nodes (starting with PDF).

 ## High-Level Architecture
 External World                  Keymaster (Security Core)                  Sandbox
@@ -29,20 +29,18 @@
 Remote Node
 text## Key Security Components

-- **Keymaster** – Main gatekeeper class. Handles `validateAndIngest()` and `pushToNode()`.
-- **FileValidator** – Performs strong type checking using magic bytes and content analysis.
-- **MetadataEngine** – Extracts metadata and generates folder paths.
-- **FolderOrganizer** – Places files into metadata-driven directories inside the Sandbox.
-- **PushGuard** – Enforces whitelist: only known document types (PDF first) can be pushed outward.
+- **Keymaster** – Main gatekeeper with `validateAndIngest(File)` and `pushToNode()`.
+- **FileValidator** – Performs strong type checking using magic bytes (PDF `%PDF-` signature).
+- **MetadataEngine / FolderOrganizer** – Creates metadata-driven paths like `/sandbox/document-types/2026-04-23-uuid123/`.
+- **PushGuard** – (Planned) Enforces whitelist for outgoing files.

 ## Secure File Flow Example (PDF)


 1. User hands a file to Keymaster.
 2. **FileValidator** checks it (magic bytes + content scan) — rejects disguised/malicious files.
-3. **MetadataEngine** extracts metadata (type, title, tags, date, etc.).
-4. **FolderOrganizer** creates path like: `/sandbox/document-types/2026-04-23-uuid123/`
-5. File is stored **only** inside the Sandbox.
-6. When pushing to a node: **PushGuard** checks the type against the whitelist. Only approved document types are allowed.
+3. File is organized into metadata-driven folder and stored **only** inside the Sandbox.
+4. MObject is created and registered with `sandboxPath`, `fileType`, `sizeBytes`, etc.
+5. When pushing to a node: **PushGuard** will check the type against the whitelist.

 ## Security Guarantees

@@ -50,19 +48,23 @@
 |---------------------------|----------------------------------------------|---------|
 | Sandbox-Only              | All files forced into isolated Sandbox      | Strong containment |
 | Strong Type Validation    | Magic bytes + full content analysis         | Prevents disguised malware |
-| Metadata Folder Structure | Auto-generated paths from extracted metadata| Better organization & auditability |
+| Metadata Folder Structure | Auto-generated paths (`/sandbox/document-types/...`) | Better organization & auditability |
 | Push Whitelist            | Only known `DocumentType` (PDF first)       | Defense-in-depth |

-## Next Steps
+## Progress & Next Steps (as of Rev 14)

-- Implement `Keymaster.java` and `FileValidator.java` in the Sandbox area.
-- Add initial `DocumentType` enum (starting with PDF).
-- Integrate with the existing PDF validation code you already have.
+- ✅ **Implemented** `Keymaster.java` + `FileValidator.java`
+- ✅ Strong PDF validation using magic bytes (content-based, not just extension)
+- ✅ Metadata-driven folder organization in Sandbox
+- ✅ `addFileToSandboxInteractively()` now uses the secure `validateAndIngest()` flow
+- ☐ Add initial `DocumentType` enum
+- ☐ Implement `PushGuard` for known-type whitelist before pushing to nodes
+- ☐ Create [[Sandbox Design]] wiki page

 **Related Pages:**  
 [[Home]]  
-[[Sandbox Design]] (create this next)
+[[Sandbox Design]] (to be created next)

 ---

-*This page is part of the Metadata System design documentation.*
+*This page is part of the Metadata System design documentation. Committed in Revision 14.*

Keymaster Security Design modified by James Harris

James Harris — Thu, 23 Apr 2026 09:43:24 -0000

--- v3
+++ v4
@@ -2,7 +2,7 @@

 **Component:** Keymaster (Security Gatekeeper)  
 **Status:** In Design / Early Implementation  
-**Last Updated:** April 2026  
+**Last Updated:** April 23, 2026  
 **Author:** jstevh

 Keymaster is the central security layer of the Metadata System. All files enter and leave exclusively through Keymaster. No direct filesystem access is allowed outside the Sandbox.
@@ -40,8 +40,7 @@

 1. User hands a file to Keymaster.
 2. **FileValidator** checks it (magic bytes + content scan) — rejects disguised/malicious files.
 3. **MetadataEngine** extracts metadata (type, title, tags, date, etc.).
-4. **FolderOrganizer** creates path like:  
-   `/sandbox/document-types/2026-04-23-uuid123/`
+4. **FolderOrganizer** creates path like: `/sandbox/document-types/2026-04-23-uuid123/`
 5. File is stored **only** inside the Sandbox.
 6. When pushing to a node: **PushGuard** checks the type against the whitelist. Only approved document types are allowed.

Kwymaster Security Design modified by James Harris

James Harris — Thu, 23 Apr 2026 09:39:40 -0000

--- v2
+++ v3
@@ -15,3 +15,55 @@

 - **Known-Type Whitelist for Push**: Only pre-approved document types can be pushed to nodes (starting with document types like PDF).

 ## High-Level Architecture
+External World                  Keymaster (Security Core)                  Sandbox
+─────────────────────           ────────────────────────────           ─────────────────────
+User drops file                 → Keymaster (Gatekeeper)
+(e.g. PDF)                      → File Type Validator
+(magic bytes + content scan)           → Sandbox Root
+→ Metadata Engine                      /sandbox/document-types/...
+(extract & enforce)                    (organized by metadata)
+→ Folder Organizer
+→ Push Guard (whitelist)
+Only known types (PDF, etc.)
+↓ (optional)
+Remote Node
+text## Key Security Components
+
+- **Keymaster** – Main gatekeeper class. Handles `validateAndIngest()` and `pushToNode()`.
+- **FileValidator** – Performs strong type checking using magic bytes and content analysis.
+- **MetadataEngine** – Extracts metadata and generates folder paths.
+- **FolderOrganizer** – Places files into metadata-driven directories inside the Sandbox.
+- **PushGuard** – Enforces whitelist: only known document types (PDF first) can be pushed outward.
+
+## Secure File Flow Example (PDF)
+
+1. User hands a file to Keymaster.
+2. **FileValidator** checks it (magic bytes + content scan) — rejects disguised/malicious files.
+3. **MetadataEngine** extracts metadata (type, title, tags, date, etc.).
+4. **FolderOrganizer** creates path like:  

+   `/sandbox/document-types/2026-04-23-uuid123/`
+5. File is stored **only** inside the Sandbox.
+6. When pushing to a node: **PushGuard** checks the type against the whitelist. Only approved document types are allowed.
+
+## Security Guarantees
+
+| Feature                    | How It Is Implemented                        | Benefit |
+|---------------------------|----------------------------------------------|---------|
+| Sandbox-Only              | All files forced into isolated Sandbox      | Strong containment |
+| Strong Type Validation    | Magic bytes + full content analysis         | Prevents disguised malware |
+| Metadata Folder Structure | Auto-generated paths from extracted metadata| Better organization & auditability |
+| Push Whitelist            | Only known `DocumentType` (PDF first)       | Defense-in-depth |
+
+## Next Steps
+
+- Implement `Keymaster.java` and `FileValidator.java` in the Sandbox area.
+- Add initial `DocumentType` enum (starting with PDF).
+- Integrate with the existing PDF validation code you already have.
+
+**Related Pages:**  
+[[Home]]  
+[[Sandbox Design]] (create this next)
+
+---
+
+*This page is part of the Metadata System design documentation.*

Kwymaster Security Design modified by James Harris

James Harris — Thu, 23 Apr 2026 09:36:51 -0000

--- v1
+++ v2
@@ -15,32 +15,3 @@

 - **Known-Type Whitelist for Push**: Only pre-approved document types can be pushed to nodes (starting with document types like PDF).

 ## High-Level Architecture
-
-```mermaid
-graph TD

-    subgraph "Keymaster (Security Core)"
-        KM[Keymaster
Security Gatekeeper]
-        Validator[File Type Validator
(magic bytes + content scan)]
-        MetadataEngine[Metadata Engine
(extract & enforce)]
-        FolderOrganizer[Folder Organizer
(metadata-based paths)]
-        PushGuard[Push Guard
(known-type whitelist)]
-    end
-
-    subgraph "Sandbox (Isolated Storage)"
-        SB[Sandbox Root
/sandbox/document-types/...]
-    end
-
-    subgraph "External"
-        User[User drops file
(e.g. PDF)]
-        Node[Remote Node]
-    end
-
-    User --> KM
-    KM --> Validator
-    Validator --> MetadataEngine
-    MetadataEngine --> FolderOrganizer
-    FolderOrganizer --> SB
-    SB --> PushGuard
-    PushGuard --> Node
-
-    style KM fill:#1e3a8a,stroke:#60a5fa

Kwymaster Security Design modified by James Harris

James Harris — Thu, 23 Apr 2026 09:34:03 -0000

Keymaster Security Design

Component: Keymaster (Security Gatekeeper)
Status: In Design / Early Implementation
Last Updated: April 2026
Author: jstevh

Keymaster is the central security layer of the Metadata System. All files enter and leave exclusively through Keymaster. No direct filesystem access is allowed outside the Sandbox.

Core Security Principles

Sandbox-Only Storage: Every file is immediately placed into an isolated Sandbox. No files ever live on the host outside Keymaster control.
Content-Based Validation: Validation uses magic bytes + content scanning (not just file extensions) to detect disguised or malicious files.
Metadata-Driven Organization: Files are automatically placed into folders based on extracted metadata (latest design focus).
Known-Type Whitelist for Push: Only pre-approved document types can be pushed to nodes (starting with document types like PDF).

High-Level Architecture

```mermaid
graph TD
subgraph "Keymaster (Security Core)"
KM[Keymaster
Security Gatekeeper]
Validator[File Type Validator
(magic bytes + content scan)]
MetadataEngine[Metadata Engine
(extract & enforce)]
FolderOrganizer[Folder Organizer
(metadata-based paths)]
PushGuard[Push Guard
(known-type whitelist)]
end

subgraph "Sandbox (Isolated Storage)"
    SB[Sandbox Root<br/>/sandbox/document-types/...]
end

subgraph "External"
    User[User drops file<br/>(e.g. PDF)]
    Node[Remote Node]
end

User --> KM
KM --> Validator
Validator --> MetadataEngine
MetadataEngine --> FolderOrganizer
FolderOrganizer --> SB
SB --> PushGuard
PushGuard --> Node

style KM fill:#1e3a8a,stroke:#60a5fa