DocWire SDK Files

This release introduces a major overhaul of the local AI subsystem, adding support for Llama.cpp and the IBM Granite model, alongside extensive naming standardization and build system improvements. A new abstract AI runner interface unifies inference backends, while rigorous snake_case normalization aligns the entire codebase with the project's coding guidelines.

The llama hums, a granite sound,
With abstract runners all around.
The names in snakes now coil and sleep,
While docs and tests more vigil keep.
A modular heart, a cleaner choice,
DocWire finds its stronger voice.
🦙🧠🔧📚

• Features
• Llama.cpp Integration: Added an optional Llama.cpp backend (docwire_ai_llama) for running GGUF-based models. A new llama_runner class implements the abstract ai_runner interface, providing process() and embed() methods with thread-safe backend management.
• IBM Granite Model Support: The new local-ai-model-granite feature installs the IBM Granite 4.0 1B Q8_0 GGUF model, making a compact yet capable LLM available out of the box.
• Abstract AI Runner Interface: Introduced docwire::ai::ai_runner – a pure virtual base that defines process(), embed(), and unload() for all local AI backends. Backends guarantee thread safety and deterministic teardown.
• Modular AI Backends: Split the local AI runtime into separate optional libraries: docwire_ai_ct2 (CTranslate2) and docwire_ai_llama (Llama.cpp). A common docwire_ai library now contains shared components such as ai::task, ai::summarize, ai::translate, and ai::embed.
• High-Level Local AI API: New convenience classes in the ai::local namespace (summarize, translate, task, passage::embedder, query::embedder) automatically select a default backend model, simplifying the most common use cases.

• Documentation: Added the DocWire SDK Manifesto and Coding Guidelines to the repository documentation (doc/manifesto.md, doc/coding_guidelines.md), outlining the project’s philosophy and architectural rules.

• Improvements

• Naming Normalization: All class/struct/enum names have been converted to snake_case according to the new coding guidelines. Publicly visible types such as ChainElement, PlainTextExporter, HtmlWriter, ParsingChain, TransformerFunc, FileStream, ZipReader, etc., are now chain_element, plain_text_exporter, html_writer, parsing_chain, transformer_func, file_stream, zip_reader, and so on. Document element tags (document::Text → document::text, mail::Mail → mail::mail, …) have been similarly renamed.
• Enum Names Standardized: Enumerations such as Model, ImageDetail, Language, output_type etc., inside various namespaces now follow snake_case (e.g., openai::model, openai::image_detail, etc.).
• Upgraded libbfio to version 20260623 for improved stability and compatibility.
• ASan Fix for Recent MSVC: Added _DISABLE_OPTIONAL_ANNOTATION to prevent link errors when building with AddressSanitizer on MSVC.
• CTranslate2 Build Fix: The custom spdlog submodule has been replaced by the standard vcpkg spdlog port, resolving spdlog‑related build errors.
• Vcpkg Conflict Resolution: Fixed a file installation conflict between the ctranslate2 and nlohmann-json ports.

• Mailio Crash Prevention: The EML parser’s BoundaryTracker now handles prematurely closed multipart sections and injects empty parts when necessary, preventing a crash in the underlying mailio library.

• Refactor

• Backend Separation: The former monolithic local_ai library has been split into docwire_ai_ct2 (CTranslate2 runner, tokenizer) and docwire_ai_llama (Llama.cpp runner). The docwire_local_ai library becomes a convenience layer that links the chosen backends.
• Namespace & Module Reorganization:
  • CTranslate2‑specific classes moved into docwire::ai::ct2 (ct2_runner, tokenizer).
  • Llama.cpp‑specific code lives in docwire::ai::llama.
  • Shared AI chain elements reside directly in docwire::ai.
  • Local default wrappers moved to docwire::ai::local.
• Configuration Strong Types: Introduced strong_type and several dedicated types (context_size, thread_count, token_limit, temperature, min_p, batch_size) in model_inference_config.h for safer Llama parameter passing.

• Build System Restructuring: New CMake options DOCWIRE_CT2 and DOCWIRE_LLAMA control optional compilation of the respective backends. Feature flags local-ai-ct2, local-ai-llama, and local-ai-model-granite are exposed in the vcpkg port.

• Tests

• Backend Integration Tests: Added local_ai_ct2_integration and local_ai_llama_integration tests that execute a full pipeline (document parsing → text extraction → local AI task) to validate the new backends.
• More Forgiving AI Tests: Local AI summarization tests now use fuzzy matching instead of exact string comparison, reducing flakiness due to model output variation.
• EML Robustness Tests: New test cases (endboundary_first.eml, missing_inner_closing_boundary.eml, nested_multiparts_missing.eml, valid_format.eml) ensure correct handling of malformed or non‑standard MIME boundaries.
• Test Infrastructure Fixes: Added a GoogleTest PrintTo stringification for the ContentTypeTestCase struct, resolving formatting issues in parameterized content‑type tests.
• Build Guard: Example tests that require local AI libraries are conditionally compiled only when the corresponding targets are available.