Datus Files

New Features

• OntologyAnalyst Subagent for Ontology Q&A - Uses Apache Ossie ontologies and mappings to understand business concepts, relationship roles, allowed enumeration values, and grouping paths, supporting both metric aggregation and detail-record queries. It uses project-level subject/ontology/ and binds models for different datasources through the mapping's semantic_model_ref.iri. Datasources with only inline-model mappings can also serve standard semantic queries; the modeling Agent explicitly blocks creating a separate model that would overwrite them. (Agent [#1524], #1538, #1539) Docs: OntologyAnalyst.
• AWS Athena Database Adapter - Uses the official boto3 SDK to query Athena and read Glue Catalog metadata. Supports Workgroup-managed query results, pagination, cancellation, execution diagnostics, and list/CSV/pandas/Arrow output. (DB [#154], Agent [#1554]) Docs: Athena Connection Guide; Repository Guide.
• SAP HANA Connectivity and Agent Workflows - Uses the official hdbcli driver with password or user-store authentication, encryption and certificate verification by default, metadata, transactions, and multiple result formats. The Agent uses adapter-provided SQL analysis for read-only and table-scope checks and supports initialization with only a schema configured. Also adds common Dosi HANA scalar functions and precise Decimal result handling. (Agent [#1518], DB [#156], #164, Agent [#1554]) Docs: HANA Connection Guide; Repository Guide.
• Aurora Authentication and Connection Lifecycle Support in the MySQL/PostgreSQL Adapters - Keeps the existing datasource types and supports password, IAM, and Secrets Manager authentication. Refreshes IAM tokens for new physical connections, updates connection pools when the Secret version changes, and checks for Aurora writer endpoint role changes. IAM requires CA and hostname verification; application SQL and transactions are not automatically replayed. (DB [#155], Agent [#1554]) Docs: Aurora Connection Guide; Repository Guide.

Enhancements

• Continue Large-Catalog Initialization and Long Tool Results in One Session - /init uses the standard agentic loop, reads full archived results through pagination as needed, and compacts context within a turn. Capacity checks include complete messages, tool schemas, output reserves, and a safety margin. Private models can optionally configure capacity and a tokenizer, while omitted settings still allow startup. DDL lookup for specified tables restricts metadata queries directly, reducing repeated full-catalog scans. Initialization still reads only metadata and records unconfirmed business interpretations separately. (Agent [#1540], #1546, #1548, DB [#176]) Docs: Init, Capacity Configuration.
• Semantic Modeling Based on Original Business Materials and Real Results - Reads SQL, documents, and engine references through existing file tools. Explicit read-only reference SQL compares complete metric groups and values, rejecting validation evidence from stale or incorrect models. Accepts structured dataset/metric arguments and detects duplicate YAML keys. Database skills retain essential query guidance while moving detailed DDL, writes, loading, and procedure instructions into on-demand references, reducing persistent context. (Agent [#1517], DB [#153]) Docs: Semantic Modeling, Skills.
• Richer Metric Explanations and More Compact AskMetrics Results - Adds get_dataset and exposes expressions, fixed-grain values, relationships, windows, parameters, required grouping, and time requirements. The preview API returns required dimensions and their reasons. Execution results use a text header and CSV rows, preserving warnings, the full cache, and paginated reading; compiled SQL is available through dry run. The bundled Dosi upgrade also adds cross-fact LOD and composite metrics and fixes filtered count, integer division, Decimal, and some time-bucket window calculations. (Agent [#1515], #1533, #1534) Docs: Metrics, AskMetrics.
• Hive/Inceptor Authentication, Metadata, and Result Protocols - Supports the Kerberos service name and GSSAPI installation, correctly distinguishes tables from views, and reads column comments. server_type: inceptor enables separate result decoding for VARCHAR2, TIME, JSON, GEO, interval, and CURSOR types. The shared SQLAlchemy execution layer preserves SQL literals such as colons and percent signs inside JSON. The Agent can route native Dosi HiveServer2 metric execution by profile and forward timeouts and available cancellation tokens; this path requires a native engine build with the corresponding capabilities. (Agent [#1519], #1527, #1531, DB [#158], #163, #167) Docs: Hive/Inceptor.
• ClickHouse, Trino, and PostgreSQL TLS Configuration - ClickHouse explicitly selects HTTPS and supports certificate verification and PEM CAs. Trino supports private CAs and validates combinations of protocol and verification settings. Agent configuration preserves TLS settings, and the PostgreSQL guide adds certificate, IAM/Secrets Manager, and verify-full configuration. (Agent [#1521], #1522, DB [#160], #161) Docs: ClickHouse, Trino, PostgreSQL.
• Snowflake Credentials from AWS Secrets Manager - Supports password and key-pair authentication, JSON field mapping or raw SecretString, and credential caching for up to 300 seconds. New physical logins obtain current credentials; cache expiry does not interrupt existing sessions or transactions. Corrects package versions and the minimum core dependency, and constrains the Snowflake SDK to <5 and PyArrow to <26 to prevent import failures from incompatible combinations. This Secret capability belongs to the Python database adapter; these Agent changes do not imply support for the same credential configuration in native Dosi Snowflake. (Agent [#1525], DB [#162], #178, #151) Docs: Snowflake.
• Redshift IAM Username Handling and Local Connection Guidance - Passes the configured username as the driver's required db_user. Documents IAM TLS requirements, SSM connections that preserve the original hostname, and the separate scope of driver SSL warnings. (Agent [#1520], DB [#157]) Docs: Redshift.
• Preserve Dosi Profile Types and Execution Diagnostics - Shared configuration conversion preserves native boolean, integer, and nested AWS option types and provides adapter namespace guidance to the Agent. Metric and attribution tools retain optional engine execution_info, including query IDs, terminal state, scan statistics, and cancellation diagnostics. This passthrough remains compatible with backends that provide no diagnostics. Native Athena/Aurora execution still depends on the installed Dosi build and cannot be inferred from configuration passthrough alone. (Agent [#1523], #1526, #1528)
• Bounded MaxCompute Partition Sampling - Prefers the latest populated partition from a bounded partition list, avoiding unpartitioned queries when full-table scans are disabled. Uses only size information already returned by the partition list, avoiding large numbers of extra metadata requests caused by PyODPS lazy loading. Adds ignore_table_patterns to exclude matching objects before fetching DDL. (DB [#146], #159)

Bug Fixes

• Bound Database Waits and Show SQL Errors Promptly - Multiple database adapters propagate timeouts to the actual driver, network I/O, or server execution boundary and invalidate connections after a timeout. Also corrects timeout classification for Doris server-side cancellation. The Agent displays completed SQL errors without waiting for all parallel sibling tools, and approved execute_sql calls have a fixed 300-second wait limit. This Agent limit bounds tool/UI waiting; a synchronous driver or remote SQL may still be running, so it does not establish that retrying is safe. (Agent [#1541], DB [#166], #170) Docs: Repository Timeout Guide (updated by [#170] in this group; the corresponding online page is not yet published). User documentation for the Agent's 300-second limit is missing.
• Synchronous MCP Tools No Longer Block the Event Loop - Moves synchronous tools and conversion work to a bounded executor while preserving request context. Health checks and other requests can continue during slow queries. Queue overload is rejected immediately; caller cancellation continues to account for capacity occupied by the real task, and connection cleanup waits for users of the connection to finish. Defaults to 8 workers and 16 queued tasks, both configurable. (Agent [#1530]) Docs: MCP.