RAG Anything Files

What's new

Features

• Native MinerU content list v2 (#347, closes [#224]): MinerU 3.x content_list_v2.json is normalized into the existing content-list contract — titles, paragraphs, lists, equations, images, tables, charts, code and algorithms, with page_idx/bbox/heading levels/captions preserved. Layout furniture (page_header, page_footer, page_number, …) is filtered by default; pass include_layout_blocks=True to keep it. Legacy content lists work unchanged, with an automatic fallback chain.
• Direct .txt/.md parsing (#333, closes [#331]): plain text and markdown no longer take a ReportLab→PDF→OCR round trip (~3× faster measured; the whole family of PDF-rendering fidelity bugs #24/#226/#316 becomes structurally impossible on this path). Markdown ATX headings carry text_level; encoding falls back UTF-8 → gbk → latin-1 → cp1252.
• force_multimodal_reprocess flag (#318, closes [#154]): opt-in parameter on process_document_complete() / insert_content_list() to re-run image/table/equation processing for an already-processed document — e.g. after switching graph/vector storage backends. Deterministic chunk ids are deduplicated on re-runs.
• Vision prompts handle diagrams (#319, closes [#313]): the default prompts now transcribe text labels verbatim and enumerate nodes/edges/topology for schematics instead of describing "colors and lighting".
• --no-recursive for the batch CLI (#327).

Fixes

• Documents inserted through process_document_complete_lightrag_api() were silently skipped by LightRAG's dedup and indexed zero chunks (#334, fixes [#328]).
• Multimodal chunks_list bookkeeping: merges are idempotent and a missing doc_status record is surfaced (#335, refs [#332]).
• Parse cache now fingerprints file content (SHA-256), so in-place replacements with preserved mtime can no longer serve stale parses (#325).
• Multimodal query cache keys include full media identity and all query options, eliminating cross-file collisions (#324).
• Batch inputs with the same filename no longer share an output directory (#346) — batch outputs now live in <stem>_<hash>/.
• LibreOffice conversion timeout is configurable via LIBREOFFICE_CONVERT_TIMEOUT (default 600 s) and timeouts are reported as timeouts, not as a missing installation (#329).
• LA (grayscale+alpha) images are composited onto white correctly before MinerU (#317).
• Markdown with bare angle brackets (score <threshold) no longer crashes convert_text_to_pdf (#316).
• Retry delays are capped at max_delay after jitter (#323).
• insert_content_list([]) now raises ValueError instead of creating an empty document record (#326).
• MinerU/pdftext PageChars crashes are self-diagnosing (#321) and prevented for fresh installs by requiring mineru[core]>=3.4.1 (#337).
• LM Studio example tolerates servers without a working /models endpoint (#349).

Migration notes (one-time effects on upgrade)

1. .txt/.md re-ingest as new documents. The content list for text files changes shape, so content-derived doc ids change. Previously ingested text documents will be treated as new on re-processing; remove the old entries if duplicates matter to you.
2. Parse caches invalidate once. [#325] (content fingerprint) and [#347] (new cache-relevant options) change the parse-cache key; every document re-parses once after upgrading.
3. insert_content_list([]) now raises ValueError instead of silently succeeding.
4. Batch output directories move from <output_dir>/<stem>/ to <output_dir>/<stem>_<pathhash>/. Incremental manifests keep skipping already-processed files; delete the manifest if you want a directory-layout rebuild.
5. MinerU lower bound: mineru[core]>=3.4.1 (pairs with pdftext ≥0.7; older MinerU crashes on Office documents).

Full history: https://github.com/HKUDS/RAG-Anything/compare/v1.3.1...v1.4.0