PixelRAG Files

Highlights

Smarter page readiness for arbitrary URLs. --wait-network-idle now uses networkidle2 semantics — it counts in-flight requests over CDP Network events (≤2 pending for 500ms, 12s cap) instead of resetting a timer on every resource completion. Analytics-heavy pages that used to ride the 12s cap settle in ~1s (measured: 15.6s → 4.0s on a 200ms-beacon page), and already-quiet pages proceed the moment load fires. The index pipeline enables it by default for the web source only; kiwix/localhost and local-file renders stay on the fast path. (#96, [#119], [#120])

Safe incremental re-runs. Tile directories now record which source document they were rendered from; re-running pixelrag index build after adding/removing/renaming files re-renders shifted positions instead of silently pairing one document's pixels with another's metadata. Manifest writes are atomic, and article_id is stamped into every manifest so the embed stage no longer guesses from directory names. (#83, [#119])

New

• Department-filtered search for local document indexes: department metadata from your directory layout, FAISS-native filtering, and a GET /departments endpoint (#115)
• Local images (PNG/JPG) render into the tile pipeline — no more silent empty index from an image folder (#95, fixes [#67])
• .md/.txt files render natively (tables, fenced code, relative images resolved) (#101, [#107])
• --cdp-url to attach to an already-running browser (#76)
• --device auto/mps accepted by pixelrag index build (#73)
• MiniMax model configs for the eval harness, and registry endpoints now actually apply (#113, [#119])
• Progress bars for the chunk stage that stay readable under multiprocessing, with log-file progress preserved for unattended runs (#108, [#119])

Fixed

• Path-traversal hardening in the eval renderers (#80, fixes [#78]/#79)
• Playwright Chromium picked by numeric revision, not lexicographic order (#74)
• Chrome --user-data-dir isolated per worker (#65, fixes [#54])
• Transparent local images composite onto white instead of black (#119)
• Non-numeric filename-stem IDs handled throughout the local pipeline (#66, [#83])
• Chat agent timeouts / loop prevention, back button and lightbox (#49)
• tqdm/pyyaml/numpy moved from the heavy torch extras into core — light stages (pixelrag chunk, index --help) work on a bare pip install pixelrag (#118)

Install

:::bash
pip install -U pixelrag           # light: pixelshot + umbrella CLI
pip install -U 'pixelrag[all]'    # + embed/index/serve stages