OpenSERP Files

Reliability release. The main theme is honest error reporting: engines now tell you why a search returned nothing instead of handing back an empty list. Plus batch URL extraction and a rebuilt browser fingerprinting layer.

Highlights

Engines no longer silently return empty results. Captcha pages, soft blocks and genuine "no results" pages are now classified separately and surfaced as captcha_detected, blocked or empty_result error codes. The detection is shared between the browser and raw HTTP paths, so /<engine>/search and /<engine>/parse behave the same across Google, Bing, DuckDuckGo, Yandex, Baidu and Ecosia. Google's /sorry/ interstitial is detected explicitly. This addresses the confusing empty responses reported in [#36] and [#33].

POST /extract/batch - extract content from up to 20 URLs in one call, returning the {page_content, metadata} array shape Open WebUI's external web loader expects. A URL that fails validation or extraction gets metadata.error and empty content instead of failing the whole batch; each item carries its own took_ms. Contributed by @AIWintermuteAI in [#39].

DuckDuckGo pagination fixed. Non-zero offsets returned wrong results because DDG's pagination doesn't work off URL params - the engine now drives the "More results" button. Fixes [#40], contributed by @subediparas5 in [#41].

Browser fingerprinting moved to native CDP. Dropped the injected patch.js and localized profile files in favor of CDP-level overrides, which is both faster and less detectable. Docker image now ships an updated Chrome.

Other changes

• Proxy sessions are scoped to credentials, so rotating proxies no longer leak sessions between accounts
• err_tunnel_connection_failed is now treated as a retryable proxy network error
• New block_resources and block_trackers config options to skip heavy subresources in browser mode
• X-Proxy-Attempts and X-Use-Profile added to the default CORS allow-list
• docker-compose.yaml now mounts config.yaml by default, with proxy config examples
• Fixed a YAML parse error in the OpenAPI spec; /extract/batch documented and exempted from the default request timeout

Upgrade notes

• proxies.allow_request_proxy_url now defaults to true. If you expose OpenSERP publicly and don't want callers supplying their own proxy URL, set it back to false.
• Searches that previously returned 200 with an empty array may now return an error with captcha_detected / blocked. Handle those codes if your client treats empty as success.

Contributors

Thanks to @AIWintermuteAI and @subediparas5 - both first-time contributors.

Full Changelog: https://github.com/karust/openserp/compare/v0.8.6...v0.8.12