fess Files

We're pleased to announce the release of Fess 15.9.0.

This release makes the static theme the default search UI and removes the JSP search UI, adds search analytics, document reports and CSV export, and brings engine-side rank fusion and RAG chat improvements. Fess is also slimmer and more modular: the embedded OpenSearch, the bundled Playwright runtime, the Groovy engine and the storage and SSO integrations are no longer part of the core distribution. The search engine baseline moves to OpenSearch 3.9.

Important — this release contains breaking changes. Read the list below before upgrading an existing installation.

• JSP search UI removed (#3497, [#3460], [#3457]). The static theme is the default; JSP themes and the JSP theme plugin type no longer exist, and the admin page design editor is gone.
• OpenSearch is no longer embedded (#3408). Install a separate OpenSearch server with bin/fess-setup install opensearch.
• JavaScript replaces Groovy as the default script engine (#3344, [#3345]). An unset script type is still treated as Groovy, but the Groovy plugin is no longer bundled (#3425) — install it with bin/fess-setup install plugin fess-script-groovy if your crawl configurations, boost rules or path mappings use it.
• S3 and GCS storage clients and the four SSO authenticators moved to plugins — fess-storage-s3, fess-storage-gcs, fess-sso-saml, fess-sso-spnego, fess-sso-entraid and fess-sso-oidc (#3424, [#3426], [#3428], [#3430]). The storage: crawl protocol is gone; use s3: (#3332).
• api.access.token.required removed (#3496).
• Node.js is installed on demand instead of bundling Playwright's copies, and the Playwright crawler ships as a plugin (#3409, [#3437]).
• Jackson 3 replaces Jackson 2 (#3352).
• Japanese kana variants and dashes written as long vowel marks are folded further (#3532). If you index Japanese content, reindex so that documents already indexed use the new normalization.

Highlights

• Static Theme as the Default UI The Bootstrap static theme is now the default and reaches parity with the former JSP search pages, including server-side support for the SPA, facet selections kept in the URL, accessibility fixes and translations in all supported languages. Errors render in place. Themes can be installed from the admin theme page by choosing a theme and version from dropdowns, from a Maven repository, or from the command line with fess-setup (#3458, [#3459], [#3460], [#3461], [#3462], [#3497], [#3603]).

• Search Log Analytics Dashboard The search log screen becomes an analytics dashboard with drill-down into zero-hit searches, usage by role and group, RAG chat usage, and CSV download of logs and reports. Related queries can be generated from search log refinements (#3521, [#3526], [#3529], [#3534], [#3535]).

• Document Report A new admin report lists duplicate and dormant documents (#3531).

• Search Results Export and Search History Users can export search results as CSV or JSON, and logged-in users can re-run their recent searches (#3519, [#3516]).

• User Tags Users can tag documents with labels of the kind tag. Tags are kept per user in a dedicated tag_type index and are limited to what the caller can see and to the virtual host of the request (#3536, [#3551], [#3540], [#3652]).

• Engine-Side Rank Fusion The search engine can fuse the searchers into one request, with a configurable pagination depth, bounded waits for searchers that do not answer, and safe fallbacks to Fess-side fusion. Highlighting and related-query handling are kept intact (#3401, [#3453], [#3484], [#3485], [#3506], [#3483], [#3502], [#3503]).

• RAG Chat Chat can answer questions about a single document, restrict itself to permitted roles and labels, respond in a configurable language, and refine its search query in a bounded loop. Usage is recorded for analytics (#3515, [#3533], [#3527], [#3530], [#3534]).

• OCR for Images and Scanned PDFs New settings enable OCR for images and scanned PDFs during crawling (#3520).

• File Owner Search The file owner and last modifier are indexed and searchable (#3528).

• Command-Line Plugin and Theme Management bin/fess-setup installs and removes plugins and themes, checks and upgrades plugins, pins a version per plugin, installs OpenSearch, and installs from the snapshot repository and GitHub releases. It ships in the rpm and deb as well as the ZIP (#3413, [#3417], [#3429], [#3431], [#3432], [#3447], [#3461]).

• JavaScript Script Engine Fess ships a JavaScript engine backed by Sai and uses it for the default jobs. Each document boost rule can choose its script engine, and DI XML expressions are evaluated as ECMAScript 6 (#3344, [#3345], [#3346], [#3347], [#3339]).

• OpenSearch 3.9 Support Fess is based on OpenSearch 3.9.0 and reports a backend older than OpenSearch 3 at startup. Faiss is now usable for vector search because the official OpenSearch distribution is used (#3537, [#3341], [#3408]).

Improvements

Search & Query

• Honor the slop of a proximity search such as "word1 word2"~N (#3514)
• Add query markers and let a rewriter redirect a search (#3482)
• Add the vanilla search engine type for an OpenSearch without the CodeLibs plugins (#3564)
• Split a narrowed query for semantic search instead of refusing it, and carry the similarity cutoff in the query (#3403, [#3404])
• Answer a search of a closed index, or one the engine has no capacity for, as 503 instead of an invalid query (#3640, [#3567])
• Tell a query timeout apart from a failed shard, and stop reporting an unreachable engine as zero results (#3440, [#3374])
• Refuse queries that cannot run: a regular expression that does not compile, a search that asks not to track the total, an invalid num or start, an invalid facet parameter (#3594, [#3373], [#3563], [#3556])
• Treat the question mark that ends a multi-word question as punctuation, and return no featured answers for a search without a query (#3571, [#3570])
• Group the role filter into terms queries (#3617)
• Keep highlighting when the search engine fuses the results, and answer a page past the last fused hit instead of failing (#3483, [#3505])
• Embed the query once per request and size the embedding connection pool for rank fusion (#3486, [#3548], [#3547])
• Keep the search page's selected facets when search options change (#3491)

Crawling & Indexing

• Honor robots.txt Crawl-delay and backoff, and use conditional GET (#3513)
• Let plugins register classes for the crawler data serializer (#3512)
• Use the Markdown front matter title as the document title (#3511)
• Record HTTP 401 responses as failure URLs by default (#3651)
• Keep nofollow in effect when a page is re-crawled unchanged (#3558)
• Pass http.nonProxyHosts to the crawler process and use the jcifs 3.x property names in the crawler JVM options (#3495, [#3433])
• Resolve crawl.order as a UrlQueueOrder component name (#3353)
• Fail the crawl when the web/file or data store process throws, and record a failure when the last batch of a data crawl cannot be committed (#3498, [#3624])
• Report documents indexed without content, every document a bulk request rejected, and crawl paths and storage failures that used to pass in silence (#3397, [#3616], [#3427])
• Give a document the language of its own text, and fold kana before language detection so Japanese is not indexed as Chinese (#3394, [#3449], [#3450])
• Warn when the crawl config fetch cap is reached, and name the crawl configuration when a stored parameter cannot be decrypted (#3638, [#3629])

Authentication & Security

• Mask secrets in the diagnostic download, System Info and the log: storage.secretkey, index.user.initial_password, sensitive JVM and -D arguments, and credentials at DEBUG/INFO level (#3636, [#3615], [#3557], [#3545], [#3565], [#3553])
• Keep authentication passwords and passphrase parameters encrypted in the configuration indices, and stop returning stored passwords from the file and web authentication admin APIs (#3641, [#3649], [#3619], [#3621])
• Apply the password policy to the admin API, and stop one user from raising their own favorite_count (#3358, [#3391])
• Attribute rate-limited requests to a hop the caller cannot choose, and match the whitelist and block list by address (#3360, [#3508])
• Accept a valid access token on /api/v2 search when login.required is on, and close the chat screens when login.required is set (#3507, [#3359])
• Keep per-address login guesses counted after a successful login, and keep user-supplied values from starting a line of the application or audit log (#3648, [#3552], [#3573], [#3477])
• Record admin REST API requests in the audit log (#3489)
• Refuse every file below a dot directory under /themes/, close the theme file denylist bypass, and make frame-ancestors of theme pages configurable (#3576, [#3523], [#3522])
• Escape the document link in the admin search list and link only the schemes Fess crawls (#3583)
• Drop the invalid reflected-xss Content-Security-Policy header from the defaults (#3559)
• Name an Entra ID user by its directory object id, recover a session whose access token expired, and bound the token cache and the parent group walk (#3356, [#3255], [#3268], [#3261], [#3259])
• Restore the opened search after an SSO login, including from the static theme (#3480, [#3476])

Administration & Configuration

• Make the admin console usable by keyboard and assistive technology: names for icon-only controls and dialog close buttons, Enter on the primary button, row activation with Enter and Space (#3587, [#3585])
• Answer an unknown id on a details page with the list, and keep list pages and admin list APIs inside the result window (#3586, [#3572])
• Follow runtime changes of system properties that were read once at startup (#3541, [#3542])
• Resolve validation messages in the locale of each request, and add the labels the admin form validation messages name (#3589, [#3584])
• Fall back to the shipped defaults for a missing property (#3338)
• Fail the index export job when the export path cannot be written, and compute the log purge cutoff in UTC (#3645, [#3635])
• Restore only the documents newer than the stored ones when restoring a backup (#3631)
• Add new bundled fields to existing config and log index mappings at startup (#3517)

API

• Answer malformed requests with 400 instead of 500: a JSON body that is not an object or not valid UTF-8, a log file id that is not base64, undecodable parameters (#3592, [#3591], [#3524])
• Answer requests the search engine cannot take with 503 instead of 500, and report an unreachable engine as service_unavailable (#3593, [#3393])
• Report a document delete that removed nothing, return the id of a created dictionary entry, and report writes to a dictionary that does not exist (#3390, [#3395], [#3376])
• Refuse a dictionary entry that repeats an existing one and report a rejected synonym or mapping entry as 400 (#3387, [#3598])
• Align the v2 OpenAPI document with what the API returns and ship it (#3398, [#3441], [#3599])
• Accept file uploads by POST on the admin API (#3494)
• Keep parallel tag creations and writes to one tag consistent (#3595, [#3588])
• Log which plugin a legacy API request needs when it is not installed (#3561)

Themes & UI

• Give each page of the bootstrap theme its own title, name icon-only buttons, and add a skip link and heading to the search screens (#3581, [#3378])
• Keep long tag names and the tag remove button usable on a phone, and lay out the My tags panel as cards (#3579, [#3653])
• Resolve the relative URLs of a cached page against its original address (#3580)
• Keep the markup administrators write in related content and notifications (#3479)
• Serve the OpenSearch description document for the running host (#3377)
• Stop password managers from flagging the default password and filling the search box (#3609)

Packaging & Operations

• Register /etc/fess configuration files as deb conffiles, replace unedited ones on the first upgrade from 15.8, reload systemd after installing, and ship NOTICE in every package (#3608, [#3627], [#3628], [#3601])
• Let dnf and rpm install the package without --nodeps, and do not require a local opensearch.service in fess.service (#3446, [#3602])
• Make bin/fess -v print the version, put the selected JAVA_HOME first on the PATH for child JVMs, and export a UTF-8 locale to the JVM (#3606, [#3604], [#3605])
• Support ImageMagick 7 and Windows for thumbnail generation (#3611)
• Bound the Tomcat JUL log size (#3355)
• Report misconfigurations once and say what to do: missing script engines, plugin jars installed twice, a missing storage plugin, non-numeric load control settings (#3451, [#3622], [#3623], [#3642])

Dependencies & Footprint

• Drop 13.9 MiB of runtime dependencies and dead font formats, five unused runtime jars, the ICU4J dependency, the Nashorn engine and the gRPC transport (#3438, [#3443], [#3444], [#3416], [#3415])
• Compile against the forked OpenSearch package and drop the opensearch jar (#3439)

Bug Fixes

• A document could not be found by id when virtual hosts were part of its id during crawling (#3654)
• bin/fess -v did not print the version, and the selected JAVA_HOME was not used by child JVMs (#3606, [#3604])
• The deb postinst did not quote the upgrade argument (#3600)
• Pressing Enter in an admin form field did not press the primary button (#3585)
• The administration error page could not be rendered, and the system error page could redirect in a loop (#3382, [#3337])
• /api/v2/documents/favorite answered 500 instead of not_found (#3389)
• Dictionary list endpoints reported the wrong record count, and kuromoji entries were not checked on the admin API (#3362, [#3361])
• The backup API wrote a bulk file without the concrete index name (#3366)
• A favorite posted right after the first search could not find its user (#3590)
• Search degraded incorrectly when user info could not be loaded, and the theme requested popular words when the server disables them (#3500, [#3501])
• Search words that failed to reach the suggest index were dropped silently (#3499)
• Notification mail failed with the bundled mail library (#3454)

We recommend upgrading to Fess 15.9.0 for the new default static theme, search analytics and reporting, rank fusion and RAG chat improvements, and the many correctness and security fixes. Review the breaking changes above and the upgrade guide before upgrading an existing installation.

:scroll: Documentation :package: Docker Image: GitHub Packages - codelibs/fess :speech_balloon: Community Forum: discuss.codelibs.org

Thank you for using Fess!