Magda Files

What's Changed

• [#3598]: Add configurable on_disk KNN vector workload mode for OpenSearch (foundation for semantic indexing); upgrade bundled OpenSearch and OpenSearch Dashboards from 2.17.1 to 2.19.1 by @JHZhang2736 in https://github.com/magda-io/magda/pull/3598
• [#3618]: Add reusable Semantic Indexing Minion SDK for building LLM-based vector-indexing minions (chunking, embedding, OpenSearch indexing), implementing the generic semantic index data structure & OpenSearch mapping design (#3536) in https://github.com/magda-io/magda/pull/3618
• [#3619]: Add new semantic-search-api microservice with /search and /retrieve endpoints, gateway routing and Helm chart by @JHZhang2736 in https://github.com/magda-io/magda/pull/3619
• [#3629]: Semantic indexer framework: include record ID in skip log messages in https://github.com/magda-io/magda/pull/3629
• [#3635]: Add filterByAccess Registry API endpoint — takes a list of record IDs and an access operation (e.g. object/record/read) and returns only the records the current user can access (shipped via [#3643])
• [#3638]: Harden semantic indexer autoDownload handling to avoid crashes on malformed compressed responses in https://github.com/magda-io/magda/pull/3638
• [#3643]: Add access control to the semantic index query flow — filter semantic search results by the current user's read access, falling back to access-scoped retrieval when no authorised results are found (fixes [#3608]) by @chensuihui in https://github.com/magda-io/magda/pull/3643
• [#3643]: Harden the Node.js builder image's Helm install step with multi-arch support (arm64/amd64) and retry + mirror fallback for get.helm.sh by @chensuihui in https://github.com/magda-io/magda/pull/3643
• [#3643]: Add magda-pdf-semantic-indexer and magda-csv-semantic-indexer as default magda chart dependencies, adding content-specific semantic indexing for PDF and CSV/tabular distributions (per [#3608]) by @chensuihui in https://github.com/magda-io/magda/pull/3643
• [#3663]: Fix global.searchEngine.semanticIndexer defaults missing from magda-core chart, causing a nil pointer when templating semantic-search-api standalone in https://github.com/magda-io/magda/pull/3663
• [#3664]: Fix gateway CrashLoopBackOff caused by null CSP directives (e.g. defaultSrc: null) under helmet v8 in https://github.com/magda-io/magda/pull/3664
• [#3665]: Fix NPM package publishing to use OIDC trusted publishing and fail loudly on real publish errors; bump the Node.js builder image's npm from 10.5.2 to 11.5.1 in https://github.com/magda-io/magda/pull/3665

New Contributors

• @chensuihui made their first contribution in https://github.com/magda-io/magda/pull/3643

Full Changelog: https://github.com/magda-io/magda/compare/v5.6.1...v6.0.0