Metarank Files

Features

• Paginate S3TrainStore.listKeys so train reads beyond 1000 parts by @Michael-1 in https://github.com/metarank/metarank/pull/1678
• add /flush api, fix double-impression issue by @shuttie in https://github.com/metarank/metarank/pull/1677
• Add support for HF retries on 429 by @shuttie in https://github.com/metarank/metarank/pull/1693
• Deduplicate train events by ID when reading S3 parts by @Michael-1 in https://github.com/metarank/metarank/pull/1682
• Add cadence and user selectors by @Michael-1 in https://github.com/metarank/metarank/pull/1685
• Run train-part encoding and compression on the blocking pool by @Michael-1 in https://github.com/metarank/metarank/pull/1686
• abstract out mutable stuff into DeduplicateByKey by @shuttie in https://github.com/metarank/metarank/pull/1694
• Read S3 train parts in parallel, configurable via readConcurrency by @Michael-1 in https://github.com/metarank/metarank/pull/1692
• add explicit ordering for concurrent download by @shuttie in https://github.com/metarank/metarank/pull/1696

Bugfixes

• fix crash on --version cli arg by @shuttie in https://github.com/metarank/metarank/pull/1674
• bump ltrlib to 0.3.0 to fix crash on close by @shuttie in https://github.com/metarank/metarank/pull/1695

Build

• Migrate to Scala3 syntax by @shuttie in https://github.com/metarank/metarank/pull/1666
• remove -Werror from sbt by @shuttie in https://github.com/metarank/metarank/pull/1673
• fix a bunch of sbt build warnings by @shuttie in https://github.com/metarank/metarank/pull/1675

Docs

• update docs about impression event being a system one by @shuttie in https://github.com/metarank/metarank/pull/1676

Deps

• Update pulsar-client, pulsar-client-admin to 4.2.4 by @scala-steward in https://github.com/metarank/metarank/pull/1668
• Update scalacheck-1-18 to 3.2.20.0 by @scala-steward in https://github.com/metarank/metarank/pull/1670
• Update shapeless3-typeable to 3.6.0 by @scala-steward in https://github.com/metarank/metarank/pull/1671
• Update kinesis, s3, sts to 2.54.3 by @scala-steward in https://github.com/metarank/metarank/pull/1672
• Update scala3-library to 3.8.4 by @scala-steward in https://github.com/metarank/metarank/pull/1669
• bump lettuce to 7.x by @shuttie in https://github.com/metarank/metarank/pull/1683
• Update sbt-assembly to 2.5.0 by @scala-steward in https://github.com/metarank/metarank/pull/1679
• Update sbt to 2.0.8 by @scala-steward in https://github.com/metarank/metarank/pull/1684
• Update sentry-logback to 8.55.0 by @scala-steward in https://github.com/metarank/metarank/pull/1688
• Update fs2-core, fs2-io to 3.14.0 by @scala-steward in https://github.com/metarank/metarank/pull/1691
• Update scala3-library to 3.9.0 by @scala-steward in https://github.com/metarank/metarank/pull/1689
• Update jcl-over-slf4j to 2.0.19 by @scala-steward in https://github.com/metarank/metarank/pull/1690
• Update s3, sts to 2.54.5 by @scala-steward in https://github.com/metarank/metarank/pull/1681

Full Changelog: https://github.com/metarank/metarank/compare/0.8.0...0.8.1