BudouX Files

Important Changes

• Japanese model is updated. You may see different segmentation results to the previous version
• Model training pipeline is fully updated.

What's Changed

• 🔒 [security fix] Validate version in bump_version.py by @tushuhei in https://github.com/google/budoux/pull/1241
• ⚡ [Java] Optimize Parser by caching feature maps by @tushuhei in https://github.com/google/budoux/pull/1240
• ⚡ [TypeScript] Optimize Parser by caching feature maps by @tushuhei in https://github.com/google/budoux/pull/1239
• ⚡ [Python] Optimize Parser by caching feature maps by @tushuhei in https://github.com/google/budoux/pull/1238
• Bump to python 3.12 for mypy check by @tushuhei in https://github.com/google/budoux/pull/1261
• refactor: deprecate partial fine-tuning across training workflow by @tushuhei in https://github.com/google/budoux/pull/1278
• feat(scripts): add standardized compiled model evaluation benchmark tool by @tushuhei in https://github.com/google/budoux/pull/1285
• Add agentic training data synthesizer by @tushuhei in https://github.com/google/budoux/pull/1287
• Refactor: Deduplicate dev dependencies in pyproject.toml by @tushuhei in https://github.com/google/budoux/pull/1288
• Clean demo dependencies by @tushuhei in https://github.com/google/budoux/pull/1290
• fix(eval): fix TSV parsing in evaluate_model.py and add unit tests by @tushuhei in https://github.com/google/budoux/pull/1289
• refactor(scripts): add end-to-end training pipeline by @tushuhei in https://github.com/google/budoux/pull/1305
• Migrate from mypy to pyrefly by @tushuhei in https://github.com/google/budoux/pull/1306
• feat(model): update Japanese model (ja.json) by @tushuhei in https://github.com/google/budoux/pull/1308
• feat(model): update Japanese model weights (ja.json) for issue [#272] by @tushuhei in https://github.com/google/budoux/pull/1310
• Colab CLI support by @tushuhei in https://github.com/google/budoux/pull/1309
• refactor(scripts): unify model evaluation and fix EPS precision bug by @tushuhei in https://github.com/google/budoux/pull/1315

Full Changelog: https://github.com/google/budoux/compare/v0.8.4...v0.9.0